A data visualization chart processing method and device
By acquiring the feature vectors of data visualization charts and matching them with declarative syntax, the problems of image-formatted charts being unmodifiable and semantic analysis issues were resolved, enabling richer processing capabilities.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- BEIJING VOLCANO ENGINE TECH CO LTD
- Filing Date
- 2023-12-08
- Publication Date
- 2026-04-21
Smart Images
Figure CN117688068B_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of computer technology, and in particular to a method and apparatus for processing data visualization charts. Background Technology
[0002] Data visualization charts can transform abstract chart data into intuitive charts, making it easier for users to understand the data.
[0003] Currently, while image-formatted data visualization charts can be used to display data visualizations as images—for example, within a document—they do not support more complex processing operations. For instance, when chart data changes, modifications cannot be made to the currently image-formatted chart; instead, a chart drawing tool is needed to redraw the chart. Furthermore, semantic analysis of the aforementioned document cannot be performed on the image-formatted data visualization chart.
[0004] Therefore, there is an urgent need for a solution to address the above problems. Summary of the Invention
[0005] To solve, or at least partially solve, the above-mentioned technical problems, embodiments of this application provide a method and apparatus for processing data visualization charts.
[0006] In a first aspect, embodiments of this application provide a method for processing data visualization charts, the method comprising:
[0007] Obtain image-formatted visualization charts of the data to be processed;
[0008] Determine a first feature vector for the data visualization chart to be processed, wherein the first feature vector is the same as the feature vector of the declarative syntax corresponding to the data visualization chart to be processed.
[0009] A second feature vector with a matching degree higher than a certain threshold is determined from a vector database. The vector database includes: multiple feature vectors and a declarative syntax corresponding to each of the multiple feature vectors, and the multiple feature vectors include the second feature vector.
[0010] Based on the declarative syntax corresponding to the second feature vector in the vector database, the declarative syntax corresponding to the visualization chart of the data to be processed is determined.
[0011] Optionally, the vector database is constructed in the following manner:
[0012] Obtain historical data visualization charts and the declarative syntax corresponding to each historical data visualization chart, wherein the historical data visualization charts are in image format;
[0013] For any one of the multiple historical data visualization charts, perform the following operation:
[0014] Determine the feature vector of the declarative syntax corresponding to the historical data visualization chart;
[0015] The declarative syntax corresponding to the historical data visualization chart, and the feature vector of the declarative syntax corresponding to the historical data visualization chart, are stored in the vector database.
[0016] Optionally, determining the feature vector of the declarative syntax corresponding to the historical data visualization chart includes:
[0017] Extract key information from the declarative syntax corresponding to the historical data visualization charts to obtain a simplified declarative syntax, which is a structured declarative syntax.
[0018] The simplified declarative syntax is converted into a string-format declarative syntax, and the feature vector of the declarative syntax corresponding to the historical data visualization chart is obtained based on the string-format declarative syntax.
[0019] Optionally, the key information includes:
[0020] Other information besides the chart interaction information, which is the information displayed in response to an interactive operation triggered by a data visualization chart.
[0021] Optionally, determining the first feature vector of the data visualization chart to be processed includes:
[0022] The data visualization chart to be processed is input into the chart processing model to obtain the first feature vector. The chart processing model is used to determine the feature vector corresponding to the input data visualization chart. The chart processing model is trained in the following manner:
[0023] For any one of the multiple historical data visualization charts, perform the following operation:
[0024] The feature vector of the declarative syntax corresponding to the historical data visualization chart is used as the label of the historical data visualization chart. The chart processing model is trained using the historical data visualization chart and the label of the historical data visualization chart, wherein: the label of the historical data visualization chart is used to indicate the feature vector of the historical data visualization chart.
[0025] Optionally, training the chart processing model using the historical data visualization chart and its labels includes:
[0026] The pre-trained model is fine-tuned using the historical data visualization charts and their labels, wherein the pre-trained model has the ability to understand the semantic relationship between images and image descriptions.
[0027] Optionally, a second feature vector with a matching degree higher than a certain threshold is determined from the vector database, including:
[0028] Determine at least one second feature vector from the vector database that matches the first feature vector;
[0029] The step of determining the declarative syntax corresponding to the data visualization chart to be processed based on the declarative syntax corresponding to the second feature vector in the vector database includes:
[0030] The declarative syntax corresponding to the target feature vector in the vector database is determined as the declarative syntax corresponding to the data visualization chart to be processed, wherein: the target feature vector is the top N second feature vectors after sorting the at least one second feature vector in descending order of matching degree with the first feature vector, and N is a positive integer.
[0031] Secondly, embodiments of this application provide a data visualization chart processing apparatus, the apparatus comprising:
[0032] The acquisition unit is used to acquire image-formatted visualization charts of the data to be processed.
[0033] The first determining unit is used to determine the first feature vector of the data visualization chart to be processed, wherein the first feature vector is the same as the feature vector of the declarative syntax corresponding to the data visualization chart to be processed.
[0034] The second determining unit is used to determine a second feature vector from a vector database whose matching degree with the first feature vector is higher than a certain threshold. The vector database includes: multiple feature vectors and a declarative syntax corresponding to each of the multiple feature vectors, and the multiple feature vectors include the second feature vector.
[0035] The third determining unit is used to determine the declarative syntax corresponding to the data visualization chart to be processed based on the declarative syntax corresponding to the second feature vector in the vector database.
[0036] Optionally, the vector database is constructed in the following manner:
[0037] Obtain historical data visualization charts and the declarative syntax corresponding to each historical data visualization chart, wherein the historical data visualization charts are in image format;
[0038] For any one of the multiple historical data visualization charts, perform the following operation:
[0039] Determine the feature vector of the declarative syntax corresponding to the historical data visualization chart;
[0040] The declarative syntax corresponding to the historical data visualization chart, and the feature vector of the declarative syntax corresponding to the historical data visualization chart, are stored in the vector database.
[0041] Optionally, determining the feature vector of the declarative syntax corresponding to the historical data visualization chart includes:
[0042] Extract key information from the declarative syntax corresponding to the historical data visualization charts to obtain a simplified declarative syntax, which is a structured declarative syntax.
[0043] The simplified declarative syntax is converted into a string-format declarative syntax, and the feature vector of the declarative syntax corresponding to the historical data visualization chart is obtained based on the string-format declarative syntax.
[0044] Optionally, the key information includes:
[0045] Other information besides the chart interaction information, which is the information displayed in response to an interactive operation triggered by a data visualization chart.
[0046] Optionally, the first determining unit is configured to:
[0047] The data visualization chart to be processed is input into the chart processing model to obtain the first feature vector. The chart processing model is used to determine the feature vector corresponding to the input data visualization chart. The chart processing model is trained in the following manner:
[0048] For any one of the multiple historical data visualization charts, perform the following operation:
[0049] The feature vector of the declarative syntax corresponding to the historical data visualization chart is used as the label of the historical data visualization chart. The chart processing model is trained using the historical data visualization chart and the label of the historical data visualization chart, wherein: the label of the historical data visualization chart is used to indicate the feature vector of the historical data visualization chart.
[0050] Optionally, training the chart processing model using the historical data visualization chart and its labels includes:
[0051] The pre-trained model is fine-tuned using the historical data visualization charts and their labels, wherein the pre-trained model has the ability to understand the semantic relationship between images and image descriptions.
[0052] Optionally, the second determining unit is used for:
[0053] Determine at least one second feature vector from the vector database that matches the first feature vector;
[0054] The step of determining the declarative syntax corresponding to the data visualization chart to be processed based on the declarative syntax corresponding to the second feature vector in the vector database includes:
[0055] The declarative syntax corresponding to the target feature vector in the vector database is determined as the declarative syntax corresponding to the data visualization chart to be processed, wherein: the target feature vector is the top N second feature vectors after sorting the at least one second feature vector in descending order of matching degree with the first feature vector, and N is a positive integer.
[0056] Thirdly, embodiments of this application provide an electronic device, the device including a processor and a memory;
[0057] The processor is configured to execute instructions stored in the memory to cause the device to perform the method as described in any one of the first aspects above.
[0058] Fourthly, embodiments of this application provide a computer-readable storage medium including instructions that instruct a device to perform the method as described in any one of the first aspects above.
[0059] Fifthly, embodiments of this application provide a computer program product that, when run on a computer, causes the computer to perform the method described in any of the first aspects above.
[0060] Compared with the prior art, the embodiments of this application have the following advantages:
[0061] This application provides a method for processing data visualization charts. The method includes: acquiring a data visualization chart in image format; and then determining a first feature vector corresponding to the data visualization chart. In this application, the first feature vector is the same as the feature vector of the declarative syntax corresponding to the data visualization chart; in other words, the first feature vector is also the feature vector of the declarative syntax corresponding to the data visualization chart. After determining the first feature vector, the declarative syntax corresponding to the data visualization chart can be determined using a vector database and the first feature vector. The vector database includes multiple feature vectors and the declarative syntax corresponding to each feature vector. As a specific example, a second feature vector with a matching degree higher than a certain threshold can be determined from the vector database. Since the first feature vector is the feature vector of the declarative syntax corresponding to the data visualization chart, the determined second feature vector is the feature vector of the declarative syntax corresponding to the data visualization chart. After determining the second feature vector, the declarative syntax corresponding to the second feature vector can be obtained from the vector database, and the declarative syntax corresponding to the data visualization chart can be determined based on the declarative syntax corresponding to the second feature vector. Therefore, by using the solution of this application embodiment, the declarative syntax corresponding to the data visualization chart to be processed in image format can be determined. In this way, when it is necessary to trigger relevant operations for the data visualization chart to be processed, the declarative syntax corresponding to the data visualization chart to be processed can be processed. Attached Figure Description
[0062] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0063] Figure 1 A schematic diagram of a data visualization chart and its corresponding declarative syntax provided for an embodiment of this application;
[0064] Figure 2 A flowchart illustrating a data visualization chart processing method provided in an embodiment of this application;
[0065] Figure 3A flowchart illustrating a method for constructing a vector database provided in an embodiment of this application;
[0066] Figure 4 A flowchart illustrating a training method for a graph processing model provided in an embodiment of this application;
[0067] Figure 5 A schematic diagram illustrating the training process of a graph processing model provided in an embodiment of this application;
[0068] Figure 6 A schematic diagram illustrating a chart processing method provided in an embodiment of this application;
[0069] Figure 7 This is a schematic diagram of the structure of a data visualization chart processing device provided in an embodiment of this application. Detailed Implementation
[0070] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present application.
[0071] The inventors of this application discovered through research that data visualization charts stored in image format cannot support more complex processing operations. For example, when chart data changes, modifications cannot be made to the current image-formatted data visualization chart; instead, a chart drawing tool is needed to redraw the chart. Furthermore, when performing semantic analysis on the aforementioned document, semantic analysis cannot be performed on the data visualization charts displayed in image format.
[0072] If data visualization charts stored in image format can be converted into corresponding declarative syntax, then when related operations need to be triggered on the data visualization charts, the declarative syntax can be processed. For example, when the chart data changes, the relevant data in the declarative syntax can be modified. Similarly, when performing semantic analysis on documents containing data visualization charts, the declarative syntax corresponding to the data visualization charts can be semantically analyzed.
[0073] Regarding the declarative syntax for data visualization charts, you can combine... Figure 1 To understand, Figure 1 This is a schematic diagram of a data visualization chart and its corresponding declarative syntax provided in an embodiment of this application. Figure 1 The left side shows a data visualization chart, and the right side shows the declarative syntax corresponding to that data visualization chart.
[0074] Determining the declarative syntax for data visualization charts stored in image formats is a problem that remains to be solved.
[0075] In view of this, embodiments of this application provide a method for processing data visualization charts, which can determine the declarative syntax corresponding to the data visualization chart to be processed in image format.
[0076] The various non-limiting embodiments of this application will now be described in detail with reference to the accompanying drawings.
[0077] Exemplary methods
[0078] See Figure 2 This figure is a schematic flowchart illustrating a data visualization chart processing method provided in an embodiment of this application. In this embodiment, the method may include, for example, the following steps: S101-S104.
[0079] S101: Obtain image-formatted visualization charts of the data to be processed.
[0080] This application does not specifically limit the channel for obtaining the image-formatted data visualization chart to be processed. In one example, the data visualization chart to be processed may be a data visualization chart included in a document. In this case, the data visualization chart to be processed can be obtained from that document. In yet another example, the data visualization chart to be processed may be a data visualization chart included in other channels, such as a webpage. In this case, the data visualization chart to be processed can be obtained from that webpage.
[0081] S102: Determine the first feature vector of the data visualization chart to be processed, wherein the first feature vector is the same as the feature vector of the declarative syntax corresponding to the data visualization chart to be processed.
[0082] After obtaining the data visualization chart to be processed, it can be further processed to obtain the first feature vector of the data visualization chart. In this embodiment, the first feature vector of the data visualization chart to be processed is the same as the feature vector of the declarative syntax corresponding to the data visualization chart to be processed. In other words, the first feature vector is not only the vector corresponding to the data visualization chart to be processed, but also the feature vector corresponding to the data visualization chart to be processed.
[0083] This application does not specifically limit the specific implementation of obtaining the first effect vector from the visualization chart of the data to be processed. In one example, a specific algorithm can be used to calculate the first feature vector from the visualization chart of the data to be processed.
[0084] In another example, a chart processing model can be used to process the data visualization chart to obtain the first feature vector. This chart processing model can obtain the feature vector corresponding to the input image-formatted data visualization chart. Furthermore, this feature vector can serve not only as the feature vector of the data visualization chart itself, but also as the feature vector of the declarative syntax corresponding to the data visualization chart. As a concrete example, the image-formatted data visualization chart to be processed can be input into the chart processing model, which can then process the image-formatted data visualization chart and output the first feature vector.
[0085] This chart processing model can be pre-trained. For details on how to train this model, please refer to the following section. Figure 4 The description section is not described in detail here.
[0086] After determining the first feature vector, the declarative syntax corresponding to the data visualization chart to be processed can be determined by combining the vector database and the first feature vector. Specifically, the following steps S103-S104 can be executed to determine the declarative syntax corresponding to the data visualization chart to be processed.
[0087] S103: Determine a second feature vector from the vector database whose matching degree with the first feature vector is higher than a certain threshold. The vector database includes: multiple feature vectors and a declarative syntax corresponding to each of the multiple feature vectors, and the multiple feature vectors include the second feature vector.
[0088] S104: Determine the declarative syntax corresponding to the data visualization chart to be processed based on the declarative syntax corresponding to the second effect vector in the vector database.
[0089] In this embodiment, the vector database may include multiple feature vectors and the declarative syntax corresponding to each feature vector. As mentioned earlier, the first feature vector is not only the feature vector of the data visualization chart to be processed, but also the feature vector of the declarative syntax corresponding to the data visualization chart. Therefore, the first feature vector can be matched with the feature vectors in the vector database to obtain a second feature vector with a matching degree higher than a certain threshold. It is easy to understand that the aforementioned multiple feature vectors include the second feature vector.
[0090] In one example, the matching degree between two feature vectors can be measured by vector similarity or vector distance. Where:
[0091] For any two feature vectors, the higher the vector similarity between them, the higher the matching degree. When the matching degree is measured by vector similarity, the matching degree between the first feature vector and the second feature vector is higher than a certain threshold, which can be the vector similarity between the first feature vector and the second feature vector being higher than a certain threshold (e.g., 0.8).
[0092] For any two feature vectors, the smaller the vector distance between them, the higher the matching degree. When the matching degree is measured by vector distance, the matching degree between the first feature vector and the second feature vector is higher than a certain threshold, which can be that the vector distance between the first feature vector and the second feature vector is less than a preset distance (e.g., 0.8).
[0093] The calculation methods for vector similarity and vector distance can be performed using traditional methods, which will not be explained in detail here.
[0094] Since the first feature vector is the feature vector of the declarative syntax corresponding to the data visualization chart to be processed, the determined second feature vector is also the feature vector of the declarative syntax corresponding to the data visualization chart to be processed. Therefore, further, the declarative syntax corresponding to the data visualization chart to be processed can be determined based on the declarative syntax corresponding to the second feature vector in the vector database.
[0095] In this embodiment of the application, after matching the first feature vector with the feature vectors included in the vector database, at least one second feature vector with a matching degree higher than a certain threshold can be obtained.
[0096] In one example, in the specific implementation of S104, the declarative syntax corresponding to each of the at least one second feature vectors can be obtained from the vector database, and the declarative syntax corresponding to each of the at least one second feature vectors can be determined as the declarative syntax corresponding to the data visualization chart to be processed. Using this method, the user can select one from the declarative syntax corresponding to each of the aforementioned at least one second feature vectors as the target declarative syntax corresponding to the data visualization chart to be processed.
[0097] In another example, in the specific implementation of S104, the at least one second feature vector can be sorted in descending order of matching degree with the first feature vector, and the top N second feature vectors in the sorting are determined as target feature vectors. Furthermore, the declarative syntax corresponding to the target feature vector is obtained from the vector database, and the declarative syntax corresponding to the target feature vector is determined as the declarative syntax corresponding to the data visualization chart to be processed.
[0098] In the embodiments of this application, N is a positive integer.
[0099] When N is 1, the target feature vector is the second feature vector in the vector database that has the highest matching degree with the first feature vector and whose matching degree with the first feature vector is higher than a certain threshold.
[0100] When N is greater than 1, the target feature vector includes multiple second feature vectors whose matching degree with the first feature vector is higher than a certain threshold. In this case, after determining the declarative syntax corresponding to the target feature vector as the declarative syntax corresponding to the data visualization chart to be processed, the user can select one declarative syntax from the declarative syntax corresponding to the multiple second feature vectors as the target declarative syntax corresponding to the data visualization chart to be processed.
[0101] As can be seen from the above description, by using the solution of the embodiments of this application, the declarative syntax corresponding to the data visualization chart to be processed in image format can be determined. In this way, when it is necessary to trigger relevant operations for the data visualization chart to be processed, the declarative syntax corresponding to the data visualization chart to be processed can be processed.
[0102] In this embodiment of the application, the aforementioned vector database may be pre-built. The construction method of this vector database will be described below.
[0103] See Figure 3 The figure is a flowchart illustrating a method for constructing a vector database according to an embodiment of this application. Figure 3 The method shown may include the following steps S201-S203.
[0104] S201: Obtain multiple historical data visualization charts and the declarative syntax corresponding to each of the multiple historical data visualization charts, wherein the historical data visualization charts are in image format.
[0105] In one example, a platform that provides image-formatted data visualization charts and feature vectors of the corresponding declarative syntax for the data visualization charts can be used to obtain multiple historical data visualization charts and the declarative syntax corresponding to each of the multiple historical data visualization charts. This application does not specifically limit the platform; the platform can be any existing or future platform.
[0106] In this embodiment of the application, the declarative syntax corresponding to each historical data visualization chart is such as Figure 1The structured declarative syntax shown on the right. The declarative syntax corresponding to historical data visualization charts includes the main information of the historical visualization chart, such as: chart type, names of the chart's axes (horizontal and vertical axes), specific data, and some style information, etc.
[0107] In one example, to broaden the applicability of the vector database, the multiple historical data visualization charts can cover as many types of data visualization charts as possible. Furthermore, the number of different similar data visualization charts is kept as balanced as possible.
[0108] For any of the multiple historical data visualization charts, execute the following steps S202-S203.
[0109] S202: Determine the feature vector of the declarative syntax corresponding to the historical data visualization chart.
[0110] In one example, the declarative syntax corresponding to the historical data visualization chart can be processed using a language model to obtain a feature vector of the declarative syntax corresponding to the historical data visualization chart. For instance, the declarative syntax corresponding to the historical data visualization chart can be input into the language model, which processes the declarative syntax and outputs a feature vector of the declarative syntax corresponding to the historical data visualization chart.
[0111] In yet another example, S202 may include the following steps A1-A2 in its specific implementation.
[0112] A1: Extract key information from the declarative syntax corresponding to the historical data visualization charts to obtain a simplified declarative syntax, which is a structured declarative syntax.
[0113] In this embodiment, the key information can be determined according to actual needs. In one example, considering that the important information for a chart is its type and the data displayed, chart interaction information is not very important, the key information may include information other than chart interaction information in one example. The chart interaction information mentioned here refers to the information displayed in the data visualization chart in response to a user's interactive action. The chart interaction information may be, for example, hover indication information, where hover indication information refers to the prompt information displayed after a user triggers a hover action in the data visualization chart.
[0114] In this embodiment, after extracting key information from the declarative syntax corresponding to the historical data visualization chart, a simplified declarative syntax can be obtained. It is easy to understand that the simplified declarative syntax is obtained by removing the aforementioned chart interaction information from the declarative syntax corresponding to the historical data visualization chart. As mentioned above, the declarative syntax corresponding to the historical data visualization chart is a structured declarative syntax; correspondingly, the simplified declarative syntax is also a structured declarative syntax.
[0115] A2: Convert the simplified declarative syntax into a string-format declarative syntax, and obtain the feature vector of the declarative syntax corresponding to the historical data visualization chart based on the string-format declarative syntax.
[0116] After obtaining the simplified declarative grammar, the feature vector of the declarative grammar corresponding to the historical data visualization chart can be obtained based on the simplified declarative grammar. For example, the simplified declarative grammar can be input into a language model, which can process the simplified declarative grammar and output the feature vector of the declarative grammar corresponding to the historical data visualization chart.
[0117] In one example, considering that the structural information included in a structured declarative grammar contains very little semantic information, after obtaining the simplified declarative grammar, it can be converted into a string-format declarative grammar. The string-format declarative grammar retains the semantic information of the aforementioned structured declarative grammar. Specifically, the various substructures in the simplified declarative grammar can be traversed, and the information included in each substructure can be converted into sentences expressed in natural language, thus obtaining the string-format declarative grammar. For example, for the declarative grammar of a chart attribute, the attribute name is first extracted, then the data attribute value type (such as numeric, text, etc.) is concatenated, and finally integrated into a long text. It is easy to understand that the string-format declarative grammar no longer includes the aforementioned structural information.
[0118] After obtaining the declarative grammar in string format, the feature vector of the declarative grammar corresponding to the historical data visualization chart can be obtained using the string format declarative grammar. For example, the string format declarative grammar can be input into a language model, which can then process the string format declarative grammar and output the feature vector of the declarative grammar corresponding to the historical data visualization chart. Another advantage of this approach is that the string format declarative grammar is easier for the language model to understand compared to the structured declarative grammar.
[0119] S203: Store the declarative syntax corresponding to the historical data visualization chart and the feature vector of the target declarative syntax corresponding to the historical data visualization chart into the vector database.
[0120] After determining the feature vector of the target declarative syntax corresponding to the historical data visualization chart, the declarative syntax corresponding to the historical data visualization chart and the feature vector of the target declarative syntax corresponding to the historical data visualization chart can be stored in the vector database.
[0121] After executing S202-S203 for any one of the multiple historical data visualization charts, the vector database is obtained.
[0122] As mentioned earlier, the graph processing model can be pre-trained. Next, we will combine... Figure 4 The training method of the aforementioned chart processing model will be introduced. Figure 4 This is a flowchart illustrating a training method for a graph processing model provided in an embodiment of this application. Figure 4 The method shown may include the following steps S301-S302.
[0123] In this embodiment of the application, S301-S302 can be executed for any of the historical data visualization charts in the aforementioned S201.
[0124] S301: Obtain the historical data visualization chart and the feature vector of the declarative syntax corresponding to the historical data visualization chart.
[0125] For details on how to obtain the historical data visualization charts, please refer to the relevant description in S201 above; it will not be repeated here.
[0126] The feature vectors of the declarative syntax corresponding to the historical data visualization charts mentioned here can be the feature vectors determined in S202 above.
[0127] S302: Use the feature vector of the declarative syntax corresponding to the historical data visualization chart as the label of the historical data visualization chart, and train the chart processing model using the historical data visualization chart and the label of the historical data visualization chart, wherein: the label of the historical data visualization chart is used to indicate the feature vector of the historical data visualization chart.
[0128] In this embodiment of the application, in order to ensure that the first feature vector obtained after the chart processing model processes the data visualization chart to be processed can serve as both the feature vector of the data visualization chart to be processed and the feature vector of the declarative syntax corresponding to the data visualization chart to be processed, when training the chart processing model, the feature vector of the declarative syntax corresponding to the historical data visualization chart can be used as the label of the historical data visualization chart. The label of the historical data visualization chart is used to indicate the feature vector of the historical data visualization chart. Then, the chart processing model is trained using the historical data visualization chart and the label of the historical data visualization chart.
[0129] It should be noted that training the chart processing model using the historical data visualization charts and their labels can be understood as updating the model parameters of the chart processing model using the historical data visualization charts and their labels. Specifically, the chart processing model can process the historical data visualization charts to obtain corresponding prediction results. These prediction results are the feature vectors predicted by the current chart processing model for the historical data visualization charts. Then, based on the prediction results and the labels, a loss function is calculated, and further, the model parameters of the chart processing model are updated based on the loss function. The smaller the value of the loss function, the closer the prediction results and labels are, and correspondingly, the closer the prediction results are to the standard value.
[0130] In one example, the loss function can be the mean squared error of the aforementioned prediction results and labels. See formula (1).
[0131]
[0132] In formula (1):
[0133] loss is the loss function;
[0134] n represents the number of historical data visualization charts used for training in a given training round;
[0135] a i Labels for historical data visualization charts;
[0136] b iThe chart processing model processes the historical data visualization chart to obtain the prediction result. As described above, when training the chart processing model, the feature vector of the declarative syntax corresponding to the historical data visualization chart is used as the standard value of the feature vector of the historical data visualization chart. Thus, after continuously adjusting the parameters of the chart processing model, the loss in formula (1) approaches 0, i.e.: a i and b i They are almost identical. Therefore, the first feature vector obtained by the chart processing model after training can also be used as the feature vector of the declarative syntax corresponding to the chart to be processed.
[0137] In one example, the graph processing model can be trained on a base model that does not have any data processing capabilities.
[0138] In another example, to improve the efficiency of training the chart processing model, it can be trained based on the pre-trained model. Specifically, the chart processing model can be trained based on a pre-trained model that has the ability to understand the semantic relationship between images and image descriptions. In practice, "training the chart processing model based on the pre-trained model" can be achieved by fine-tuning the pre-trained model using historical data visualization charts and their labels to obtain the chart processing model. In a specific example, an additional network layer can be added to the later layers of the pre-trained model. This network layer maps the feature vectors output by the pre-trained model to another vector space to obtain the aforementioned prediction result.
[0139] The training process of the aforementioned chart processing model can be combined with... Figure 5 To understand, Figure 5 This is a schematic diagram illustrating the training process of a graph processing model provided in an embodiment of this application.
[0140] like Figure 5 As shown, for historical data visualization charts and their corresponding declarative syntax, the declarative syntax can correspond to a feature vector a, which serves as the label for the historical data visualization chart.
[0141] After inputting the historical data visualization chart into the chart processing model, the chart processing model can predict the feature vector b for the historical data visualization chart.
[0142] A loss function is calculated based on the aforementioned feature vectors a and b, and the parameters of the graph processing model are updated based on this loss function. The parameters of the graph processing model mentioned here may include the parameters of the aforementioned network layers. In one example, the parameters of the graph processing model may also include the parameters of the pre-trained model.
[0143] The chart processing method provided in the embodiments of this application has been described above. Next, a possible implementation of the embodiments of this application will be introduced in conjunction with a specific scenario.
[0144] See Figure 6 This figure is a schematic diagram of a chart processing method provided in an embodiment of this application. Figure 6 As shown:
[0145] For a data visualization chart to be processed, the data visualization chart can be input into the chart processing model, which can output the first feature vector of the data visualization chart.
[0146] Then, the first feature vector is matched with the vector database. For example... Figure 6 The vector database includes corresponding stored feature vectors and declarative syntax.
[0147] Furthermore, if the first feature vector matches a second feature vector in the vector database, the declarative syntax corresponding to the second feature vector is retrieved from the vector database, and this declarative syntax is determined as the declarative syntax corresponding to the data visualization chart to be processed. Here, the first feature vector matching the second feature vector in the vector database can be achieved when the matching degree between the first and second feature vectors is higher than a certain threshold.
[0148] Exemplary device
[0149] Based on the methods provided in the above embodiments, this application also provides an apparatus, which will be described below with reference to the accompanying drawings.
[0150] See Figure 7 The figure is a schematic diagram of the structure of a data visualization chart processing device provided in an embodiment of this application. The device 700 may specifically include, for example, an acquisition unit 701, a first determination unit 702, a second determination unit 703, and a third determination unit 704.
[0151] Acquisition unit 701 is used to acquire image-formatted visualization charts of the data to be processed;
[0152] The first determining unit 702 is used to determine the first feature vector of the data visualization chart to be processed, wherein the first feature vector is the same as the feature vector of the declarative syntax corresponding to the data visualization chart to be processed.
[0153] The second determining unit 703 is used to determine a second feature vector from a vector database whose matching degree with the first feature vector is higher than a certain threshold. The vector database includes: multiple feature vectors and a declarative syntax corresponding to each of the multiple feature vectors, and the multiple feature vectors include the second feature vector.
[0154] The third determining unit 704 is used to determine the declarative syntax corresponding to the data visualization chart to be processed based on the declarative syntax corresponding to the second feature vector in the vector database.
[0155] Optionally, the vector database is constructed in the following manner:
[0156] Obtain historical data visualization charts and the declarative syntax corresponding to each historical data visualization chart, wherein the historical data visualization charts are in image format;
[0157] For any one of the multiple historical data visualization charts, perform the following operation:
[0158] Determine the feature vector of the declarative syntax corresponding to the historical data visualization chart;
[0159] The declarative syntax corresponding to the historical data visualization chart, and the feature vector of the declarative syntax corresponding to the historical data visualization chart, are stored in the vector database.
[0160] Optionally, determining the feature vector of the declarative syntax corresponding to the historical data visualization chart includes:
[0161] Extract key information from the declarative syntax corresponding to the historical data visualization charts to obtain a simplified declarative syntax, which is a structured declarative syntax.
[0162] The simplified declarative syntax is converted into a string-format declarative syntax, and the feature vector of the declarative syntax corresponding to the historical data visualization chart is obtained based on the string-format declarative syntax.
[0163] Optionally, the key information includes:
[0164] Other information besides the chart interaction information, which is the information displayed in response to an interactive operation triggered by a data visualization chart.
[0165] Optionally, the first determining unit 702 is used to:
[0166] The data visualization chart to be processed is input into the chart processing model to obtain the first feature vector. The chart processing model is used to determine the feature vector corresponding to the input data visualization chart. The chart processing model is trained in the following manner:
[0167] For any one of the multiple historical data visualization charts, perform the following operation:
[0168] The feature vector of the declarative syntax corresponding to the historical data visualization chart is used as the label of the historical data visualization chart. The chart processing model is trained using the historical data visualization chart and the label of the historical data visualization chart, wherein: the label of the historical data visualization chart is used to indicate the feature vector of the historical data visualization chart.
[0169] Optionally, training the chart processing model using the historical data visualization chart and its labels includes:
[0170] The pre-trained model is fine-tuned using the historical data visualization charts and their labels, wherein the pre-trained model has the ability to understand the semantic relationship between images and image descriptions.
[0171] Optionally, the second determining unit 703 is used for:
[0172] Determine at least one second feature vector from the vector database that matches the first feature vector;
[0173] The step of determining the declarative syntax corresponding to the data visualization chart to be processed based on the declarative syntax corresponding to the second feature vector in the vector database includes:
[0174] The declarative syntax corresponding to the target feature vector in the vector database is determined as the declarative syntax corresponding to the data visualization chart to be processed, wherein: the target feature vector is the top N second feature vectors after sorting the at least one second feature vector in descending order of matching degree with the first feature vector, and N is a positive integer.
[0175] Since the device 700 is a device corresponding to the data visualization chart processing method provided in the above method embodiments, the specific implementation of each unit of the device 700 is based on the same concept as in the above method embodiments. Therefore, the specific implementation of each unit of the device 700 can be referred to the relevant descriptions in the above method embodiments, and will not be repeated here.
[0176] This application also provides an electronic device, which includes a processor and a memory;
[0177] The processor is used to execute instructions stored in the memory so that the device performs the data visualization chart processing method provided in the above method embodiments.
[0178] This application provides a computer-readable storage medium including instructions that instruct a device to perform the data visualization chart processing method provided in the above method embodiments.
[0179] This application also provides a computer program product that, when run on a computer, causes the computer to execute the data visualization chart processing method provided in the above method embodiments.
[0180] Other embodiments of this application will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this application that follow the general principles of this application and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this application are indicated by the following claims.
[0181] It should be understood that this application is not limited to the precise structure described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this application is limited only by the appended claims.
[0182] The above description is only a preferred embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.
Claims
1. A method for processing data visualization charts, characterized in that, The method includes: Obtain image-formatted visualization charts of the data to be processed; Determine a first feature vector for the data visualization chart to be processed, wherein the first feature vector is the same as the feature vector of the declarative syntax corresponding to the data visualization chart to be processed. A second feature vector with a matching degree higher than a certain threshold is determined from a vector database. The vector database includes: multiple feature vectors and a declarative syntax corresponding to each of the multiple feature vectors, and the multiple feature vectors include the second feature vector. Based on the declarative syntax corresponding to the second feature vector in the vector database, determine the declarative syntax corresponding to the visualization chart of the data to be processed; The step of determining the first feature vector of the data visualization chart to be processed includes: inputting the data visualization chart to be processed into a chart processing model to obtain the first feature vector, wherein the chart processing model is used to determine the feature vector corresponding to the input data visualization chart, and the chart processing model is trained in the following way: for any historical data visualization chart among multiple historical data visualization charts, the following operation is performed: the feature vector of the declarative syntax corresponding to the historical data visualization chart is used as the label of the historical data visualization chart, and the chart processing model is trained using the historical data visualization chart and the label of the historical data visualization chart, wherein the label of the historical data visualization chart is used to indicate the feature vector of the historical data visualization chart.
2. The method according to claim 1, characterized in that, The vector database is constructed in the following manner: Obtain historical data visualization charts and the declarative syntax corresponding to each historical data visualization chart, wherein the historical data visualization charts are in image format; For any of the multiple historical data visualization charts, perform the following operations: Determine the feature vector of the declarative syntax corresponding to the historical data visualization chart; The declarative syntax corresponding to the historical data visualization chart, and the feature vector of the declarative syntax corresponding to the historical data visualization chart, are stored in the vector database.
3. The method according to claim 2, characterized in that, Determine the feature vector of the declarative syntax corresponding to the historical data visualization chart, including: Extract key information from the declarative syntax corresponding to the historical data visualization charts to obtain a simplified declarative syntax, which is a structured declarative syntax. The simplified declarative syntax is converted into a string-format declarative syntax, and the feature vector of the declarative syntax corresponding to the historical data visualization chart is obtained based on the string-format declarative syntax.
4. The method according to claim 3, characterized in that, The key information includes: Other information besides the chart interaction information, which is the information displayed in response to an interactive operation triggered by a data visualization chart.
5. The method according to claim 1, characterized in that, The step of training the chart processing model using the historical data visualization charts and their labels includes: The pre-trained model is fine-tuned using the historical data visualization charts and their labels, wherein the pre-trained model has the ability to understand the semantic relationship between images and image descriptions.
6. The method according to claim 1, characterized in that, Determining a second feature vector from the vector database whose matching degree with the first feature vector is higher than a certain threshold includes: Determine at least one second feature vector from the vector database that matches the first feature vector; The step of determining the declarative syntax corresponding to the data visualization chart to be processed based on the declarative syntax corresponding to the second feature vector in the vector database includes: The declarative syntax corresponding to the target feature vector in the vector database is determined as the declarative syntax corresponding to the data visualization chart to be processed, wherein: the target feature vector is the top N second feature vectors after sorting the at least one second feature vector in descending order of matching degree with the first feature vector, and N is a positive integer.
7. A data visualization chart processing device, characterized in that, The device includes: The acquisition unit is used to acquire image-formatted visualization charts of the data to be processed. The first determining unit is used to determine the first feature vector of the data visualization chart to be processed, wherein the first feature vector is the same as the feature vector of the declarative syntax corresponding to the data visualization chart to be processed. The second determining unit is used to determine a second feature vector from a vector database whose matching degree with the first feature vector is higher than a certain threshold. The vector database includes: multiple feature vectors and a declarative syntax corresponding to each of the multiple feature vectors, and the multiple feature vectors include the second feature vector. The third determining unit is used to determine the declarative syntax corresponding to the visualization chart of the data to be processed based on the declarative syntax corresponding to the second feature vector in the vector database. Specifically, the first determining unit is used to input the data visualization chart to be processed into the chart processing model to obtain the first feature vector. The chart processing model is used to determine the feature vector corresponding to the input data visualization chart. The chart processing model is trained in the following way: for any historical data visualization chart among multiple historical data visualization charts, the following operation is performed: the feature vector of the declarative syntax corresponding to the historical data visualization chart is used as the label of the historical data visualization chart. The chart processing model is trained using the historical data visualization chart and the label of the historical data visualization chart. The label of the historical data visualization chart is used to indicate the feature vector of the historical data visualization chart.
8. An electronic device, characterized in that, The device includes a processor and a memory; The processor is configured to execute instructions stored in the memory to cause the device to perform the method described in any one of claims 1-6.
9. A computer-readable storage medium, characterized in that, Includes instructions that instruct the device to perform the method described in any one of claims 1-6.
Citation Information
Patent Citations
Decoding method and electronic equipment
CN112019847A
Standard file information extraction method and system based on neural network and template matching
CN114842483A