Method for extracting diagram data from vector graphics

Abstract

The invention relates to a computer system and a computer-implemented method of extracting graphically presented data from vector graphics from electronic data, the method comprising the steps: a) reading the electronic data and acquiring all graphical and typographical objects, b) identifying all objects representing figures or diagrams, from which the graphically presented data shall be extracted, c) for each figure or diagram identifying the reference system, to which the data shown in the figure or diagram are geometrically related, by identifying the objects representing the reference system, d) for each figure or diagram identifying the typographical and graphical objects, which represent the graphically presented data, e) processing the objects collected in steps c) and d) according to their graphical appearance, attributes or character encoding, and determining the information they are intended to graphically depict, f) finding the coordinate transformation from the objects collected in step c) towards their actual meaning according to step e) and transforming the objects obtained in step d) according to this coordinate transformation, and saving, exporting or processing these transformed numerical values.

Description

[0001] METHOD FOR EXTRACTING DIAGRAM DATA FROM VECTOR GRAPHICS

[0002] The present invention relates to a computer-implemented method and a computer system of extracting graphically presented data from vector graphics from electronic data.

[0003] Background of the invention

[0004] Data in scientific or non-scientific publications are very often illustrated by graphical illustrations in the form of quantitative diagrams. In order to use the underlying data for further investigations, the actual data must be available in the form of numerical values. Since the actual data are often not accessible, they must frequently be taken from the diagram or figure. This process is commonly referred to as digitization. Previously available methods and software tools for digitization are based on a procedure which is manually and technically complex, and also inevitably involves loss of information.

[0005] In the sense of a descriptive and expedient presentation of various data, these are often visualized graphically in the form of quantitative diagrams. These can be for example a simple line diagram of the temporal evolution of a measured variable, or a complex bubble diagram of the relationship between three or more dimensions, e.g., fuel consumption depending on car weight, engine power and number of cylinders. In a bubble diagram the information may be displayed not only by the positioning of data points, but also by the size of the bubble symbols and possibly by their colour, style or other features. On top of that many other forms of quantitative diagrams are common. However, these are always based on the same principle: numerical values are represented graphically with respect to a reference system in the sense of an illustration.

[0006] The publication and dissemination of graphical illustrations are nowadays mostly done electronically, for example, in an electronic document or on a website. The illustrations are stored either as raster graphics or as vector graphics. These different graphic formats have an elementary difference: Raster graphics consist of a raster-shaped arrangement of so-called pixels, with colour information being assigned to each pixel. Vector graphics, on the other hand, are composed of graphical or typographical elements which are defined on the basis of their geometric or semiotic specification. The appearance of a vector graphic is thus determined by its image description. The elementary difference between raster graphics and vector graphics can be illustrated by scaling their size. An enlarged raster graphic is visibly pixelated.

[0007] In contrast, a vector graphic can be scaled without any loss of quality. Furthermore, there is another essential difference between raster graphics and vector graphics. Each pixel of a conventional raster graphic represents a single colour information (possibly in terms of multiple colour channel values). Thus, a conventional raster graphic can only represent a single layer of information. In contrast, vector graphics can represent several layers of information. Hence, while raster graphics inevitably suffer from a loss of information if graphical objects overlap, vector graphics retain all information as long as they are passed on as vector graphics.

[0008] The digitization of data represented graphically in quantitative diagrams has been covered in numerous publications for many years and there are many different software tools for this task. In the following, some significant scientific publications on this topic are summarized.

[0009] In the publication "Extraction of Vectorized Graphical Information from Scientific Chart Images" by Huang et al. (2007) an algorithm is presented, which serves the digitization of data from diagrams. The term “vector” in the title of this publication is insofar ambiguous that it does not refer to the extraction of data from vector graphics. Rather, an algorithm is proposed with which the lines represented in a raster graphic are first described mathematically in the form of vectors. On basis of this vectorization then data are extracted. This complicated procedure is probably caused by the circumstance that at that time raster graphics were used much more frequently than vector graphics.

[0010] In the publication “Re Vision: Automated Classification, Analysis and Redesign of Chart Images” by Savva et al. (2011) the software tool Re Vision is presented, which serves the digitization of data from diagrams. This software tool is based on the analysis of raster graphics. Similarly, in the publication "ChartSense: Interactive Data Extraction from Chart Images" by Jung et al. (2017), the software tool ChartSense is presented. This software tool is also based on the analysis of raster graphics. In this publication, it is claimed that ChartSense offers advantages compared to the existing software tools Re Vision and WebPlotDigitizer. In the publication "CrowdChart: Crowdsourced Data Extraction From Visualization Charts" by Chai et al. (2020) it is claimed that the methods known so far are not always effective. Therefore, a new approach called CrowdChart is presented, which is based on crowdsourcing. Thereby a task schema is presented, which is advantageous for the outsourcing of subtasks to volunteers.

[0011] Another example is the software WebPlotDigitizer, which is freely available in the form of a web application. The functionality of this software is the same as that of the above literature sources and is based on an analysis of raster graphics.

[0012] Masson et al. (Damien Masson, Sylvain Malacria, Daniel Vogel, Edward Lank, Gery Casiez. “ChartDetective: Easy and Accurate Interactive Data Extraction from Complex Vector Charts" CHI 2023 - ACM Conference on Human Factors in Computing Systems, Apr 2023, Hamburg, Germany. https: / / doi / 10.1 145 / 3544S48 3S81 113) disclose a system called ChartDetective. This is a semi-automatic system, which leverages vector diagrams to extract the underlying data. ChartDetective uses a drag-and-drop interface combined with selection, filtering and previsualization features for extracting the data. However, the publication reveals no information on a more precise and accurate design of the graphical extraction.

[0013] Summarizing, previous methods for extracting data from diagrams are based on the analysis of raster graphics. Since raster graphics only approximate the actual data, they are inherently subject to information loss, as described above. Such losses occur, for example due to pixilation, scaling or image compression. Concerning the representation of data in diagrams, data points can also be superimposed, as e.g., in scatter plots with lots of data points lying close to each other. In case of an analysis of such diagrams as a raster graphic important information can be lost (depending on the graphic rendition of the data points and the image resolution). Thus, already the raw data, on which previous methods rely, show considerable deficiencies. Overall, these methods have significant disadvantages in terms of accuracy and efficiency for the extraction of data.

[0014] Short description of the invention

[0015] Thus, the object of the present invention is to provide a method for an efficient and precise extraction of graphically represented data contained in electronic documents in vector format, wherein especially a measure for the accuracy of the definition of reference system and / or data should be provided.

[0016] A solution is provided by a computer-implemented method of extracting graphically presented data from vector graphics from electronic data comprising the steps: a) reading the electronic data and acquiring all graphical and typographical objects, b) identifying all objects representing figures or diagrams, from which the graphically presented data shall be extracted, c) for each figure or diagram identifying the reference system, to which the data shown in the figure or diagram are geometrically related, by identifying the objects representing the reference system, d) for each figure or diagram identifying the typographical and graphical objects, which represent the graphically presented data, e) processing the objects collected in steps c) and d) according to their graphical appearance, attributes or character encoding, and determining the information they are intended to graphically depict, f) finding the coordinate transformation from the obj ects collected in step c) towards their actual meaning according to step e) and transforming the objects obtained in step d) according to this coordinate transformation, and saving, exporting or processing these transformed numerical values.

[0017] The inventive method allows efficient and precise extraction of data from diagrams and similar illustrations available in vector format. The basic idea of the inventive method is not to evaluate the graphical appearance as with previously available tools, but to directly analyse the image description of the objects stored electronically in the vector graphics, to interpret these objects and their attributes according to their intended meaning and finally to reference them quantitatively to each other.

[0018] The starting point for efficient and accurate extraction of diagram data is an electronic document or data stream containing such an illustration in vector format, the electronic data being read in a first step a). In a special embodiment these figures or diagrams, from which the graphically presented data shall be extracted, are in a first step extracted from the electronic document or data stream. For example, the desired vector graphic may be first extracted from an extensive electronic document in Portable Document Format (PDF). Thereby, the corresponding page of the electronic document is extracted, and the layout area is clipped to the relevant content.

[0019] However, such a preparation is optional and does not affect the actual data extraction method.

[0020] Thus, in certain embodiments, before processing the electronic document irrelevant content may be excluded and / or pages containing diagrams may be clipped to the relevant content.

[0021] In any case, to process the actual figures or diagrams, the electronic data containing the vector graphics must be read, so its contents must be interpreted according to the respective syntax and semantics. For reading e.g., PDF files, several open-source tools are available. Other electronic data containing vector graphics can be processed similarly. Examples are vector graphic formats like EMF or SVG and other container formats like HTML or XML.

[0022] After identifying all figures or diagrams in step b), the data shown in the diagram have to be interpreted according to their intended purpose or meaning. Therefore, the next step c) is to identify the reference system to which the data shown in the diagram are geometrically related.

[0023] In a certain embodiment a graphical user interface is used for manually identifying the reference system of a diagram, e.g., based on a two-dimensional Cartesian coordinate system, wherein a user identifies at least two selection boxes on each axis of the diagram thereby indicating the objects defining the reference system. For example, this may be done by means of a selection box drawn by the user via mouse cursor or any other pointing device. The position attributes of these objects determine their position with respect to the layout area.

[0024] However, the method for efficient and precise extraction of vector graphic data is not limited to the restricted functionality of the example described above, but aims at a much more general functionality:

[0025] - With respect to the reference system, this means that an analogous approach is also possible for general reference systems - in addition to general coordinate systems (such as logarithmic scaling or polar coordinates), the same methodology can be applied to a wide variety of quantitative diagrams (such as bar charts, radar charts, or bubble charts).

[0026] - The procedure for identifying the reference system can also be understood in general terms: To begin with, the manual selection by a user can be generalized to allow imprecise user input: If the user selection contains an ambiguous number of objects, the most suitable objects representing the coordinate system have to be identified. For example, when selecting the far end of the abscissa, the user could select several line objects, which represent not only an axis tick mark but also an arrow at the end of the axis. In this example, the intended object can be determined by a simple algorithm selecting the single object with only vertical and no horizontal extent. In addition, the same methodology can be used for a partially or totally automated procedure for identifying the reference system. On the one hand, this applies to conventional, algorithmic program sequences, such as with a step-by-step search for short lines that are arranged orthogonally at the beginning or end to lines of great length. On the other hand, artificial intelligence techniques can be used alternatively on the basis of appropriate training data.

[0027] Regardless of a possible generalization or further development of the functionality of the method, the basic methodology remains the same: in the current step c) of the method, those objects are to be identified which, in their entirety, describe a definition of the reference system of the graphically displayed data.

[0028] The next step d) of the inventive method is to identify those objects that represent the graphically displayed data itself.

[0029] In one embodiment a graphical user interface is used for identifying the objects, which represent the graphically presented data. Thus, similar to the previous step, in an exemplary embodiment the user may specify a selection box. Then all those objects of the vector graphic which lie within this selection according to their position data are automatically selected. For example, several line objects may be selected. The attributes of all these objects indicate their position with respect to the layout area.

[0030] Similar to the previous step of identifying the reference system, the basic method described above is applicable to much more general problems. The objective of the present step d) is to identify those typographic and graphical objects which describe the graphically represented data.

[0031] In the last steps e) and f) of the inventive method, the graphically represented data are to be quantified according to their intended meaning. For this purpose, in the step e) the objects collected in steps c) and d) are at first processed according to their graphical appearance, attributes or character encoding, to determine the information they are intended go graphically depict. Finally, in step f) a coordinate transformation is formulated based on the objects describing the reference system according to step c) considering the information they are intended to graphically depict according to step e). In other words, the geometric positions of the objects identified in step c) are on the one hand related to the layout area of the vector graphic. On the other hand, the graphical appearance, attributes or character encoding of these objects define their intended meaning. For example, a line object representing an axis tick mark determines the geometric position with respect to the layout area of the vector graphic. The corresponding text object representing the axis tick label determines the intended meaning with respect to the actual reference system of the figure or diagram.

[0032] The totality of this information forms the basis for the mathematical formulation of a coordinate transformation which can be used to convert from position information with respect to the layout area towards their actual meaning. For the example with a two-dimensional Cartesian coordinate system, this is a linear transformation in terms of non-uniform scaling. The mathematical parameters of the transformation can be determined from the totality of intended nominal values according to step e) and position attributes of the objects according to step c). The underlying method is again not limited to Cartesian coordinate systems but can also be applied to much more general problems (such as logarithmic scaling or polar coordinates) when using an appropriate coordinate transformation.

[0033] After the coordinate transformation is mathematically known, the set of objects representing the actual data as identified in step d) can be quantified according to their intended meaning: applying the coordinate transformation to the position attributes of these objects yields the actual data underlying the graphical representation. The set of numerical values determined in this way may be stored programmatically in a data object of any collection type. Further, these values can be saved, exported or fed to further electronic data processing.

[0034] The object of the present invention is solved by a computer system for extracting graphically presented data from vector graphics from electronic data, the computer system comprising:

[0035] - a processor capable of executing instructions to parse and interpret electronic data containing a vector graphic representation of a figure or diagram, acquiring all graphical and typographical objects stored in the vector format,

[0036] - preferably a memory for storing the parsed electronic data and the acquired graphical and typographical objects,

[0037] - a processing logic configured to identify the objects representing figures or diagrams, and for each diagram to identify the objects representing the reference system, to which the data shown in the figure or diagram are geometrically related, and the graphical and typographical objects representing the graphically presented data and to process these objects according to their graphical appearance, attributes or character encoding,

[0038] - a mathematical transformation logic configured to formulate a coordinate transformation based on the identified and processed objects representing the reference system,

[0039] - a data quantification logic configured to apply the coordinate transformation to the attributes of the identified and processed graphical and typographical objects, generating numerical values representing the actual data presented in the figure or diagram.

[0040] Thus, with the help of this computer system it is possible to extract data efficiently and precisely from diagrams and similar illustrations available in vector format. With the help of the processing logic steps b - e) of the inventive method can be executed. Further, step f) may be executed via the mathematical transformation logic and the data quantification logic which allow for a coordinate transformation of the objects. The memory allows to store any data. It is particularly preferred to store the numerical values corresponding to the data depicted in the figure or diagram to use them for further applications. Further, the memory may comprise a data storage capability to store the quantified numerical values in a data object of any suitable collection type for further electronic data processing or exportation.

[0041] In another embodiment the system comprises a user interface configured to allow a user to interact with the computer system for identifying the objects corresponding to the reference system to which the data depicted in the figure or diagram is geometrically related and / or the typographical and graphical objects, which represent the graphically presented data.

[0042] Preferably, the user interface comprises a graphical user interface configured to enable the user to manually draw selection boxes for each axis of the figure or diagram to identify the reference system and / or the data set. Thus, with the help of a user the reference system as well as the objects which present the data in the figure or diagram can be easily identified.

[0043] In yet another embodiment the processing logic is configured to utilize algorithms or artificial intelligence techniques to identify appropriate graphical and typographical objects representing the reference system and / or the graphically presented data. With the help of these algorithms or artificial intelligence techniques it is possible to allow for a partially or totally automated procedure of identifying and processing all relevant objects without the need of the help of a user. Detailed description and preferred embodiments

[0044] The foregoing and other objects, features and advantages of the invention will become more apparent from the following detailed description, which proceeds with reference to the accompanying figures.

[0045] Fig. 1 shows a block diagram illustrating the main steps of the method for extracting graphically presented data from vector graphics.

[0046] Fig. 2 shows a very simple example of a diagram with two axes depicting the temporal evolution of a variable.

[0047] Fig. 3 shows an example of a complex bubble diagram of a biological data set for penguin populations.

[0048] As illustrated in Fig. 1 the inventive computer-implemented method of extracting graphically presented data from vector graphics from electronic data comprises the following main steps. In step a) the electronic data, that is the document or the data stream, is read in. In the next step b) all objects representing figures or diagrams, from which the graphically presented data shall be extracted, are identified. Next, as illustrated in Fig. 1 by means of steps c-e), a database is gathered, in which the vector graphic objects as well as the information the individual objects are intended to graphically depict are recorded. In more detail, in step c) the reference system, to which the data shown in the figure or diagram are geometrically related, is identified for each figure or diagram by means of identifying the respective objects. In step d) the typographical and graphical objects, which represent the graphically presented data, are identified for each figure or diagram. In step e) the objects collected in steps c) and d) are processed according to their graphical appearance, attributes or character encoding, and the information they are intended to graphically depict is determined. Finally, in the last step f) of the inventive method the coordinate transformation from the objects collected in step c) towards their actual meaning according to step e) is found and the objects obtained in step d) are transformed according to this coordinate transformation. Further, it is possible to save, export or process these transformed numerical values which correspond to the figures or diagrams.

[0049] In step a) the electronic data, that is the document or the data stream, is read in and any existing layout areas, such as individual pages of a PDF file or layout elements of a web page, are processed and all graphical and typographical objects are acquired. In a certain embodiment, it is possible to exclude irrelevant content and / or clip layout areas to the relevant content before further processing the electronic data or data stream.

[0050] In step b) a graphical user interface, algorithm or artificial intelligence technique may be used for identifying all objects representing figures or diagrams, from which the graphically presented data shall be extracted. When using a graphical user interface, a user may indicate a selection box containing all relevant objects. For example, such a selection box may indicate a specific layout area on a specific page of a PDF file. Further, also algorithms or artificial intelligence techniques can be used for identifying all relevant objects, e.g., by searching for figure captions, or other characteristics of figures or diagrams, such as layout areas containing sparse text but appropriate graphical objects.

[0051] In step c) a graphical user interface, algorithm or artificial intelligence technique may be used for identifying the reference system of the figure or diagram. Preferably for each dimension of the figure or diagram one or more objects defining at least two positions are identified, wherein the objects define the reference system. When using a graphical user interface, a user may indicate the position areas which help to identify the reference system. This can be done by e.g., drawing selection boxes. Further, also algorithms or artificial intelligence techniques can be used for identifying the reference system. With the help of these a partially or totally automated procedure for identifying the reference system may be possible. For example, relevant axis tick marks can be found with a step-by-step search for short lines that are arranged orthogonally at the beginning or end to lines of great length, that is the axis lines. Similarly, regularly arranged axis tick marks or labels, or regularly arranged grid lines may be searched. Based on certain training data also different artificial intelligence techniques may be used. With the help of these methods, it is possible to recognize and select appropriate graphical and typographical objects representing the reference system in step c) of the inventive method. If in step c) surplus objects were identified by any of the methods described above, it is also possible to use a graphical user interface, algorithm or artificial intelligence technique to select the most suitable objects representing the reference system out of these surplus objects.

[0052] Further, a graphical user interface, algorithm or artificial intelligence technique can be also used in step d) for identifying the typographical and graphical objects, which represent the graphically presented data. For this purpose, all objects representing the data set are selected. With the help of a graphical user interface this can be again done e.g., by a user drawing a selection box around the data set. Via algorithms or artificial intelligence techniques it is again possible to select these objects via partially or totally automated procedures.

[0053] Below, an example is described how electronic data may be processed. For this example, the diagram of Fig. 2 is considered, which shows a simple line diagram for the development of a variable x over time t.

[0054] The underlying electronic document contains various graphical and typographical objects, which are stored in the form of unstructured data. When the file is read, a large number of different objects are detected (although a minimal example is considered). The following list shows samples of some of these objects (with their respective memory address): cobject : document at 0x000003Fl> cobject : page at 0x000007D3> cobject : text at 0x00000BB9> cobject : line at 0x0000138B> cobject : line at 0x00001777>

[0055] As a whole, the graphical and typographical objects describe the diagram, but in the electronic document the individual objects are only stored as an unstructured collection of diverse objects (without hierarchy or the like) which cannot be interpreted as intended - the attributes of the individual objects only describe their graphical depiction and the positions of the individual object in relation to the layout area of the vector graphic. For example, the polygonal line object representing the line graph may read:

[0056] Object entity: cobject : polyline at 0x00001F49>

[0057] Position data: { ' x0 ' : 55.76, ' y0 ' : 64.66, . . . }

[0058] Display style: { ' colour ' : . . . , ' line width ' : . . . , . . . }

[0059] The position data defines the geometry of the polygonal line object with respect to the layout area. These coordinates do not directly describe the data, which the figure or diagram is intended to depict. However, after the electronic data has been read, all objects are known which in their entirety describe the graphical appearance of the diagram.

[0060] Let us now consider an example of how the reference system can be identified with the help of a user. For this example, again the diagram of Fig. 2 is considered. The procedure can be as follows: the user identifies at least two position areas on each axis of the diagram, that is the x- axis corresponding to time t and the y-axis corresponding to variable x in the example of Fig. 2, by drawing a selection box with the mouse over the axis tick mark and the corresponding axis tick label. For example, the user may do a manual selection of the origin value of the abscissa. After the user has drawn the selection box, those graphical and typographical objects can automatically be selected which, according to their position data, lie completely within the selection box. For the sake of simplicity, it may be assumed that exactly one text object and one line object remain. Then, the analysis of the intended axis value is directly possible: the text content corresponds to the intended nominal value of the data axis and the position attributes of the axis tick mark define its position with respect to the layout area.

[0061] For example, the text obj ect representing the axis tick label for the origin of the abscissa selected by the user reads:

[0062] Object entity: <obj ect : text at 0x00002329>

[0063] Text content: ' 0.0 '

[0064] Text style: { ' font ' : ' font size ' : . . . }

[0065] Therein, the text content defines the nominal value of the intended definition of the axis origin in the diagram. However, the complete definition of the axis origin includes not only the text object but also the corresponding axis tick mark. The latter is not linked to the axis tick label in the electronic data but was identified by the manual selection of the user. The axis tick mark is described by the following object:

[0066] Object entity: cobject : line at 0x00002717>

[0067] Position data: { ' x0 ' : 29.76, ' y0 ' : 27.52, ' xl ' : 29.76, ' yl ' : 145.59}

[0068] The position data defines the geometry of the line object with respect to the layout area. To completely define the Cartesian coordinate system for the diagram in Fig. 2, two position areas must be selected for each axis in this way. Similarly, a coordinate axis with logarithmic scaling can also be defined by selecting one or more objects defining at least two positions. More general, the same methodology can be used for other coordinate systems: for example, for polar coordinates the centre point, the radial axis and the angular axis have to be defined by selecting appropriate objects.

[0069] In the case that the user's selection of the position areas contains surplus objects, the most suitable objects defining the reference system can be selected by requesting additional user input, or utilizing algorithms or artificial intelligence techniques.

[0070] For example, if the selection box for the origin of the abscissa contains two lines, that is besides the vertical x-axis tick mark, a horizontal y-axis tick mark was selected by mistake, a simple algorithm may be used for choosing the relevant object. This algorithm can exploit the fact that the axis tick mark for the abscissa has a larger vertical extension than horizontal one.

[0071] Similarly, if more than one axis tick mark of the same axis is selected, the one closest to the relevant label may be selected. Likewise, if more than a single pair of axis tick mark and corresponding tick label is selected, the most suitable pair may be chosen to be the one which covers a larger part of the axis.

[0072] In a certain embodiment, surplus objects defining the reference system may be used to estimate the accuracy of the reference system definition. For example, consider defining the abscissa of a Cartesian coordinate system, which is uniquely defined by two pairs of axis tick mark and corresponding tick label. Any surplus pair of objects may subsequently be used to compute potential discrepancies of the reference system definition, which may occur due to rounding errors within the image description of the vector graphics or other imprecisions. The consequential accuracy may be used to detect mistakenly selected objects or to inform the user of the accuracy of the extracted diagram data.

[0073] In contrast to the former example, Fig. 3 shows an example of a more complex bubble diagram. In the diagram of Fig. 3 exemplary a biological data set for penguin populations is depicted. The abscissa corresponds to the beak length, while the ordinate corresponds to the flipper length of the penguins. Moreover, the information is displayed not only by the positioning of the data points, but also by the size and style of the bubble symbols. More precisely, the size of the bubble symbols corresponds to the body mass of the penguins, and the differently styled bubble symbols represent different penguin species. This means that the objects identified to represent the data have to be further processed in order to obtain all of the information they represent. For example, for each bubble symbol in the bubble diagram, the centre of the bubble has to be found, which corresponds to the actual data point defining a certain flipper length and beak length. Similarly, the bubble size has to be analysed to evaluate the corresponding body mass. In general, the bubble size can be related to either its area or its radius, depending on the specification of the bubble diagram. Furthermore, the reference value is not defined by a conventional coordinate system, the reference system is rather implicitly defined by the bubble sizes in the diagram legend, the figure caption or any other description.

[0074] By the example of Fig. 3, also the identification of a coherent data set can be described. For example, if not the data for all three penguin species are of interest, but only the data for a single species, all bubble symbols with an equal style have to be identified, which form the coherent data set for this very species. This selection may be done based on a partially or totally automated procedure utilizing an algorithm or artificial intelligence technique to identify all data points which look alike in terms of colour, or any other style or any other feature. Generally, the identification of such a coherent data set is not only of interest for bubble diagrams, but also for any other type of figure or diagram representing one or more data sets being distinguishable with respect to one or more characteristics.

[0075] More general, for certain figures or diagrams the identified objects may have to be processed according to their graphical appearance, attributes or character encoding in order to determine the information the objects are intended to graphically depict. In Fig. 3, for example, the bubble symbols for species Z are depicted with a style of linear hatching, consisting of dense diagonal lines. These lines do not represent any actual data, but merely constitute the graphic style for distinguishing the data points. This has to be taken into account when processing such a vector graphic: for analysing the actual data points, such objects merely representing a graphic style have to be ignored, while they may be crucial for the identification of a coherent data set.

[0076] Similarly, for certain figures and diagrams the objects representing actual data may require processing to determine the information they are intended to graphically depict. For example, a data curve may not be represented by a simple polygonal line object, but a narrow polygon representing the contour of a line shape filled with colour. In order to determine the actual data curve, the contour of the polygon must be reduced to its centre line. Thereby, potential graphical anomalies such as rounded comers have to be taken into account. In general, appropriate processing of vector objects according to their graphical appearance, attributes or character encoding enables to determine the information the objects are intended to graphically depict.

Claims

Claims:

1. A computer-implemented method of extracting graphically presented data from vector graphics from electronic data comprising the steps: a) reading the electronic data and acquiring all graphical and typographical objects, b) identifying all objects representing figures or diagrams, from which the graphically presented data shall be extracted, c) for each figure or diagram identifying the reference system, to which the data shown in the figure or diagram are geometrically related, by identifying the objects representing the reference system, d) for each figure or diagram identifying the typographical and graphical objects, which represent the graphically presented data, e) processing the objects collected in steps c) and d) according to their graphical appearance, attributes or character encoding, and determining the information they are intended to graphically depict, f) finding the coordinate transformation from the obj ects collected in step c) towards their actual meaning according to step e) and transforming the objects obtained in step d) according to this coordinate transformation, and saving, exporting or processing these transformed numerical values, wherein a graphical user interface is used for identifying the reference system of the figure or diagram, wherein for each dimension of the figure or diagram one or more position areas are selected to identify the objects defining the reference system or wherein the identification of the reference system is done by utilizing algorithms or artificial intelligence techniques to recognize and select appropriate graphical and typographical objects representing the reference system, wherein a graphical user interface, algorithm or artificial intelligence technique is used for selecting the most suitable objects representing the reference system if surplus objects were identified, wherein one or more surplus objects defining the reference system are used to determine a measure of the accuracy for the definition of the reference system.

2. Method according to claim 1, wherein one or more surplus objects comprises one or more surplus information or surplus objects and / or surplus attributes of one or more objects.

3. Method according to claim 2, wherein one or more surplus information is used to determine a measure of the accuracy for the definition of the reference system and / or the data.

4. Method according to any of the preceding claims, wherein a graphical user interface, algorithm or artificial intelligence technique is used for identifying the typographical and graphical objects, which represent the graphically presented data.

5. Method according to any of the preceding claims, wherein an algorithm or artificial intelligence technique is used for identifying the objects representing a coherent data set, wherein the coherent data set is a subset of the complete data set, the subset data being distinguishable from the other data with respect to one or more characteristics of the data set.

6. Method according to any of the preceding claims, wherein the figures or diagrams, from which the graphically presented data shall be extracted, are in a first step identified from the electronic data and the corresponding graphical and typographical objects are acquired.

7. Method according to any of the preceding claims, wherein before processing the electronic data or data stream, irrelevant content is excluded and / or layout areas are clipped to the relevant content.

8. A computer system for extracting graphically presented data from vector graphics from electronic data, the computer system comprising:- a processor capable of executing instructions to parse and interpret electronic data containing a vector graphic representation of a figure or diagram, acquiring all graphical and typographical objects stored in the vector format,- preferably a memory for storing the parsed electronic data and the acquired graphical and typographical objects,- a processing logic configured to identify the objects representing figures or diagrams, and for each diagram to identify the objects representing the reference system, to which the data shown in the figure or diagram are geometrically related, and the graphical and typographical objects representing the graphically presented data and to process these objects according to their graphical appearance, attributes or character encoding,- a mathematical transformation logic configured to formulate a coordinate transformation based on the identified and processed objects representing the reference system,- a data quantification logic configured to apply the coordinate transformation to the attributes of the identified and processed graphical and typographical objects, generating numerical values representing the actual data presented in the figure or diagram.

9. A system according to claim 8, wherein the system further comprises a user interface configured to allow a user to interact with the computer system for identifying the objects corresponding to the reference system to which the data depicted in the figure or diagram is geometrically related and / or the typographical and graphical objects, which represent the graphically presented data.

10. A system according to claim 9, wherein the user interface comprises a graphical user interface configured to enable the user to manually select one or more position areas for each dimension of the figure or diagram to identify the reference system and / or to manually select the complete data set or a subset of the data set.

11. A system according to any of the claims 8 to 10, wherein the processing logic is configured to utilize algorithms or artificial intelligence techniques to identify appropriate graphical and typographical objects representing the reference system and / or the graphically presented data.

12. A system according to any of the claims 8 to 11, wherein the memory comprises a data storage capability to store the quantified numerical values in a data object of any suitable collection type for further electronic data processing or exportation.

Images