A chart de-rendering system, method, and program for extracting metadata and data information from charts using artificial intelligence.

Abstract

A chart de-rendering system, method, and program for extracting metadata and data information from a chart using artificial intelligence are disclosed. The system includes a memory containing an AI model which includes an image encoder that takes a chart as input and outputs it in the form of a first embedded which can be processed by an AI model; a meta-decoder that outputs a second embedded which contains metadata from the first embedded; and a data decoder that outputs a fourth embedded which contains data information from a third embedded which contains information about entities contained in the second embedded; and a processor for running or training the AI ​​model. The data information contained in the fourth embedded can be tokenized into a single token for each piece of data information, and a predefined iterative template can be used when extracting data information from a single token.

Description

Technical Field

[0001] The present invention relates to a chart rendering system, method, and program for extracting meta information and data information included in a chart, and more particularly, to a chart rendering system, method, and program capable of extracting meta information and data information included in a chart using artificial intelligence.

Background Art

[0002] Charts included in papers, reports, teaching materials, etc. are generally generated through a process of transferring a data table set with numbers and groups, codes defining an overall layout (e.g., type, direction, color / figure composition), etc. to a rendering engine.

[0003] Chart de-rendering is a process opposite to chart rendering, which means analyzing and grouping the visual patterns and information of a chart to extract core information, and then extracting information related to data (e.g., numbers, groups, etc.), information related to the chart layout, etc., from it.

[0004] In the initial chart de-rendering process, a rule-based model was used. The rule-based model means a model that extracts chart information using predefined functions (e.g., color detection, value extraction of chart axes, etc.) and combines or analyzes each piece of information using predefined rules. The rule-based model has the advantage of excellent accuracy, but has the disadvantage of lacking scalability because separate models are required for each type of chart.

[0005] To compensate for these shortcomings, generative models, which utilize a single trained artificial intelligence (AI), have been introduced (DePlot: One-shot visual language reasoning by plot-to-table translation. In Findings of the Association for Computational Linguistics: ACL 2023, pages 10381-10399). Generative models offer improved scalability compared to rule-based models, as a single model can be easily applied to any type of graph.

[0006] However, conventional generative models used a method of including metadata (e.g., X-axis and Y-axis names, entity (single unit of data) group names recorded in the legend, etc.) and data information (e.g., X-axis and / or Y-axis values ​​for each entity, etc.) in a single data format without distinguishing between them. For example, as shown in Figure 6, the chart title (Meta Chart), X-axis name (Epoch), Y-axis name (Experiment result), X-axis (Epoch) value [1,2,3,4,5] and all Y values ​​for each corresponding entity group (Model1 and Model2) (i.e., Model1 is [10,nan,30,nan,50], Model2 is [1,4,9,16,25]) are all recorded in a single data format. Such a data format has problems such as representing a lot of information, having a complex data representation, the constraint that all Y values ​​dependent on each X value must be represented, entity values ​​(e.g., "nan" values) not being identified, and a high possibility of formatting errors.

[0007] Furthermore, traditional generative models generally treated numbers as text. Therefore, they represented numbers by dividing a single number into irregular units, and then dividing each unit into multiple tokens for input. This method unnecessarily uses many tokens to represent a single number (for example, to represent the number 13.1, two tokens are needed, one for "1" and the other for "3.1"). This resulted in traditional language models using an unnecessarily large number of tokens, leading to wasted tokens and slower inference speeds. [Prior art documents] [Patent Documents]

[0008] [Non-Patent Document 1] DePlot:One-shot visual language reasoning by plot-to-table translation.In Findings of the Association for Computational Linguistics: ACL 2023, pages 10381-10399. [Overview of the project] [Problems that the invention aims to solve]

[0009] The problem that this invention aims to solve is to provide a system, method, and program that can extract numerical information efficiently and accurately while providing a data format that can easily represent the content of data included in a graph.

[0010] The problems that this invention aims to solve are not limited to those mentioned above, and other problems not mentioned will be clearly understood by those skilled in the art from the following description. [Means for solving the problem]

[0011] A system for implementing an AI model that extracts metadata and data information contained in a chart of the present invention comprises at least one processor and at least one memory that stores instructions or information causing the at least one processor to perform an operation, wherein the operation performed by the instruction includes the steps of: inputting the chart into an image encoder and converting it into a first embedded that can be processed by the AI ​​model; inputting the first embedded into the AI ​​model, outputting a second embedded containing the metadata from the first embedded, outputting a fourth embedded containing the data information from a third embedded containing information about entities contained in the second embedded; and outputting a first data format in which the metadata contained in the second embedded is recorded and a second data format in which the data information contained in the fourth embedded are recorded, respectively.

[0012] In the aforementioned system, the data information included in the fourth embedded can be distinguished for each group of entities.

[0013] In the aforementioned system, the metadata may include the chart title, the axis names, and the names of entity groups included in the legend.

[0014] In the system described above, the data information may include numerical information contained in the chart.

[0015] In the system described above, the data information is tokenized into a single token for each piece of data information and included in the fourth embedded, and when recording the data information included in the fourth embedded into the second data format, the tokenized data information can be extracted from the single token and recorded in the second data format.

[0016] In the aforementioned system, the data information can be extracted from the single token using a multilayer perceptron (MLP).

[0017] In the aforementioned system, when extracting data information from a single token, multiple pieces of data information can be extracted simultaneously by inputting a predefined iterative template for the single token.

[0018] A method for extracting metadata and data information contained in a chart according to another aspect of the present invention involves inputting the chart into an image encoder to convert it into a first embedded that can be processed by an AI model, inputting the first embedded into the AI ​​model, outputting a second embedded containing the metadata from the first embedded, outputting a fourth embedded containing data information from a third embedded containing entity information contained in the second embedded, and outputting a first data format in which the metadata contained in the second embedded is recorded, and a second data format in which the data information contained in the fourth embedded is recorded.

[0019] In the above method, the data information included in the fourth embedded can be distinguished for each group of entities.

[0020] In the above method, the metadata may include the title of the chart, the names of the axes, and the names of the entity groups included in the legend.

[0021] In the above method, the data information may include numerical information contained in the chart.

[0022] In the method, the data information is tokenized into a single token for each piece of data information and included in the fourth embedding. When recording the data information included in the fourth embedding in a second data format, the tokenized data information can be extracted from the single token and recorded in the second data format.

[0023] In the method, the data information can be extracted from the single token using a multi-layer perceptron (MLP).

[0024] In the method, when extracting the data information from the single token, the single token can be input into a predefined iterative template to extract a plurality of data information simultaneously.

[0025] A program according to another aspect of the present invention may be stored in a computer-readable recording medium for implementing an AI model that is coupled to a computer and extracts meta-information and data information included in a chart through the AI model according to an embodiment of the present invention.

Advantages of the Invention

[0026] According to the present invention, by separately extracting metadata, the length of each data format is shortened, and the content of the data can be easily expressed, thereby reducing the possibility of format errors.

[0027] Also, according to the present invention, since raw data is classified and independently expressed for each entity group, there is no need to input a "nan" value, which is difficult to process for blank data values, minimizing format errors caused by the inability to identify "nan" values and optimizing the use of tokens.

[0028] Also, according to the present invention, each numerical value included in the entity group of the chart is a single token ( <num>By using this method for output, the separation of tasks between text comprehension and data extraction is facilitated, and the reduction in tokens used to represent numerical values ​​enables efficient learning and prediction within the model.

[0029] Furthermore, according to the present invention, a sufficient length of " <num>By pre-supplying tokens, the need for automatic regression transfer can be minimized, and inference speed can be significantly improved.

[0030] The effects of the present invention are not limited to those described above, and other effects not mentioned will be clearly understood by those skilled in the art from the following description. [Brief explanation of the drawing]

[0031] [Figure 1] This is a schematic diagram of a system for realizing an artificial intelligence-based chart de-rendering method according to one embodiment of the present disclosure. [Figure 2] This is a block diagram illustrating the configuration of an apparatus for performing an artificial intelligence-based chart de-rendering method according to one embodiment of the present disclosure. [Figure 3] This is a block diagram illustrating a method for extracting metadata and data information from a chart according to an embodiment of the present invention. [Figure 4] This is a conceptual diagram illustrating a method for embedding the digit "-0412920" through an SNE according to an embodiment of the present invention. [Figure 5] Figure 5a is a conceptual diagram showing a conventional language model that recognizes and represents numbers using an autoregressive method. Figure 5b is a diagram showing a method of recognizing and representing numbers using an autoregressive method via SNE according to an embodiment of the present invention. Figure 5c is a diagram showing a method of recognizing and representing numbers using an iterative template via SNE using a non-autoregressive method according to an embodiment of the present invention. [Figure 6] This is an example of a data format used in conventional generative models. [Modes for carrying out the invention]

[0032] The embodiments described below are provided as examples to fully convey the concept of the present invention to those skilled in the art to which the present invention pertains. Therefore, the present invention is not limited to the embodiments described below and can be embodied in other forms.

[0033] Throughout this invention, the same reference numerals refer to the same components. This invention does not describe all elements of the embodiments, and general content in the art to which this invention belongs or content that overlaps between embodiments is omitted. The terms “parts, modules, components, and blocks” as used herein can be embodied in software or hardware, and in some embodiments, multiple “parts, modules, components, and blocks” may be embodied as a single component, or a single “part, module, component, and block” may include multiple components.

[0034] Throughout this specification, the term "connected" to another part includes not only direct connection but also indirect connection, the indirect connection including connection via a wireless communication network.

[0035] Furthermore, when a part "includes" a certain component, unless otherwise specified, it means that it can include other components rather than excluding them.

[0036] Throughout the specification, the phrase "on top of" another member includes not only cases where one member is in contact with another member, but also cases where another member exists between the two members.

[0037] The terms "First," "Second," etc., are used to distinguish one component from another, and the components are not limited by the aforementioned terms.

[0038] Unless there is a clear exception in the context, singular expressions include plural expressions.

[0039] The identification codes at each stage are used for explanatory purposes only and do not indicate the order of the stages. The stages may be performed in an order different from the one stated unless the context clearly indicates a specific order.

[0040] The chart de-rendering system according to the present invention may include a device, which may include any various devices capable of performing calculations and providing results to a user. For example, the chart de-rendering system according to the present invention may include at least one computer, server, and mobile terminal, or any other form having the same or similar functions. However, the present invention is not limited thereto.

[0041] Here, the computer may include, for example, a laptop computer, desktop computer, laptop computer, tablet PC, or slate PC equipped with a web browser.

[0042] The server device may include, as a server that communicates with external devices and processes information, an application server, computing server, database server, file server, game server, mail server, proxy server, and web server, etc.

[0043] The aforementioned mobile terminals may include, for example, any handheld-based wireless communication devices of any kind that guarantee portability and mobility, such as PCS (Personal Communication System), GSM (Global System for Mobile communications), PDC (Personal Digital Cellular), PHS (Personal Handyphone System), PDA (Personal Digital Assistant), IMT (International Mobile Telecommunication)-2000, CDMA (Code Division Multiple Access)-2000, W-CDMA (W-Code Division Multiple Access), WiBro (Wireless Broadband Internet) terminals, smartphones, etc., as well as wearable devices such as watches, rings, bracelets, anklets, necklaces, glasses, contact lenses, or head-mounted devices (HMDs).

[0044] Hereinafter, embodiments of the present invention will be described in detail with reference to the attached drawings.

[0045] The present invention relates to a chart de-rendering system, method, and program for extracting metadata and data information contained in a chart, and more particularly to a chart de-rendering system, method, and program that can extract metadata and data information contained in a chart using artificial intelligence.

[0046] Figure 1 is a schematic diagram of a system that can implement a method for de-rendering a chart according to one embodiment of the present invention.

[0047] As shown in Figure 1, the system 1000 may include a device 100, a database 200, and an AI model 300.

[0048] The device 100, database 200, and AI model 300 included in system 1000 can communicate via network W. Here, network W can include wired and wireless networks. For example, the network can include various types of networks such as local area networks (LANs), metropolitan area networks (MANs), and wide area networks (WANs).

[0049] Furthermore, network W may also include the well-known World Wide Web (WWW). However, network W according to embodiments of the present invention is not limited to the networks listed above, and may also include, at least partially, well-known wireless data networks, well-known telephone networks, and well-known wired and wireless television networks.

[0050] The device 100 can input charts based on the AI ​​model 300 and output information related to the charts.

[0051] The charts input to the AI ​​model 300 can be vertical / horizontal bar charts, line graphs, pie charts, region charts, dispersion charts, radial charts, histograms, and / or waterfall charts. The chart may be a single form or a combination of multiple forms. However, the charts used in the present invention are not limited to these, and the chart can include any form of chart. In addition to information that visualizes numerical values, etc. (e.g., lines, circles, etc.), the chart may also include text information, specifically, chart annotations, chart legends or titles, names of each axis (e.g., X-axis, Y-axis, Z-axis, etc.), and numerical values ​​of the raw data of the points included in the chart (e.g., numerical values ​​of the X-axis and Y-axis of the points included in the chart).

[0052] The chart information output from AI model 300 may include metadata, data information, etc., as information that represents the characteristics of the chart. Metadata may include the names of the X and Y axes, the names of entity groups recorded in the legend, etc. Data information may include the numerical values ​​of the X and / or Y axes of each entity.

[0053] According to the present invention, when the device 100 outputs information about a chart based on an acquired chart (image, etc.), it can output metadata and data information included in the chart information separately.

[0054] Database 200 can store various training data for training the AI ​​model 300. Database 200 may also store chart images, chart-related information, and in various embodiments, output data generated by the AI ​​model 300. However, system 1000 may not include database 200 once the training of the AI ​​model 300 is complete.

[0055] Figure 1 shows a case where the database 200 is implemented outside of the device 100. In this case, the database 200 can be connected to the device 100 via wired or wireless connection. However, this is only one embodiment, and the database 200 can also be implemented as a component of the device 100.

[0056] Figure 1 shows the AI ​​model 300 implemented outside the device 100 (for example, implemented in a cloud-based manner, but is not limited to this, and can also be implemented as a component within the device 100).

[0057] Figure 2 is a block diagram illustrating the configuration of an apparatus for performing a method for extracting metadata and data information from charts using artificial intelligence, according to one embodiment of the present invention.

[0058] As shown in Figure 2, the device 100 may include a memory 110, a communication module 120, a display 130, an input module 140, and a processor 150. However, it is not limited to these, and the software and hardware configuration of the device 100 may be modified, added to, or omitted within the scope that is obvious to an ordinary engineer, depending on the required operation. The device 100 may also be replaced by a system, and the device 100 may include multiple devices, in which case each component included in the device 100 may be included in at least one of the multiple devices.

[0059] Memory 110 can store data supporting various functions of the device 100, programs for the operation of the processor 150, input / output data, and numerous application programs (or applications) driven by the device, data for the operation of the device 100, instructions, and AI models. At least some of these application programs can be downloaded from an external server via wireless communication.

[0060] Such memory 110 may include at least one type of storage medium from among flash memory type, hard disk type, SSD type (Solid State Disk type), SDD type (Silicon Disk Drive type), multimedia card micro type, card type memory (e.g., SD or XD memory), RAM (random access memory), SRAM (static random access memory), ROM (read-only memory), EEPROM (electrically erasable programmable read-only memory), PROM (programmable read-only memory), magnetic memory, magnetic disk, and optical disk.

[0061] Furthermore, the memory 110 can be separated from the device and may include a database connected by wire or wireless. The database 200 shown in Figure 1 can be implemented as a component of the memory 110.

[0062] The communication module 120 may include one or more components that enable communication with external devices, for example, at least one of a broadcast receiving module, a wired communication module, a wireless communication module, a short-range communication module, and a location information module.

[0063] Wired communication modules can include not only various wired communication modules such as local area network (LAN) modules, wide area network (WAN) modules, or value-added network (VAN) modules, but also various cable communication modules such as USB (Universal Serial Bus), HDMI (High Definition Multimedia Interface), DVI (Digital Visual Interface), RS-232 (recommended standard 232), power line communication, or POTS (plain old telephone service).

[0064] Wireless communication modules can include Wi-Fi modules, Wi-Fi (Wireless broadband) modules, and wireless communication modules that support various wireless communication methods such as GSM (Global System for Mobile Communication), CDMA (Code Division Multiple Access), WCDMA (Wideband Code Division Multiple Access), UMTS (Universal Mobile Telecommunications System), TDMA (Time Division Multiple Access), LTE (Long Term Evolution), 4G, 5G, and 6G.

[0065] The display 130 displays (outputs) information or data processed by the device 100, data input or output through the AI ​​model 300, etc. The display 130 can also display execution screen information of an application driven by the device 100 (for example, an application), or UI (User Interface) or GUI (Graphical User Interface) information based on such execution screen information.

[0066] The input module 140 is for receiving information from the user. When information is received through the user input unit, the processor 150 can control the operation of the device 100 in accordance with the received information.

[0067] Such an input module 140 may include hardware physical keys (e.g., buttons, dome switches, jog wheels, jog switches, etc., located on at least one of the front, back, and side of the device) and software touch keys. For example, the touch keys may consist of virtual keys, soft keys, or visual keys displayed on a touchscreen-type display 130 through software processing, or touch keys located in areas other than the touchscreen. On the other hand, the virtual keys or visual keys can be displayed on the touchscreen in various forms, and may consist of graphics, text, icons, videos, or combinations thereof.

[0068] The processor 150 can be implemented with a memory that stores data relating to an algorithm or a program that reproduces an algorithm for controlling the operation of the components within the device 100 (including learning and executing an AI model), and at least one processor (not shown) that uses the data stored in the memory to perform the aforementioned operations. In this case, the memory and the processor can be implemented on separate chips, or they can be implemented on a single chip.

[0069] In one embodiment, the system 1000 or apparatus 100 according to the present invention may include at least one processor, and if it includes multiple processors, the multiple processors may be contained within different apparatuses 100.

[0070] Furthermore, the processor 150 can be controlled by any one or more combinations of the above-mentioned components in order to implement the various embodiments of the present disclosure described below on the device 100.

[0071] Figure 3 is a block diagram illustrating a method for extracting metadata and data information from a chart according to an embodiment of the present invention.

[0072] Referring to Figure 3, chart 410, including the chart title, axis names, legend information, numerical values, etc., is input to the image encoder 420. As explained earlier, the form of chart 410 may be, but is not limited to, vertical / horizontal bar graphs, line graphs, pie charts, area charts, dispersion charts, radial charts, histograms, and / or waterfall charts. Furthermore, the form of the chart may be a single form or a form in which multiple forms are combined. For example, chart 410a, shown in Figure 3, can be used.

[0073] The image encoder 420 is an encoder that follows a commonly used encoder architecture and can be implemented in the AI ​​model, specifically ViT, ResNet, etc. The image encoder 420 plays the role of converting the input chart 410a into a first embedded 421 that the AI ​​model 300 can process.

[0074] Next, the first embedded 421 output from the image encoder 420 is input to an AI model 400 which includes a meta decoder 430 that performs meta decoding and a data decoder 450 that performs data decoding.

[0075] First, the first embedded 421 is decoded through the metadecoder 430, which can operate in an automatic regression manner. The metadecoder 430 can recognize metadata from the embedded 421, such as the names of the X and Y axes, the names of entity groups recorded in the legend, etc., but the metadata recognized is not limited to these. The metadecoder 430 outputs a second embedded 431 containing the recognized metadata. For example, when using chart 410a, the second embedded 431 will have the chart title, as shown in embedded 431a. <title> Meta Chart< / title> ),X-axis name( <xtitle> Epoch< / xtitle> ), Y-axis name ( <ytitle> Experiment result< / ytitle> The legend may include a list of entity groups recorded (Epoch|Model1|Model2).

[0076] This invention, by separately extracting metadata in addition to the data information described later, shortens the length of each data format, allows for a concise representation of the data content, and consequently reduces the likelihood of format errors.

[0077] Next, the entity information 431b contained in the second embedded 431 functions as a prompt that guides the data decoder 450, which decodes data from the AI ​​model 400, to output an embedded containing raw data. When the entity information 431b is input to the data encoder as the third embedded 441, a fourth embedded 451 containing raw data for each entity is output. At this time, the fourth embedded 451 generates the raw data in the format [x1|y1 &; x2|y2 &; ... xn|yn &; ], but separates and independently represents the raw data for each entity group. For example, when using the chart 410a shown in Figure 3, the raw data for Model 1 ([1|10 &; 3|30 &; 5|50 &... ]) is represented as shown in embedded 451a. This represents the ]) and, independently of this, represents the raw data for Model 2 ([1|1 &; 2|4 &; 3|9 &; 4|16 &; 5|25 &]).

[0078] When comparing the data format output when chart 410a is input using the present invention with that of a conventional generative model, the following differences can be observed. In the conventional generative model, when the Epoch value in chart 410a is 2, Model 2 shows a value of "4," but Model 1 shows no value. To indicate this, Model 2 inputs a value of "4," while Model 1's value is entered as "nan" to represent it as empty (see Figure 6). In contrast, the present invention expresses the output embedded 451a in a format that does not define at all when the Epoch value is 2 for data related to Model 1, because the raw data is represented independently for each entity group. In other words, the present invention can avoid outputting "nan" values, which have a high error frequency during the system processing process.

[0079] This invention uses a data format that represents each entity group independently, eliminating the need to input difficult-to-process "nan" values ​​for empty data values, minimizing formatting errors that occur when "nan" values ​​cannot be identified, and eliminating the need to assign tokens to "nan" values, thereby optimizing token usage.

[0080] Here, the present invention can use a Singularized Number Embedding (SNE) method as a method for representing numbers in the fourth embedded 451 (this is in contrast to conventional generative models that treat numbers as text). SNE is a method in which each number contained in an entity is represented as a single token within the data decoder 450 that decodes the data in the AI ​​model 400. <num>This means tokenizing each element and using those tokens to output a number to the fourth embedded 451.

[0081] A single tokenized token ( <num>A multi-layer perceptron (MLP) can be used as a method for extracting numbers using ). Here, an MLP consists of an input layer, one or more hiding layers, and an output layer, where each layer contains weights and an activation function. The weights used in the MLP used in this invention can be pre-optimized through a learning algorithm so that they can predict nonlinear patterns and relationships.

[0082] For example, as shown in Figure 4, when the number "-0412920" is embedded through SNE, the data decoder 450, which decodes the data in the AI ​​model 400, converts each of the numbers contained in the entity into a single token. <num>Each 451b is tokenized, and the tokens 451b are input to the MLP to be represented numerically on the fourth embedded 451.

[0083] This invention is expected to further improve performance by facilitating the separation of tasks between text comprehension and data extraction, and by reducing the number of tokens used to represent numbers, enabling efficient learning and prediction within the model. <num>By repeatedly using tokens, the inference speed can be significantly improved.

[0084] On the other hand, the present invention can further improve inference speed by inputting a predefined iterative template of tokens used in SNE and outputting a number.

[0085] Specifically, referring to Figure 5a, conventional language models treated digits as text and used an autoregressive method that predicted each token sequentially. This method infers digits by sequentially supplying tokens at time stage t to the model and generating output until an EOS (End of Sentence) token is reached. As explained earlier, this method suffers from the problem of wasted tokens and reduced inference speed.

[0086] This problem can be solved by using a Single Token through an SNE, as shown in Figure 5b. <num>The results can be partially improved by representing the numbers using an automated regression method.

[0087] Here, as shown in Figure 5c, token <num>When inferring numbers through this, multiple tokens are used in a predefined iterative template. <num>Performance can be further improved by adding a non-autonomous regression method that infers multiple numbers at once by inputting them all at once (at this time, tokens are used via an iterative template). <num>Enter the following, and discard any output after the first output of the EOS token.

[0088] This invention provides a sufficient length of " <num>By supplying tokens in advance, the need for automatic regression transmission can be minimized, and the inference speed can be significantly improved.

[0089] In this way, through the processing process using the AI ​​model, it is possible to ultimately obtain a data format 460a in which metadata is recorded, and a data format 460b in which data information contained in the fourth embedded is recorded.

[0090] On the other hand, the method for extracting metadata and data information from a chart according to an embodiment of the present invention can be carried out by the system described with reference to Figure 3.

[0091] An AI model according to an embodiment of the present invention can be controlled, executed, learned, driven, etc., by a processor, thereby enabling the AI ​​model to perform at least one of the tasks of execution, learning, and driving by at least one processor. Furthermore, the AI ​​model can be stored in memory, and the feature data according to the present invention can also be stored in memory.

[0092] On the other hand, the disclosed embodiments can be implemented in the form of a recording medium for storing computer-executable instructions. These instructions can be stored in the form of program code, which, when executed by a processor, can generate program modules to perform the operations of the disclosed embodiments. The recording medium can be implemented as a computer-readable recording medium.

[0093] Computer-readable storage media include all types of storage media that store instructions that can be deciphered by a computer. Examples include ROM (Read Only Memory), RAM (Random Access Memory), magnetic tape, magnetic disks, flash memory, and optical data storage devices.

[0094] The embodiments disclosed above have been described with reference to the attached drawings. Those with ordinary skill in the art to which this disclosure belongs will understand that the disclosure may be carried out in ways different from the disclosed embodiments without altering the technical idea or essential features of the disclosure. The disclosed embodiments are illustrative and should not be constrained.< / num> < / num> < / num> < / num> < / num> < / num> < / num> < / num> < / num> < / num> < / num>

Claims

1. A system for implementing an AI model that extracts metadata and data information contained in a chart, At least one processor, The system comprises at least one memory that stores instructions or information for causing the at least one processor to perform an operation, The operation performed by the aforementioned instruction is: The steps include inputting the chart into an image encoder and converting it into a first embedded data that can be processed by the AI ​​model, The steps include inputting the first embedded into the AI ​​model, outputting a second embedded containing the metadata from the first embedded, and outputting a fourth embedded containing the data information from a third embedded containing information about entities included in the second embedded, A system comprising the steps of outputting a first data format in which the metadata contained in the second embedded is recorded, and a second data format in which the data information contained in the fourth embedded is recorded.

2. In the system described in claim 1, The data information contained in the fourth embedded is separated for each group of entities in the system.

3. In the system described in claim 1, The aforementioned metadata is a system that includes the chart title, axis names, and entity group names included in the legend.

4. In the system described in claim 1, The aforementioned data information includes numerical information contained in the chart, and is part of a system.

5. In the system described in claim 1, The aforementioned data information is tokenized into a single token for each piece of data information and included in the fourth embedded, A system that, when recording the data information contained in the fourth embedded in a second data format, extracts the tokenized data information from the single token and records it in the second data format.

6. In the system described in claim 4, A system for extracting data information from the aforementioned single token using a multilayer perceptron (MLP).

7. In the system according to claim 5 or 6, A system that extracts data information from a single token by inputting a predefined iterative template to the single token, thereby simultaneously extracting multiple pieces of data information.

8. A method for extracting metadata and data information contained in a chart. The aforementioned chart is input to an image encoder and converted into a first embedded data that can be processed by the AI ​​model. The first embedded is input to the AI ​​model, and a second embedded containing the metadata is output from the first embedded. From the third embedded which contains information about entities included in the second embedded, a fourth embedded which contains data information is output. A method for outputting a first data format in which the metadata contained in the second embedded is recorded, and a second data format in which the data information contained in the fourth embedded is recorded.

9. In the method described in claim 8, A method wherein the data information contained in the fourth embedded is distinguished for each group of entities.

10. In the method described in claim 8, The method wherein the metadata includes the title of the chart, the names of the axes, and the names of the entity groups included in the legend.

11. In the method described in claim 8, The data information includes numerical information contained in the chart, and the method is as follows.

12. In the method described in claim 8, The aforementioned data information is tokenized into a single token for each piece of data information and included in the fourth embedded, A method for recording the data information contained in the fourth embedded into a second data format, wherein the tokenized data information is extracted from the single token and recorded in the second data format.

13. In the method according to claim 12, A method for extracting data information from a single token using a multilayer perceptron (MLP).

14. In the method according to claim 12 or 13, A method for extracting data information from a single token, wherein the single token is input to a predefined iterative template to simultaneously extract multiple pieces of data information.

15. It is a program, A program, stored on a computer-readable recording medium, which is coupled to a computer and used to perform the method according to any one of claims 8 to 14.

Images