39 results about "Table (information)" patented technology

Arrangements for partition handling for multi table scan semi join operations are provided. Partition information of a first logical relation may be determined. Partition information of a first physical operator may be determined based on the partition information of the first logical relation. The partition information of the first physical operator may be stored in a data structure which includes partition information of a fact table and of each reducer in a predefined sequence. Partition information of a second logical relation may be determined. The partition information of a second physical operator may be determined based on the partition information of the second logical relation and the partition information of the first physical operator. The partition information of the second physical operator may be stored in the data structure. Execution of the second physical operator may be performed using the stored partition information, skipping mappings between pairs of partitions.

We provide the system. [Solution] Information processing means for receiving old and new data, A text extraction means for extracting text data from old and new documents, A difference detection means that compares the text data of old and new documents and detects the differences, A table generation means that classifies the detected differences and generates a revised table, An output means for outputting the generated current revised table, A system that includes this.

The invention discloses a table biaxial position coding and task loss function combined large model table question and answer method. Collecting table, question and answer data, and performing structuralization and biaxial position coding processing on the table data to obtain a text word vector set and biaxial position codes; performing fusion processing to obtain a position code after biaxial fusion; after combination processing, sequentially inputting into a model for processing to obtain a table global representation vector; and establishing a loss function training model, and performing prediction processing on an actual application scene by using the trained model to obtain a table answer result. According to the method, through double-axis position coding, the two-way constraint relation between rows and columns is effectively captured, representation is made to better conform to the inherent structure of a table, through mask cell modeling, known data can be learned, missing information can be speculated, the incomplete data processing capacity in a real scene is improved, and the method is suitable for being popularized and applied. The accuracy and robustness in complex table data processing can be remarkably improved.

The application discloses a table information extraction method, system and device based on a compressed pre-training language model. The application models the table information extraction as a reading comprehension problem and inputs the pre-training language model, thereby avoiding error propagation and reducing the participation of rules. Meanwhile, the strong generalization ability and robustness of the pre-training language model can obtain relatively accurate cells in the table that can answer natural language questions, thereby completing the extraction of the associated information in the table.

This invention relates to a table-based question-and-answer method and apparatus. The table-based question-and-answer method includes: structuring a table in a technical standard document to obtain table information; receiving an input user question and processing it through a preset model to obtain an SQL statement; and querying the answer corresponding to the user question based on the SQL statement and the table information.

This application relates to the technical fields of data processing and artificial intelligence, and discloses a method and apparatus for evaluating and processing retrieved tabular documents. The method includes: parsing the tabular document to obtain at least its table structure information, data content, and metadata information; performing multi-dimensional evaluation based on the table structure information, data content, and metadata information to obtain at least two of the following: table standardization evaluation result, metadata integrity evaluation result, data clarity evaluation result, and contextual relevance evaluation result; and weighting at least two of these results to obtain a comprehensive evaluation result for the tabular document. This method allows for a comprehensive evaluation of tabular documents, providing users with clear optimization guidance.

This invention relates to the field of computer vision technology and discloses a method, apparatus, device, and readable storage medium for table recognition. The method includes: acquiring an image to be recognized; inputting the image to be recognized into a preset table detection model to identify a table image from the image; inputting the table image into the preset table recognition model to determine the endpoint coordinates of each cell in the table image and the connection relationships between the endpoints; and generating a target table based on the endpoint coordinates and connection relationships. By implementing this invention, table recognition is achieved through endpoint extraction and endpoint connection relationship extraction, eliminating reliance on table line extraction, ensuring the accuracy of table structure recognition, and improving the accuracy of table information recognition.

The application provides a big data SQL join intelligent optimization method and device, and belongs to the technical field of SQL optimization application. The method comprises the following steps: S1: collecting and recording user SQL information; S2: analyzing the recorded SQL information to obtain key information of the SQL, wherein the key information comprises a table name, a table partition and a join field; S3: reading fsimage information according to the table name and the table partition through a presto engine to obtain the actual size of the current used table partition in the fsimage information; S4: judging whether the SQL join order is arranged according to a preset order, if not, adjusting the SQL join order to the preset order according to the size of the table partition, and if yes, continuing to execute downwards; and S5: judging whether data skew is caused according to the join field, and if yes, confirming the SQL after rewriting. The application improves the running speed of big data SQL, saves the resources of a big data cluster, and solves the problem of insufficient experience of engineers and the problem of not knowing how to optimize SQL.

The application discloses a construction method table identification method and device, computer equipment and a storage medium, and relates to the technical field of table identification. The method comprises the following steps: acquiring a construction method table, extracting text rows from the construction method table; performing semantic classification on the text rows to determine the field types of the text rows; dividing all method content fields into at least one method block based on semantics; and determining the method number fields, method name fields and applicable part fields matched with the method block based on the position information of the method block, the method number fields, the method name fields and the applicable part fields in the construction method table. The application overturns the traditional table identification process of "restoring the physical structure first, and then extracting the content according to the columns", and breaks through the dependence of traditional table identification on the physical structure through the "semantic aggregation + spatial reverse matching based on the method block" strategy. The "semantic aggregation + spatial positioning" method significantly improves the field recognition accuracy for complex construction method tables.

The invention provides a complex table recognition method and system, and relates to the field of computer vision, and the method comprises the steps: recognizing a table contour in a table image based on a plurality of detection methods to obtain a plurality of recognition results, and fusing the recognition results based on a preset rule to obtain a table bounding box; detecting trunk separation elements capable of segmenting table contents based on multiple image processing methods, and dividing the table into one or more block regions by synthesizing the trunk separation elements obtained by the multiple image processing methods; carrying out independent and parallel analysis processing on the block regions to obtain all cells, and identifying the text content of the cells in each block region by means of an OCR algorithm to obtain the text content; and establishing table hierarchical structure information according to the relative positions of the block regions, and recombining the text content according to the table hierarchical structure information to obtain a table recognition result. A table containing any complex row-crossing and column-crossing merged cells can be accurately identified and reconstructed.

This application discloses a table generation method, apparatus, electronic device, and medium, relating to the field of data processing technology. A specific implementation of the method includes: modifying information in a table to be modified based on preset change rules to obtain a table template file containing at least one variable placeholder; generating a target mapping table file based on at least one variable placeholder and at least one set of actual variable information, the target mapping table file being used to establish a variable mapping relationship between at least one variable placeholder and at least one actual variable in each set of actual variable information; and performing at least one variable substitution operation on the table template file based on at least one set of variable mapping relationships in the target mapping table file to obtain at least one target information table. This achieves automated, standardized, and batch generation of information tables, significantly improving the efficiency and standardization of maintenance document generation, while balancing the flexibility, maintainability, and low-cost requirements of table generation.

The invention discloses a data configuration method and device, a data access method and device, equipment and a storage medium, and relates to the technical field of computers and the Internet. The method comprises the steps of displaying a table configuration interface of a table tool; in response to an operation for a new table control, displaying identification information of a new first table, identification information of a first record contained in the first table, and an attribute configuration interface of the first record; in response to an operation for the attribute configuration interface, configuring at least one self-defined attribute contained in the first record and an attribute parameter corresponding to the at least one self-defined attribute in the attribute configuration interface; in response to the operation of adding the record in the first table, displaying at least one other record, except the first record, added in the first table in the table configuration interface; the data in the first table supports access by a visual programming tool. According to the method, the process of configuring and accessing the data is simplified.

Embodiments of the present disclosure disclose data archiving method, device, electronic equipment, medium and program product. A specific embodiment of the method comprises: performing a main table archiving task corresponding to main table archiving task information to generate main table data to be archived, wherein the main table corresponding to the main table archiving task information has a field association relationship with a set of sub-tables; generating at least one sub-table archiving task information corresponding to at least one target sub-table in the set of sub-tables; for each archiving task information in the set of archiving task information, the following consumption steps are performed: generating an archiving message group for the archiving task information; performing message consumption on the archiving message group to obtain a consumption result group; and generating archiving information for the main table and the at least one target sub-table according to the obtained set of consumption result groups. This embodiment is related to data archiving, and utilizes the generation and consumption of messages to perform data archiving on the data in the main table and the plurality of sub-tables, thereby improving the completeness and archiving efficiency of data archiving.

The invention relates to a document processing method and device, electronic equipment and a storage medium. The method comprises the steps of positioning a class table structure text region from a to-be-processed document; key value pair structure information in the class table structure text area is identified; and based on the key value pair structure information, carrying out resetting processing on the class table structure text region. Therefore, the problems that a traditional typesetting mode is low in efficiency, poor in adaptability, unstable in typesetting effect and the like can be effectively solved, and a complete, efficient and reliable automatic solution is provided for processing the content of the class table structure in the streaming document.

The invention discloses a table processing method and device, a storage medium and a product, and relates to the technical field of table processing, a to-be-processed table file and a multi-task cue word are input into a preset large model, the preset large model is guided to complete table content recognition, table cross-page judgment and table merging scoring, and finally a table merging strategy is generated. In the embodiment of the invention, the to-be-processed table file is accurately identified through the preset large model, and the table structure and the table content of the table are obtained. Inputting the table structured information of each page number into a preset large model to obtain table multi-dimensional similarity of adjacent page numbers, obtaining a merging score of a cross-page table according to row and column similarity, semantic similarity and value domain similarity of adjacent tables, and judging whether merging is needed or not according to the merging score and a preset merging threshold value; therefore, the recognition and combination accuracy of the cross-page table is improved.

The embodiment of the application provides a kind of text and table semantic interaction data processing method and device, equipment, storage medium, belong to the field of artificial intelligence technology.The method comprises: obtaining input data comprising text information and table information;Text information of input data is extracted to obtain text features, and table information of input data is extracted to obtain table features;Text features and table features are input into the embedding layer of preset neural network model, so that the embedding layer converts text features and table features to obtain token embedding, position embedding and token type embedding;Each embedding item converted by the embedding layer is input into the transformer layer of preset neural network to obtain mixed semantic vector;Mixed semantic vector is input into the classification layer of preset neural network to obtain the prediction probability information of text and table semantics corresponding to input data.Uniform conversion is carried out on text features and table features by the embedding layer, and deep semantic interaction of text and table is realized in the transformer layer.

Embodiments of the present application provide a DDL statement processing method and device, electronic equipment and storage medium. In the embodiments of the present application, the optimized DDL statement is relatively shorter than the to-be-processed DDL statement before optimization. For syntax analysis of the optimized DDL statement, only the statement segment related to the description of the library table name information is subjected to syntax analysis, and the existing table structure information is reused, thereby greatly reducing the syntax analysis difficulty of the DDL statement and improving the analysis performance of the DDL statement. Especially in the SaaS service scenario or the library and table separation scenario, a large number of DDL statements triggered in a short time can be effectively coped with, the syntax analysis performance of the DDL statement is greatly improved, and a basis is provided for better concurrent processing of DDL statements by DTS.

This invention discloses a Text-to-SQL method and system based on hierarchical multi-candidate SQL scoring. In an online state, it receives query questions and performs table retrieval; it then performs mSchema synthesis, which includes table structure information, sample data of table fields, and foreign key relationships between tables; it selects m reference examples, using one example at a time, and generates n SQL statements using a large model, resulting in a total of m*n SQL statements. The temperature parameter of the large model is set to 1, and the sampling parameter is set to n. A first-level scoring method filters some SQL statements; a second-level scoring method filters others; a third-level scoring method obtains the cumulative score for each SQL statement; the scores of each SQL statement are summed based on the scores obtained in the previous step to obtain the total score for each SQL statement; finally, the SQL statement with the highest score is selected as the final SQL statement, and the result is returned. This invention can lower the data query threshold and improve the accuracy and scenario adaptability of Text-to-SQL generation.

This application provides a data processing method, apparatus, electronic device, storage medium, and program product, solving the technical problem that related technologies cannot meet the needs of complex table processing. The data processing method includes: obtaining metadata of a table file to be analyzed; determining the first-level result of the table file to be analyzed based on the metadata; the first-level result being single-level table data; and determining a table analysis report of the table file to be analyzed based on intent information and the first-level result. This application, based on metadata, flattens multi-level table files to be analyzed into single-level tables, fully preserving the original row and column indexes and content. It can adapt to complex tables, eliminates the need for repeated rule development, and avoids data loss and misinterpretation to improve accuracy. Combining intent information with the selection of analysis tools shortens processing time and improves efficiency. Therefore, it meets the needs of complex table processing.

The application belongs to the technical field of network information processing, and specifically provides a kind of html document attribute information disambiguation method and system, wherein the method comprises: html document is parsed into text data and table data and is stored by XPath;Attribute key and information value are obtained by using rule extraction or model extraction for line-by-line processing;Attribute key in text data and table data is disambiguated using context information respectively.The scheme converts html document into chapter and table, and then disambiguates text and table using some context information, improving the accuracy of information extraction.

The invention provides a log analysis method of a data link, a controller, a medium and a product, and is applied to the technical field of data links, and the method comprises the following steps: obtaining a text log and a plurality of table logs; traversing the text log to obtain text log information and a text time element time frame time slot corresponding to the text log information, and storing the text log information and the text time element time frame time slot to obtain a text dictionary; traversing each table log to obtain corresponding table log information and a table time element, time frame and time slot corresponding to the table log information, and storing the table log information and the table time element, time frame and time slot to obtain a table dictionary; and performing traversal analysis on the text dictionary and the table dictionary to determine an abnormal condition. According to the method, the text dictionary and the table dictionary which are obtained by traversing the text log and the table logs are traversed, so that the determined abnormal condition is obtained, and log analysis of the data stream is realized.

This invention provides an efficient method for extracting tables from journal articles based on keyword and border features, belonging to the field of PDF file information extraction. The method involves constructing a table data structure based on the border and character characteristics of journal tables. The table data structure consists of two parts: keywords and table borders. Keyword cleaning rules and table border cleaning rules are constructed separately. The top-left corner coordinates of the journal table area are calculated using keywords, and the bottom-right corner coordinates are calculated using table borders to accurately locate the table area. Finally, the table data is output. This method is specifically designed for tables with few borders in journal articles and is fully applicable to both Chinese and English literature. It can achieve fully automated table data extraction and ensure the accuracy of the logical relationships within the table data. Further filtering of tables in related fields can be achieved by adding keywords, making it widely applicable.

The invention discloses a large language model fact checking system and method based on text-table knowledge, which effectively improves the fact checking capability of a large language model in a complex structured data scene through an improved GraphRAG architecture by combining information retrieval and a knowledge graph construction technology. The method comprises the following steps: firstly, carrying out coarse-grained retrieval on an input statement by utilizing a BM25 algorithm to obtain a document set related to the input statement; then segmenting text paragraphs and table data in the document into information blocks, and extracting entities and relationships through a large language model to construct a knowledge graph; then, performing semantic matching on entity relationships in the knowledge graph to obtain related evidence information, and performing comprehensive reasoning in combination with an original text; finally, the authenticity of the statement is judged through the large language model, and a checking result and explanation are returned. According to the method, the problem that an existing fact checking system is difficult to process text-table mixed data is solved, the semantic recall ratio and the reasoning accuracy are improved, and the method has good universality and expandability.

The application discloses a table recognition method and device and related equipment, and relates to the technical field of artificial intelligence, to solve the problem of low accuracy of table recognition results. The method comprises the following steps: obtaining a first target image, a second target image, a third target image and a fourth target image corresponding to a first table image based on a pre-trained semantic segmentation model; obtaining table structure information and text information corresponding to the first table image based on the first target image, the second target image, the third target image and the fourth target image; the first target image comprises image information representing horizontal frame lines of the first table image, the second target image comprises image information representing vertical frame lines of the first table image, the third target image comprises image information representing horizontal edges of text of the first table image, and the fourth target image comprises image information representing vertical edges of text of the first table image. The embodiment of the application can improve the accuracy of table recognition structure.

The invention discloses a complex table text conversion method based on a tree structure. The method comprises the following steps of: firstly, converting a table image into structured JSON (JavaScript Object Notation) data; then, performing type judgment on cells in the JSON data by utilizing a prompt word normal form of LLM, and supplementing cell type information; performing row / column position matching and connectivity verification according to two basic principles of a title-data relationship and a row-column position relationship, thereby automatically constructing two subtrees of a row tree and a column tree; and finally, performing row and / or column-to-dimension text conversion based on a sub-tree structure by utilizing a tree traversal strategy assisted by a queue and a stack. The problem that a traditional table serialization mode cannot process hierarchical tables, merge cells and nesting relations is solved, and meanwhile the problem that a table conversion mode does not consider content semantic relations is solved.

Embodiments of the present application provide a table generation method and device based on multiple row headers, equipment and storage medium, belong to data processing technical field. The method comprises: obtaining multiple header fields, state information and association information, the state information is used to represent the column freezing state or column non-freezing state corresponding to the header field; determining a first header field in each of the header fields based on the state information; determining a target header field in each of the header fields based on the first header field and the association information; updating the state information of the target header field to make the column corresponding to the target header field freeze; constructing a header tree structure according to the header field and the association information; generating a table template according to the header tree structure and the state information. Embodiments of the present application can improve the table production efficiency of multiple row headers, and ensure the display effect.

This invention belongs to the field of data management and discloses a table processing method and apparatus based on an adaptive strategy. The method includes: acquiring table data and generating structured data based on the table data; determining the characteristics of the structured data and selecting a corresponding processing strategy based on the characteristics; and processing the structured data based on the processing strategy to obtain a corresponding metadata model. This invention dynamically selects an appropriate processing strategy based on the characteristics of the structured data, transforming the table data into a unified standard metadata model. This adaptive processing method can flexibly handle various types of tables without requiring customized processing for specific table structures, thus improving the efficiency of table processing. Simultaneously, this process reduces the problems of structure recognition errors and semantic information loss when dealing with complex tables, improving the accuracy of table processing and consequently improving the retrieval accuracy and recall rate of the knowledge base.

Embodiments of the present application provide a table recognition method and device, electronic equipment and readable storage medium, which are applied to the field of computer vision, and the method comprises the following steps: segmenting a table region from a target image to obtain a table image; using a pre-trained table recognition model to recognize the table structure and the position information of the cells in the table image; wherein the table recognition model comprises an encoding layer for extracting image features of the table image, and a decoding layer for generating the table structure; the input of the decoding layer is the input content of the encoding layer and the output content of the encoding layer; the text region of the table image and the text content in the text region are recognized; and an electronic spreadsheet file is generated based on the position information of the cells, the table structure, the text region and the text content in the text region. Through the present solution, the accuracy of table recognition can be improved.

An information processing system includes a processor configured to: accept, as a piece of teacher data, a piece of first table data and a piece of second table data between which results of matching indicate correctness; combine each of pieces of first column data in the piece of first table data and each of pieces of second column data in the piece of second table data, the each of the pieces of first column data and the each of the pieces of second column data being identical to each other in data type, to generate a piece of second teacher data; and use a model having undergone learning, the model being generated from the piece of second teacher data, to present candidates for associating each of the pieces of first column data and each of the pieces of second column data with each other.

The application discloses a kind of multi-modal financial document's event extraction method and system, belong to data processing technical field, the method includes: obtaining to be extracted area and its object, the object includes table and text;The table is parsed, obtains the content of table, the content includes event and argument;According to the event filter of the table, the text is filtered, obtains relevant sentence;Argument role and / or supplementary argument are extracted from the relevant sentence.It is relatively easy to extract information from semi-structured table, then use the information obtained from table to assist and improve the information extraction from text, improve extraction efficiency and reliability;Information extracted in table and text can be supplemented each other, obtain more complete information extraction result.