55 results about "Table (information)" patented technology

The invention provides a form text boundary adaptive information extraction method and system in the technical field of computer information processing. The method comprises the following steps: S1, identifying a title frame from a form image; s2, calculating the aspect ratio of the title frame to determine the title arrangement direction of the table titles; s3, detecting a text detection box of the table image, and performing classification prediction of a text arrangement direction on the text detection box by combining a text direction model with a title arrangement direction; s4, controlling affine transformation of the table image through the text arrangement direction and the text detection box to obtain a corrected image; s5, identifying a table title in the corrected image, and matching a table type from the table type configuration file based on the table title; s6, extracting keywords from the corrected image based on the table type; and S7, processing the keyword to extract table information. The method has the advantages that the adaptability, accuracy and efficiency of table information extraction are greatly improved.

The application discloses a method for automatically inferring semantics of a data cleaning script, comprising loading an original table to the data cleaning script and obtaining an input table and an output table; constructing a table change space, obtaining a table change characteristic set based on comparison between the input / output table and the table change space; taking data conversion operation types that all satisfy inevitable occurrence, impossible occurrence and possible occurrence as candidate data conversion operation types; converting and splicing multiple descriptive texts corresponding to function names, function parameters and table change characteristics to obtain a semantic vector, inputting the semantic vector into a twin convolutional neural network model to obtain candidate data conversion operation type ranking; constructing a parameter slot of each data conversion operation type, inferring the parameter slot of the candidate data conversion operation type by using data cleaning code, table content and table change information of a corresponding row, and obtaining the type and parameters of the data conversion operation based on the inference result. The method can simply and conveniently infer the type and parameters of the data conversion operation in the cleaning code.

Arrangements for partition handling for multi table scan semi join operations are provided. Partition information of a first logical relation may be determined. Partition information of a first physical operator may be determined based on the partition information of the first logical relation. The partition information of the first physical operator may be stored in a data structure which includes partition information of a fact table and of each reducer in a predefined sequence. Partition information of a second logical relation may be determined. The partition information of a second physical operator may be determined based on the partition information of the second logical relation and the partition information of the first physical operator. The partition information of the second physical operator may be stored in the data structure. Execution of the second physical operator may be performed using the stored partition information, skipping mappings between pairs of partitions.

We provide the system. [Solution] Information processing means for receiving old and new data, A text extraction means for extracting text data from old and new documents, A difference detection means that compares the text data of old and new documents and detects the differences, A table generation means that classifies the detected differences and generates a revised table, An output means for outputting the generated current revised table, A system that includes this.

The invention discloses a table biaxial position coding and task loss function combined large model table question and answer method. Collecting table, question and answer data, and performing structuralization and biaxial position coding processing on the table data to obtain a text word vector set and biaxial position codes; performing fusion processing to obtain a position code after biaxial fusion; after combination processing, sequentially inputting into a model for processing to obtain a table global representation vector; and establishing a loss function training model, and performing prediction processing on an actual application scene by using the trained model to obtain a table answer result. According to the method, through double-axis position coding, the two-way constraint relation between rows and columns is effectively captured, representation is made to better conform to the inherent structure of a table, through mask cell modeling, known data can be learned, missing information can be speculated, the incomplete data processing capacity in a real scene is improved, and the method is suitable for being popularized and applied. The accuracy and robustness in complex table data processing can be remarkably improved.

The application discloses a table information extraction method, system and device based on a compressed pre-training language model. The application models the table information extraction as a reading comprehension problem and inputs the pre-training language model, thereby avoiding error propagation and reducing the participation of rules. Meanwhile, the strong generalization ability and robustness of the pre-training language model can obtain relatively accurate cells in the table that can answer natural language questions, thereby completing the extraction of the associated information in the table.

The invention discloses a table structure extraction method based on a deep cascade network, and the method comprises the steps: obtaining a table image, extracting the visual features of the table image, coding the visual features to obtain enhanced features, and obtaining target query information according to the enhanced features; according to target query information, decoding the enhanced features to obtain a prediction result of a text region bounding box; decoding the prediction result of the text region bounding box to obtain a prediction result of a cell region bounding box; and performing logical position allocation according to a prediction result of the cell region bounding box to obtain table structure content.

This invention relates to a table-based question-and-answer method and apparatus. The table-based question-and-answer method includes: structuring a table in a technical standard document to obtain table information; receiving an input user question and processing it through a preset model to obtain an SQL statement; and querying the answer corresponding to the user question based on the SQL statement and the table information.

This application relates to the technical fields of data processing and artificial intelligence, and discloses a method and apparatus for evaluating and processing retrieved tabular documents. The method includes: parsing the tabular document to obtain at least its table structure information, data content, and metadata information; performing multi-dimensional evaluation based on the table structure information, data content, and metadata information to obtain at least two of the following: table standardization evaluation result, metadata integrity evaluation result, data clarity evaluation result, and contextual relevance evaluation result; and weighting at least two of these results to obtain a comprehensive evaluation result for the tabular document. This method allows for a comprehensive evaluation of tabular documents, providing users with clear optimization guidance.

The invention discloses a complex table structure understanding and information extraction method, system and device and a medium, belongs to the cross technical field of artificial intelligence, computer vision and natural language processing, and aims to solve the technical problem of how to realize accurate understanding and information extraction of a complex table structure. According to the technical scheme, the method comprises the following steps: multi-modal feature extraction: representing table data as a combination of a visual feature vector, a text semantic vector and a layout feature vector, namely, a multi-modal table vector = the visual feature vector, the text semantic vector and the layout feature vector; multi-modal fusion: generating a multi-modal table vector based on the multi-modal large model architecture, performing efficient processing and fusion on the multi-modal table vector, and fully fusing multi-dimensional information of a visual feature vector, a text semantic vector and a layout feature vector; structure perception decoding and semantic enhancement reconstruction are carried out; an adaptive output is generated.

The invention provides a code generation method and system based on a table structure, a server and a medium, and relates to the technical field of computers.The method comprises the steps that table structure analysis is conducted on a data table in a preset data source, and table structure meta-information is obtained; performing field semantic recognition on the table structure meta-information according to a preset rule base, and determining a front-end control type and a back-end verification rule; according to a preset generation strategy, table structure meta-information, a front-end control type and a back-end verification rule, calling a code template to execute rendering processing, and generating an original code; and carrying out front-end and rear-end interface binding on the original code to generate a target code. The project development efficiency and the code quality can be effectively improved.

This invention relates to the field of computer vision technology and discloses a method, apparatus, device, and readable storage medium for table recognition. The method includes: acquiring an image to be recognized; inputting the image to be recognized into a preset table detection model to identify a table image from the image; inputting the table image into the preset table recognition model to determine the endpoint coordinates of each cell in the table image and the connection relationships between the endpoints; and generating a target table based on the endpoint coordinates and connection relationships. By implementing this invention, table recognition is achieved through endpoint extraction and endpoint connection relationship extraction, eliminating reliance on table line extraction, ensuring the accuracy of table structure recognition, and improving the accuracy of table information recognition.

The application provides a big data SQL join intelligent optimization method and device, and belongs to the technical field of SQL optimization application. The method comprises the following steps: S1: collecting and recording user SQL information; S2: analyzing the recorded SQL information to obtain key information of the SQL, wherein the key information comprises a table name, a table partition and a join field; S3: reading fsimage information according to the table name and the table partition through a presto engine to obtain the actual size of the current used table partition in the fsimage information; S4: judging whether the SQL join order is arranged according to a preset order, if not, adjusting the SQL join order to the preset order according to the size of the table partition, and if yes, continuing to execute downwards; and S5: judging whether data skew is caused according to the join field, and if yes, confirming the SQL after rewriting. The application improves the running speed of big data SQL, saves the resources of a big data cluster, and solves the problem of insufficient experience of engineers and the problem of not knowing how to optimize SQL.

The application discloses a construction method table identification method and device, computer equipment and a storage medium, and relates to the technical field of table identification. The method comprises the following steps: acquiring a construction method table, extracting text rows from the construction method table; performing semantic classification on the text rows to determine the field types of the text rows; dividing all method content fields into at least one method block based on semantics; and determining the method number fields, method name fields and applicable part fields matched with the method block based on the position information of the method block, the method number fields, the method name fields and the applicable part fields in the construction method table. The application overturns the traditional table identification process of "restoring the physical structure first, and then extracting the content according to the columns", and breaks through the dependence of traditional table identification on the physical structure through the "semantic aggregation + spatial reverse matching based on the method block" strategy. The "semantic aggregation + spatial positioning" method significantly improves the field recognition accuracy for complex construction method tables.

The invention provides a complex table recognition method and system, and relates to the field of computer vision, and the method comprises the steps: recognizing a table contour in a table image based on a plurality of detection methods to obtain a plurality of recognition results, and fusing the recognition results based on a preset rule to obtain a table bounding box; detecting trunk separation elements capable of segmenting table contents based on multiple image processing methods, and dividing the table into one or more block regions by synthesizing the trunk separation elements obtained by the multiple image processing methods; carrying out independent and parallel analysis processing on the block regions to obtain all cells, and identifying the text content of the cells in each block region by means of an OCR algorithm to obtain the text content; and establishing table hierarchical structure information according to the relative positions of the block regions, and recombining the text content according to the table hierarchical structure information to obtain a table recognition result. A table containing any complex row-crossing and column-crossing merged cells can be accurately identified and reconstructed.

The invention relates to a table model generation method and device, computer equipment, a readable storage medium and a program product. The method comprises the steps of determining a table weight of each initial table in an initial table model based on a table structure relationship of the initial table model; based on a table weight and field weight calculation strategy, determining weight configuration information of each table field in each initial table; and generating a target table model based on a preset table model generation model, the weight configuration information and a table field of the to-be-generated table model. By adopting the method, the generation efficiency of the table model can be improved.

This application discloses a table generation method, apparatus, electronic device, and medium, relating to the field of data processing technology. A specific implementation of the method includes: modifying information in a table to be modified based on preset change rules to obtain a table template file containing at least one variable placeholder; generating a target mapping table file based on at least one variable placeholder and at least one set of actual variable information, the target mapping table file being used to establish a variable mapping relationship between at least one variable placeholder and at least one actual variable in each set of actual variable information; and performing at least one variable substitution operation on the table template file based on at least one set of variable mapping relationships in the target mapping table file to obtain at least one target information table. This achieves automated, standardized, and batch generation of information tables, significantly improving the efficiency and standardization of maintenance document generation, while balancing the flexibility, maintainability, and low-cost requirements of table generation.

The invention discloses a data configuration method and device, a data access method and device, equipment and a storage medium, and relates to the technical field of computers and the Internet. The method comprises the steps of displaying a table configuration interface of a table tool; in response to an operation for a new table control, displaying identification information of a new first table, identification information of a first record contained in the first table, and an attribute configuration interface of the first record; in response to an operation for the attribute configuration interface, configuring at least one self-defined attribute contained in the first record and an attribute parameter corresponding to the at least one self-defined attribute in the attribute configuration interface; in response to the operation of adding the record in the first table, displaying at least one other record, except the first record, added in the first table in the table configuration interface; the data in the first table supports access by a visual programming tool. According to the method, the process of configuring and accessing the data is simplified.

Embodiments of the present disclosure disclose data archiving method, device, electronic equipment, medium and program product. A specific embodiment of the method comprises: performing a main table archiving task corresponding to main table archiving task information to generate main table data to be archived, wherein the main table corresponding to the main table archiving task information has a field association relationship with a set of sub-tables; generating at least one sub-table archiving task information corresponding to at least one target sub-table in the set of sub-tables; for each archiving task information in the set of archiving task information, the following consumption steps are performed: generating an archiving message group for the archiving task information; performing message consumption on the archiving message group to obtain a consumption result group; and generating archiving information for the main table and the at least one target sub-table according to the obtained set of consumption result groups. This embodiment is related to data archiving, and utilizes the generation and consumption of messages to perform data archiving on the data in the main table and the plurality of sub-tables, thereby improving the completeness and archiving efficiency of data archiving.

The invention relates to a document processing method and device, electronic equipment and a storage medium. The method comprises the steps of positioning a class table structure text region from a to-be-processed document; key value pair structure information in the class table structure text area is identified; and based on the key value pair structure information, carrying out resetting processing on the class table structure text region. Therefore, the problems that a traditional typesetting mode is low in efficiency, poor in adaptability, unstable in typesetting effect and the like can be effectively solved, and a complete, efficient and reliable automatic solution is provided for processing the content of the class table structure in the streaming document.

The application relates to the field of data processing and natural language processing, in particular to a table selection method and device based on multidimensional knowledge base retrieval and cross matching. The method comprises the following steps: inputting a query text; performing multidimensional knowledge base retrieval based on the query text and generating corresponding retrieval results; the multidimensional knowledge base retrieval comprises table dimension knowledge base retrieval, index dimension knowledge base retrieval and user information retrieval; inputting the retrieval results of the multidimensional knowledge base retrieval into a cross matching engine for cross matching; and outputting the results of the cross matching performed by the cross matching engine. The application innovatively combines the multidimensional knowledge base retrieval and the cross matching engine, compared with a traditional scheme, can accurately understand a natural language query containing business terms, can automatically avoid risks such as data authority violation, can reduce the threshold of complex data analysis to the level of natural language dialogue, and truly realizes the intelligent analysis concept of "letting data speak".

The invention discloses a table processing method and device, a storage medium and a product, and relates to the technical field of table processing, a to-be-processed table file and a multi-task cue word are input into a preset large model, the preset large model is guided to complete table content recognition, table cross-page judgment and table merging scoring, and finally a table merging strategy is generated. In the embodiment of the invention, the to-be-processed table file is accurately identified through the preset large model, and the table structure and the table content of the table are obtained. Inputting the table structured information of each page number into a preset large model to obtain table multi-dimensional similarity of adjacent page numbers, obtaining a merging score of a cross-page table according to row and column similarity, semantic similarity and value domain similarity of adjacent tables, and judging whether merging is needed or not according to the merging score and a preset merging threshold value; therefore, the recognition and combination accuracy of the cross-page table is improved.

The embodiment of the invention provides a table structure information reconstruction method and a related device, and the method comprises the steps: correcting a table intersection point coordinate extracted from a to-be-processed table image based on the position proportion of a standard table intersection point in a table line, and obtaining a corrected table image; carrying out OCR (Optical Character Recognition) on the corrected table image to obtain at least one textbox coordinate; performing cell position detection on the corrected table image to obtain at least one cell coordinate; and reconstructing table structure information based on the at least one cell coordinate and the at least one textbox coordinate to obtain reconstructed table structure information. In the embodiment of the invention, the table structure information is reconstructed based on the cell coordinates and the textbox coordinates of the corrected table image, and even if the table in the to-be-processed table image is distorted, the recognition accuracy can be ensured, so that the accuracy of the reconstructed table structure information is improved.

The application discloses a kind of flat detection method, device, method includes: timing trigger flat detection, for each processing instance, the actual table structure of each processing instance corresponding expected field information and target Hive table is acquired, wherein expected field information defines the field that target Hive table should contain and processing logic;The actual table structure of each processing instance corresponding expected field information and target Hive table is compared, and comparison result is obtained;According to comparison result, corresponding table structure change operation is executed;After table structure change operation succeeds, based on the SQL template of preestablished, expected field information and the calculation engine type specified by processing instance, processing SQL sentence is generated by replacing placeholder in SQL template;Processing SQL sentence is deployed to scheduling platform, to create or update corresponding data processing task.The application improves flat detection efficiency.

The embodiment of the application provides a kind of text and table semantic interaction data processing method and device, equipment, storage medium, belong to the field of artificial intelligence technology.The method comprises: obtaining input data comprising text information and table information;Text information of input data is extracted to obtain text features, and table information of input data is extracted to obtain table features;Text features and table features are input into the embedding layer of preset neural network model, so that the embedding layer converts text features and table features to obtain token embedding, position embedding and token type embedding;Each embedding item converted by the embedding layer is input into the transformer layer of preset neural network to obtain mixed semantic vector;Mixed semantic vector is input into the classification layer of preset neural network to obtain the prediction probability information of text and table semantics corresponding to input data.Uniform conversion is carried out on text features and table features by the embedding layer, and deep semantic interaction of text and table is realized in the transformer layer.

Embodiments of the present application provide a DDL statement processing method and device, electronic equipment and storage medium. In the embodiments of the present application, the optimized DDL statement is relatively shorter than the to-be-processed DDL statement before optimization. For syntax analysis of the optimized DDL statement, only the statement segment related to the description of the library table name information is subjected to syntax analysis, and the existing table structure information is reused, thereby greatly reducing the syntax analysis difficulty of the DDL statement and improving the analysis performance of the DDL statement. Especially in the SaaS service scenario or the library and table separation scenario, a large number of DDL statements triggered in a short time can be effectively coped with, the syntax analysis performance of the DDL statement is greatly improved, and a basis is provided for better concurrent processing of DDL statements by DTS.

This invention discloses a Text-to-SQL method and system based on hierarchical multi-candidate SQL scoring. In an online state, it receives query questions and performs table retrieval; it then performs mSchema synthesis, which includes table structure information, sample data of table fields, and foreign key relationships between tables; it selects m reference examples, using one example at a time, and generates n SQL statements using a large model, resulting in a total of m*n SQL statements. The temperature parameter of the large model is set to 1, and the sampling parameter is set to n. A first-level scoring method filters some SQL statements; a second-level scoring method filters others; a third-level scoring method obtains the cumulative score for each SQL statement; the scores of each SQL statement are summed based on the scores obtained in the previous step to obtain the total score for each SQL statement; finally, the SQL statement with the highest score is selected as the final SQL statement, and the result is returned. This invention can lower the data query threshold and improve the accuracy and scenario adaptability of Text-to-SQL generation.

This application provides a data processing method, apparatus, electronic device, storage medium, and program product, solving the technical problem that related technologies cannot meet the needs of complex table processing. The data processing method includes: obtaining metadata of a table file to be analyzed; determining the first-level result of the table file to be analyzed based on the metadata; the first-level result being single-level table data; and determining a table analysis report of the table file to be analyzed based on intent information and the first-level result. This application, based on metadata, flattens multi-level table files to be analyzed into single-level tables, fully preserving the original row and column indexes and content. It can adapt to complex tables, eliminates the need for repeated rule development, and avoids data loss and misinterpretation to improve accuracy. Combining intent information with the selection of analysis tools shortens processing time and improves efficiency. Therefore, it meets the needs of complex table processing.

The specification discloses a distributed table lock application method and device, a storage medium and an electronic device. The distributed table lock application method comprises the following steps: a master node receives a data processing request, determines a data table where to-be-processed data corresponding to the data processing request is located, generates a table lock for the data table, performs a lock operation on the data table, generates lock state information corresponding to the data table, writes the lock state information into a log in the master node, synchronizes the log to a slave node, so that the slave node saves the log, and after the master node restarts, judges whether the master node stores the log, and if yes, according to the lock state information recorded in the log, the lock operation on the data table is restored.

The application belongs to the technical field of network information processing, and specifically provides a kind of html document attribute information disambiguation method and system, wherein the method comprises: html document is parsed into text data and table data and is stored by XPath;Attribute key and information value are obtained by using rule extraction or model extraction for line-by-line processing;Attribute key in text data and table data is disambiguated using context information respectively.The scheme converts html document into chapter and table, and then disambiguates text and table using some context information, improving the accuracy of information extraction.