Support variable column excel file parsing method and system

An excel and variable column technology, applied in the field of data analysis, can solve problems such as OOM memory overflow, high memory usage, and slow parsing speed

Active Publication Date: 2021-09-10
上海微亿智造科技有限公司
View PDF8 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

It uses traditional POI technology for analysis, loads the entire file at one time, takes up a lot of memory, and when the file is large, the analysis speed is slow, and OOM memory overflow is prone to occur

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Support variable column excel file parsing method and system
  • Support variable column excel file parsing method and system
  • Support variable column excel file parsing method and system

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0046] Such as figure 1 As shown, the Excel file parsing method that supports variable columns includes the following steps:

[0047] Step 1: Verify the header of the Excel file to be analyzed and the abstracted basic template file, determine the type of product detection data in the Excel file, record the path of the Excel file to be analyzed and the type mapping relationship it belongs to, and send the mapping relationship To the message middleware MQ;

[0048] The basic template file includes a process-testing data basic template and a quality-testing data basic template, and the process-testing data basic template includes basic process testing information, and test results for the process, different products, production lines The test items that need to be performed will be different, so the number of test results is variable. This template is compatible with the input of test item information for different products and production lines. It is not necessary to issue a se...

Embodiment 2

[0063] Embodiment 2 can be regarded as a preferred example of Embodiment 1. The variable-column-supporting Excel file parsing system described in Embodiment 2 utilizes the steps of the variable-column-supporting Excel file parsing method described in Embodiment 1.

[0064] An Excel file parsing system that supports variable columns, including:

[0065] Mapping module: Verify the header of the Excel file to be analyzed and the basic template file, determine the type of product detection data in the Excel file, record the mapping relationship between the path of the Excel file to be analyzed and the type, and send the mapping relationship to In the message middleware MQ;

[0066] Loading module: the message middleware MQ consumer listens to the message, obtains the mapping relationship from the message, uses the type in it to load the pre-established parsing object java model according to the mapping relationship, and loads the Excel file corresponding to the file path ...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The present invention provides a method and system for parsing Excel files that support variable columns. The header of the Excel file to be parsed and the basic template file is verified, the type of product detection data is determined, and the mapping relationship between the file path and the type is recorded. , and send the mapping relationship to the message middleware MQ; the message middleware MQ consumer end listens to the message and obtains the mapping relationship, uses the type to load the pre-established parsing object java model, and loads the Excel file corresponding to the file path to In the memory; read each line of data in the Excel file in the memory and store it in the corresponding parsing object java model; serialize the parsing object java model containing the data into a JSON string and send it to Kafka for storage. The Excel analysis method that supports variable columns is used to solve the problem of entering the factory product inspection data into the database, and to help the digitization of the factory product inspection data.

Description

technical field [0001] The invention relates to the technical field of data analysis, in particular to a method and system for analyzing Excel files supporting variable columns. Background technique [0002] Various types of product testing data (such as quality, process) in the factory are kept in the form of Excel files. The fixed part of the table header is the basic information of the product, and the variable part is the specific testing item information. The traditional processing method is to use POI to parse the Excel table. After loading the entire table file at one time with fixed rows and columns, the Java function package is used for parsing. The parsing speed is slow and consumes a lot of memory. [0003] Patent document CN109933769A discloses a method for quickly analyzing report data based on Excel. First, it defines the format standard that needs to be parsed in Excel workbooks; then quickly extracts the formats in different Sheets in the Excel workbook to ob...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Patents(China)
IPC IPC(8): G06F40/12
CPCG06F40/12
Inventor 林大王星宇师文庆旷黎明
Owner 上海微亿智造科技有限公司
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap