Data analysis and data model building method and related equipment

Abstract

The present application provides a data analysis and data model construction method, including: obtaining a target file and preliminarily identifying a target page of the target file; traversing the target page, determining the target data, and determining the data storage method of the target page according to the target data; determining the indicator name of the target data according to the target data and the data storage method; determining a first mapping relationship between target data and indicator names in any target file, and storing the target data and indicator names in a standardized manner using a preset format; determining a second mapping relationship between target data and indicator names in different target files according to the standardized storage; and determining a visual relationship model between target data and indicator names in different target files according to the second mapping relationship.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to a method for data analysis and data model building and related equipment. Background Art

[0002] With the rapid development of the digital economy, enterprise operations management is undergoing unprecedented changes. The widespread application of new technologies such as informatization and digitization involves more content and more data. However, for many traditional industries, data is still generally scattered among different organizations and stored in different documents in different organizations. Different documents will have various formats according to the habits of different users, making the document formats for storing data different. In order to achieve unified data management, different data needs to be integrated. Due to different document formats, it is impossible to read them uniformly and effectively, which brings corresponding difficulties to the integration of data scattered in different documents. When applying the data, it is also impossible to read and analyze the data effectively and intuitively. Summary of the invention

[0003] In view of this, the purpose of the present application is to propose a data analysis and data model construction method and related equipment that overcome the above-mentioned problems or at least partially solve the above-mentioned problems.

[0004] Based on the above objectives, the first aspect of the present application provides a data analysis and data model construction method, including:

[0005] Acquire a target file and preliminarily identify a target page of the target file;

[0006] Traversing the target page, determining target data, and determining a data storage method of the target page according to the target data;

[0007] Determine the indicator name of the target data according to the target data and the data storage method;

[0008] Determine a first mapping relationship between the target data and the indicator name in any of the target files, and store the target data and the indicator name in a standardized manner using a preset format;

[0009] Determining, according to the standardized storage, a second mapping relationship between the target data and the indicator name in different target files;

[0010] According to the second mapping relationship, a visual relationship model between the target data and the indicator name in different target files is determined.

[0011] Optionally, the acquiring the target file and preliminarily identifying the target page of the target file includes:

[0012] Determine the page number and page names of the target file;

[0013] In response to the existence of the page name, fuzzy search all pages using the page name to determine that a page matching the page name is the target page;

[0014] In response to the absence of the page name, all the pages are traversed, starting from the first cell of the page, and all cells are traversed, using the beginning and end of the number plus an indefinite length wildcard to identify whether there is data in the current page. If there is data, the current page is determined to be the target page.

[0015] Optionally, traversing the target page, determining target data, and determining a data storage method of the target page according to the target data includes:

[0016] Starting from the first cell of the target page, traverse all cells of the target page along the row direction and the column direction respectively, determine the cell where the number is located, and determine that the number is the target data;

[0017] In response to the cells where the target data are located being arranged in any row or any column, determining that the data storage mode is row or column storage, and locating the target data;

[0018] In response to the cells where the target data are located occupying multiple rows and multiple columns, and the cells where the target data are located are continuous, determining that the data storage mode is row-column multi-dimensional storage, and locating the target data;

[0019] In response to the cells where the target data are located occupying multiple rows and / or multiple columns, and the cells where the target data are located are not continuous, it is determined that the data storage mode is local cell storage, and the target data is located.

[0020] Optionally, determining the indicator name of the target data according to the target data and the data storage method includes:

[0021] In response to the data storage method being row-based storage, determining the column where the target data is located, starting from the column where the target data is located, sequentially acquiring data contents of all cells in the remaining columns in a first target direction until the accumulated data contents in the remaining columns are not empty, and determining that the data contents in the remaining columns are the indicator names;

[0022] In response to the data storage method being column storage, determining the row where the target data is located, starting from the row where the target data is located, sequentially acquiring data contents of all cells in the remaining rows in a second target direction until the accumulated data contents in the remaining rows are not empty, and determining that the data contents in the remaining rows are the indicator names;

[0023] In response to the data storage method being row-column multidimensional storage, determining a target data area according to the positioning of the target data, taking the row and column where the first target data in the target data area is located as a starting point, determining the data contents of all cells in the remaining rows and columns in accordance with the first target direction and the second target direction in sequence, until the data contents in the remaining rows and columns are all not empty, determining the data contents in the remaining rows and columns as the indicator name;

[0024] In response to the data storage method being local cell storage, determine whether there is a character in front of the first bit of the target data. If so, the character is the indicator name of the target data. If not, check the surrounding cells of the target data in turn. If the surrounding cells have data content, determine that the data content is the indicator name.

[0025] Optionally, determining a first mapping relationship between the target data and the indicator name in any target file, and storing the target data and the indicator name in a standardized manner using a preset format, includes:

[0026] In response to the data storage method being row-based storage, determining an indicator name association with the same row of the target data;

[0027] In response to the data storage method being column storage, determining an indicator name association with the same column as the target data;

[0028] In response to the data storage mode being row-column multi-dimensional storage, determining an indicator name association with the same row and the same column as the target data;

[0029] In response to the data storage mode being local cell storage, determining an indicator name association closest to the target data;

[0030] According to the association, the indicator names are categorized by dimensions, the indicator names are filled in priority by rows or columns according to different dimensional categories, and the target data are filled in the rows or columns corresponding to the indicator names.

[0031] Optionally, determining, according to the standardized storage, a second mapping relationship between the target data in different target files and between the target data and the indicator name includes:

[0032] Determine the sources of different target files, and determine the connection between different target files by taking the sources as entities and the relationships between the sources as edges;

[0033] Determining a first relationship between the indicator names in different target files according to connections between the different target files;

[0034] Determining, according to the relationship between the indicator names, a second relationship between the target data in different target files;

[0035] According to the first relationship and the second relationship, a second mapping relationship between the target data and the indicator name in different target files is determined.

[0036] Optionally, determining, according to the second mapping relationship, a visual relationship model between the target data and the indicator name in different target files includes:

[0037] Determine the relationship model between the indicator names and the target data in different target files by taking the indicator names and the target data in different target files as nodes and taking the second mapping relationship as edges;

[0038] The nodes and edges are visualized.

[0039] In a second aspect of the present application, a data analysis and data model building device is provided, comprising:

[0040] An acquisition module, used to acquire a target file and preliminarily identify a target page of the target file;

[0041] A storage mode module traverses the target page, determines the target data, and determines the data storage mode of the target page according to the target data;

[0042] An indicator name module, used to determine the indicator name of the target data according to the target data and the data storage method;

[0043] A standardization module, used to determine a first mapping relationship between the target data and the indicator name in any target file, and to store the target data and the indicator name in a standardized manner using a preset format;

[0044] A mapping relationship module, used for determining a second mapping relationship between the target data and the indicator name in different target files according to the standardized storage;

[0045] A visualization model module is used to determine a visualization relationship model between the target data and the indicator name in different target files according to the second mapping relationship.

[0046] According to a third aspect of the present application, an electronic device is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the method described in the first aspect is implemented.

[0047] According to a fourth aspect of the present application, a non-transitory computer-readable storage medium is provided, wherein the non-transitory computer-readable storage medium stores computer instructions, wherein the computer instructions are used to enable a computer to execute the method described in the first aspect.

[0048] From the above, it can be seen that the data parsing and data model building method and related equipment provided by the present application first obtain all target files, extract valid files therein, remove invalid files, and further judge that the valid pages are target pages, identify the target data and indicator names in a more fine-grained manner, and store non-standard target files in different formats as standard files, so as to facilitate the reading of target data and improve the reading efficiency of target data. At the same time, a second mapping relationship between indicator names and target data in different target files is established, and the relationship between indicator names and target data in different target files is determined. The relationship is used to realize the visual display between indicator names and target data in different target files, and the association between different target files is more intuitively displayed, thereby improving practicality without the need to switch back and forth between all target files to find connections.

[0049] The above description is only an overview of the technical solution of the present invention. In order to more clearly understand the technical means of the present invention, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] In order to more clearly illustrate the technical solutions in the present application or related technologies, the drawings required for use in the embodiments or related technical descriptions are briefly introduced below. Obviously, the drawings described below are only embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0051] Figure 1 A flow chart of a method for data analysis and data model building according to an embodiment of the present application;

[0052] Figure 2 A flowchart of identifying a target page according to an embodiment of the present application;

[0053] Figure 3 A flowchart of determining a data storage method according to an embodiment of the present application;

[0054] Figure 4 A flowchart of determining the indicator name of target data according to an embodiment of the present application;

[0055] Figure 5 A flowchart of standardized storage according to an embodiment of the present application;

[0056] Figure 6 A flowchart of determining a second mapping relationship between target data and indicator names in different target files according to an embodiment of the present application;

[0057] Figure 7 A schematic diagram of row-based storage in an embodiment of the present application;

[0058] Figure 8 A schematic diagram of multi-dimensional storage of rows and columns according to an embodiment of the present application;

[0059] Fig. 9 A standardized storage schematic diagram of an embodiment of the present application;

[0060] Fig.10 A schematic diagram of a visual relationship model of an embodiment of the present application;

[0061] Fig.11 A schematic diagram of a data analysis and data model building device according to an embodiment of the present application;

[0062] Fig.12 A schematic diagram of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0063] In order to make the objectives, technical solutions and advantages of the present application more clearly understood, the present application is further described in detail below in combination with specific embodiments and with reference to the accompanying drawings.

[0064] It should be noted that, unless otherwise defined, the technical terms or scientific terms used in the embodiments of the present application should be the usual meanings understood by people with ordinary skills in the field to which the present application belongs. The "first", "second" and similar words used in the embodiments of the present application do not represent any order, quantity or importance, but are only used to distinguish different components. "Including" or "comprising" and similar words mean that the elements or objects appearing in front of the word cover the elements or objects listed after the word and their equivalents, without excluding other elements or objects. "Connect" or "connected" and similar words are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. "Up", "down", "left", "right" and the like are only used to indicate relative positional relationships. When the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0065] refer to Figure 1 As shown, it is a schematic diagram of a data analysis and data model building method 100 provided in an embodiment of the present application. The method 100 begins with step S101, obtaining a target file, and preliminarily identifying a target page of the target file.

[0066] In some optional application scenarios, the target file may be a related summary file in each organizational department of the enterprise. To obtain the target file, the target file of the specified path may be loaded through the internal network. This step involves opening, reading and preliminary identification of the file, and the format and content of the file have been determined. For example, taking an Excel file as an example, the file format is determined to be .xls or .xlsx, etc. There may be different pages in a file, so the different pages in the target file are identified accordingly to ensure that all data can be accurately identified and summarized.

[0067] Specifically, refer to Figure 2 As shown, step S101 includes: S1101, determining the page quantity and page name of the target file.

[0068] In some exemplary embodiments, taking Excel files as an example, in order to facilitate data aggregation and save the number of Excel files, multiple Sheet pages are included in the same Excel file. During the reading of the target file, the number of pages and page names of the target file are first identified.

[0069] S1102: In response to the existence of a page name, fuzzy search all pages using the page name to determine that a page matching the page name is a target page.

[0070] Exemplarily, one or several Sheet pages specified by the Excel file are located by fuzzy searching the name of the target Sheet page, which is recorded as a Sheet page set {B}, ie, the target page.

[0071] S1103. In response to the absence of a page name, all pages are traversed, starting from the first cell of the page, and all cells are traversed, using the beginning and end of the number plus an indefinite length wildcard to identify whether there is data in the current page. If there is data, the current page is determined to be the target page.

[0072] In this step, if there is no clear judgment on whether the Sheet page is the target page, for example, the Sheet page does not have a page name and it is impossible to effectively judge whether the Sheet page is the target page, each Sheet page is traversed, and whether the current Sheet page is a Sheet page that stores data is determined by identifying whether it contains data content. For all Sheet pages, each Sheet page is traversed first. For each Sheet, start from the A1 cell (the first cell) in the upper left corner and traverse to the lower right to check whether each cell contains data content. With respect to the data content, the purpose of the embodiment of the present application is to parse the data, and numbers are what can directly feedback the data, so numbers are used as matching items to screen the target page.

[0073] Regular expressions are used to match cell contents to identify cells containing numbers. That is, a variable-length wildcard is added to the first digit of a number, such as %x%, where % is a variable-length wildcard and x represents a number (which can be a decimal). Since numbers are usually accompanied by units, such as a total of 100 kg, 2 million yuan, etc., it is necessary to use a variable-length wildcard to identify the unit. For the identified cells containing numbers, their positions are recorded, and the Sheet page is determined as the target page, thereby filtering out all Sheet page sets {B}, i.e., the target page.

[0074] In some optional embodiments, when there is no page name, all pages are traversed and all pages are identified simultaneously using the first and last digits plus an indefinite-length wildcard to determine whether data exists, or each page is identified in turn using the first and last digits plus an indefinite-length wildcard to determine whether data exists.

[0075] In some exemplary embodiments, if it is determined that cells containing numbers in a page account for more than 5% of all cells containing content, the current page is determined to be the target page.

[0076] Afterwards, in step S102, the target page is traversed to determine the target data, and the data storage method of the target page is determined according to the target data.

[0077] Specifically, refer to Figure 3 As shown, step S102 includes: S1201, starting from the first cell of the target page, traverse all cells of the target page along the row direction and the column direction respectively, determine the cell where the number is located, and determine that the number is the target data;

[0078] S1202, in response to the cells where the target data are located being arranged in any row or any column, determining that the data storage method is row-by-row or column-by-column storage, and locating the target data;

[0079] S1203, in response to the target data occupying multiple rows and columns in the cell distribution, and the cells where the target data are located are continuous, determining that the data storage mode is row-column multi-dimensional storage, and locating the target data;

[0080] S1204: In response to the cells where the target data are located occupying multiple rows and / or multiple columns, and the cells where the target data are located are not continuous, the data storage mode is determined to be local cell storage, and the target data is located.

[0081] In this embodiment, in the process of parsing, integrating and summarizing data, it is not possible to simply summarize the numbers. Even if the corresponding digital data is identified and summarized, the corresponding digital data cannot intuitively reflect its meaning. Therefore, it is further necessary to parse, integrate and summarize the indicator names accordingly. For different target files, due to different recording habits, the data storage methods are also different. For a certain target page of a certain target file, it takes too long and the efficiency is too low to identify them one by one. Therefore, by identifying different storage methods, different methods are used to determine the indicator names to improve the recognition efficiency.

[0082] If one or several rows in the target file are all target data, it can be determined that the target data in the target file is stored in columns. If one or several columns are all target data, it can be determined that the target data in the target file is stored in rows, and the specific positions of the rows and columns of the target file are recorded.

[0083] If the target file contains target data in several consecutive rows and columns of cells, it can be determined that the storage method of the target data is row-column multi-dimensional storage, and the area where the target data is located is recorded as the data storage area.

[0084] However, in this case, each row and column at the edge of the area must be analyzed to determine whether the format is consistent with other rows and columns. For example, in the data storage area, if the second and subsequent rows are pure numeric cells, but the first row is a combination of numbers + Chinese or letters, the first row content should be removed from the data storage area; if the second and subsequent columns are pure numeric cell areas, but the first column is a combination of numbers + Chinese or letters, the first column content should be removed from the data storage area.

[0085] For all cells in the target page of the target file, no continuous rows or columns are found to store the target data. Only a few separate cells store the target data, and the cells storing the target data are not connected. It can be considered that the storage method of the target data in the current target page is local cell storage. Record the location of the cells storing the target data, and only extract the target data from these cells later.

[0086] At this point, the storage method of the target data in the target file is determined, and then the indicator name corresponding to the target data is determined, and the indicator name represents the description of the target data.

[0087] In step S103, the indicator name of the target data is determined according to the target data and the data storage method.

[0088] Specifically, refer to Figure 4As shown, step S103 includes: S1301, in response to the data storage method being row storage, determine the column where the target data is located, take the column where the target data is located as the starting point, and obtain the data content of all cells in the remaining columns in sequence according to the first target direction until the accumulated data content in the remaining columns is not empty, and determine that the data content in the remaining columns is the indicator name.

[0089] For the case of row storage, first locate the first column of the target data from left to right (determined by step S1201), such as Figure 7 As shown, the cell where the first target data at the top of the data column is defined as E1, and the first numerical cell at the bottom of the target data column is defined as En. For the table stored in rows, the indicator name column is usually only the first column on the left of the indicator column, so starting from the first column on the left of the current column, column D, traverse the indicator names from column D1 to the leftmost cell A1 (first target direction) until the cumulative contents from column D to the left are all not empty, that is, all indicator names are recognized, and columns A1 to D1 are defined as the indicator name area. If there are data columns after column E, start from the first column on the left of the next data column and traverse to the left in sequence (first target direction) until it is found that all the data contents in column E are not empty. Stop. Figure 7 For example, in column D, only D6 and D7 are cells with non-empty data content, so column D does not meet the indicator name column condition, so it is necessary to traverse the cumulative indicator name area to the left until column A is found. For each row, the cumulative indicator names from column A to column D are not empty, so the area from A2 to D11 is the indicator name area.

[0090] S1302. In response to the data storage method being column-based storage, determine the row where the target data is located, take the row where the target data is located as the starting point, and sequentially obtain the data contents of all cells in the remaining rows in the second target direction until the accumulated data contents in the remaining rows are not empty, and determine that the data contents in the remaining rows are indicator names.

[0091] For storage by column, start from the row above the first row of target data cells and search for indicator name areas in sequence, similar to storage by row, until all indicator name areas are found and recorded.

[0092] S1303. In response to the data storage method being row-column multidimensional storage, the target data area is determined according to the location of the target data, and the row and column where the first target data in the target data area is located are taken as the starting point. The data contents of all cells in the remaining rows and columns are determined in sequence according to the first target direction and the second target direction until the data contents in the remaining rows and columns are all not empty, and the data contents in the remaining rows and columns are determined as indicator names.

[0093] For multi-dimensional storage of rows and columns, first locate the target data area, start with the row and column where the upper left corner cell of the target data area is located, and search for the indicator name area from left to top, column by column and row by row. Figure 8 As shown, the numerical value area is B2 to B11. First, locate to the position shown by B2 in the upper left corner, and determine whether the left A1 column is a non-empty indicator name area. Then locate to see whether the first row is an indicator name area. It can be seen that B1 to H1 in the first row are all indicator name areas and non-empty indicator name areas. Therefore, the first row and A1 column are both recorded as indicator name areas.

[0094] S1304. In response to the data storage method being local cell storage, determine whether there is a character in front of the first digit of the target data. If so, the character is the indicator name of the target data. If not, check the surrounding cells of the target data in turn. If the surrounding cells have data content, determine that the data content is the indicator name.

[0095] For local cell storage, first locate the target data cell (determined in step S1204), and determine whether the cell contains pure numbers or Chinese or English characters + numbers. If it contains Chinese or English characters + numbers, the characters before the numbers in the cell are determined to be the indicator name of the cell, and the numbers are the values. If not, first check the cells to the left of the cell to determine whether the cell contains non-empty Chinese or English symbols. If so, the cell is identified as the indicator name. If not, continue to check the remaining adjacent cells to determine whether there are non-empty Chinese or English characters. If so, it is identified as the indicator name of the cell.

[0096] In some optional embodiments, if the indicator name has a date indicator, a date regular expression is used for matching to determine the date indicator. In some exemplary embodiments, the date regular expression is expressed as one or more of: %xxxxyearxxmonthxxday%, %xxyearxxmonthxxday%, %xmonthxday%, %xxmonthxxday%, %xx-xx%, %xx / xx%. Where % is an indefinite length wildcard, and x is a one-character number.

[0097] Specifically, a date regular expression is used to match in the indicator name area, and if a date format exists, the date format is determined to be the indicator name;

[0098] Identify the target page name, and if a date format exists, determine the date format as the indicator name;

[0099] Identify the non-target data and indicator name area, and if a date format exists, determine the date format as a unified date of the target data and indicator name, and determine the same date as the indicator name.

[0100] After the indicator name and target data are identified, the target data needs to be cleaned to make the target data format uniform. For example, after identifying numbers as target data, remove extra spaces, outliers, and special symbols.

[0101] Then, in step S104, a first mapping relationship between target data and indicator names in any target file is determined, and the target data and indicator names are stored in a standardized manner using a preset format.

[0102] In order to facilitate the interpretation of target data, the target data and indicator names are stored in a standardized manner to ensure data security and facilitate access.

[0103] Specifically, refer to Figure 5 As shown, step S104 includes: S1401, in response to the data storage mode being row-based storage, determining the indicator name association with the same row of target data;

[0104] S1402: In response to the data storage method being column storage, determining an indicator name association with the same column as the target data;

[0105] S1403: In response to the data storage mode being row-column multidimensional storage, determining an indicator name association with the same row and column as the target data;

[0106] S1404, in response to the data storage mode being local cell storage, determining an indicator name association closest to the target data;

[0107] S1405. According to the association, the indicator names are categorized into dimensions, the indicator names are filled in rows or columns in priority according to the different dimensional categories, and the target data is filled in the rows or columns corresponding to the indicator names.

[0108] After parsing the target data and indicator names, they are classified according to different indicator dimensions, and the indicator names and target data in different dimensions are stored accordingly, such as Fig. 9 shown.

[0109] In the same target file, the relationship between the indicator name and the target data is determined through standardized storage. However, different target files may have different sources. For example, target file A comes from department A, and target file B comes from department B. Standardized storage can only provide feedback on the correlation within target file A or the correlation within target file B. If you want to achieve the correlation between different target files, you need to further determine the relationship between different target files.

[0110] In step S105, a second mapping relationship between target data and indicator names in different target files is determined according to the standardized storage.

[0111] Specifically, refer to Figure 6 As shown, step S105 includes, S1501, determining sources of different target files, and using the sources as implementation and the relationship between the sources as edges to determine connections between different target files.

[0112] For example, target file A originates from department A, and target file B originates from department B. If department A is the superior department of department B, then department A and department B are connected vertically, that is, target file A and target file B are connected vertically. If department A and department B are departments of the same level, then department A and department B are connected horizontally, that is, target file A and target file B are connected horizontally.

[0113] S1502. Determine a first relationship between indicator names in different target files according to connections between different target files.

[0114] Exemplarily, the indicator name of target file A is total cost, and the indicator name of target file B is each item of cost, then it can be determined that the first relationship between the indicator names is an inclusion relationship.

[0115] S1503. Determine a second relationship between the target data in different target files based on the relationship between the indicator names.

[0116] For example, the first relationship is an inclusion relationship, and the relationship between the target data must also conform to the inclusion relationship. For example, if the total expenditure is 10 million yuan, then the expenditures for each item are 500,000 yuan, 1 million yuan, and 2 million yuan respectively. The total of 3.5 million yuan is included in 10 million yuan, and the rest may be the expenditure of other departments.

[0117] S1504. Determine a second mapping relationship between target data in different target files and the indicator name according to the first relationship and the second relationship.

[0118] According to the first relationship and the second relationship, it can be determined that the association relationship between the indicator name total expenditure and the target data 500,000 yuan, 1 million yuan, and 2 million yuan is the second mapping relationship.

[0119] Then in step S106, a visual relationship model between target data and indicator names in different target files is determined according to the second mapping relationship.

[0120] The second mapping relationship clarifies the relationship between the indicator names and target data between different target files. However, analysis and query are also required. In order to more conveniently and intuitively display the relationship between the indicator names and indicator data in different target files, a visual relationship model is used for intuitive display.

[0121] Specifically, step S106 includes: using the indicator names and target data in different target files as nodes and the second mapping relationship as an edge, respectively, to determine a relationship model between the indicator names and the target data in different target files;

[0122] The nodes and edges are visualized.

[0123] refer to Fig.10 As shown, the indicator names of target file A are total expenditure and XX year, and the target data is 10 million; the indicator names of target file B are first expenditure, second expenditure, third expenditure, January 10, February 11, and May 7, and the target data are 500,000, 1 million, and 2 million, respectively. The corresponding relationship model visualization shows the relationship between different target files.

[0124] It should be noted that the method of the embodiment of the present application can be performed by a single device, such as a computer or server. The method of this embodiment can also be applied to a distributed scenario and completed by multiple devices cooperating with each other. In the case of such a distributed scenario, one of the multiple devices can only perform one or more steps in the method of the embodiment of the present application, and the multiple devices will interact with each other to complete the described method.

[0125] It should be noted that the above describes some embodiments of the present application. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the above embodiments and still achieve the desired results. In addition, the processes depicted in the accompanying drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0126] Based on the same technical concept, corresponding to any of the above-mentioned embodiment methods, the present application also provides a data analysis and data model building device.

[0127] refer to Fig.11 , the data analysis and data model building device comprises:

[0128] The acquisition module 1001 is used to acquire a target file and preliminarily identify a target page of the target file;

[0129] Storage mode module 1002, traverses the target page, determines target data, and determines the data storage mode of the target page according to the target data;

[0130] An indicator name module 1003 is used to determine the indicator name of the target data according to the target data and the data storage method;

[0131] A standardization module 1004 is used to determine a first mapping relationship between the target data and the indicator name in any target file, and to store the target data and the indicator name in a standardized manner using a preset format;

[0132] A mapping relationship module 1005, used for determining a second mapping relationship between the target data and the indicator name in different target files according to the standardized storage;

[0133] The visualization model module 1006 is used to determine the visualization relationship model between the target data and the indicator name in different target files according to the second mapping relationship.

[0134] For the convenience of description, the above device is described in terms of functions divided into various modules. Of course, when implementing the present application, the functions of each module can be implemented in the same or multiple software and / or hardware.

[0135] The device of the above embodiment is used to implement the corresponding data analysis and data model building method in any of the above embodiments, and has the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0136] Based on the same technical concept, corresponding to any of the above-mentioned embodiment methods, the present application also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the data analysis and data model building method described in any of the above-mentioned embodiments is implemented.

[0137] Fig.12 A more specific schematic diagram of the hardware structure of an electronic device provided in this embodiment is shown, and the device may include: a processor 1010, a memory 1020, an input / output interface 1030, a communication interface 1040, and a bus 1050. The processor 1010, the memory 1020, the input / output interface 1030, and the communication interface 1040 are connected to each other through the bus 1050 in the device.

[0138] The processor 1010 can be implemented by a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this specification.

[0139] The memory 1020 may be implemented in the form of ROM (Read Only Memory), RAM (Random Access Memory), static storage device, dynamic storage device, etc. The memory 1020 may store an operating system and other application programs. When the technical solutions provided in the embodiments of this specification are implemented by software or firmware, the relevant program codes are stored in the memory 1020 and are called and executed by the processor 1010.

[0140] The input / output interface 1030 is used to connect the input / output module to realize information input and output. The input / output module can be configured in the device as a component (not shown in the figure), or it can be externally connected to the device to provide corresponding functions. The input device may include a keyboard, a mouse, a touch screen, a microphone, various sensors, etc., and the output device may include a display, a speaker, a vibrator, an indicator light, etc.

[0141] The communication interface 1040 is used to connect a communication module (not shown) to realize communication interaction between the device and other devices. The communication module can realize communication through a wired mode (such as USB, network cable, etc.) or a wireless mode (such as mobile network, WIFI, Bluetooth, etc.).

[0142] The bus 1050 includes a path that transmits information between the various components of the device (eg, the processor 1010, the memory 1020, the input / output interface 1030, and the communication interface 1040).

[0143] It should be noted that, although the above device only shows the processor 1010, the memory 1020, the input / output interface 1030, the communication interface 1040 and the bus 1050, in the specific implementation process, the device may also include other components necessary for normal operation. In addition, it can be understood by those skilled in the art that the above device may also only include the components necessary for implementing the embodiments of the present specification, and does not necessarily include all the components shown in the figure.

[0144] The electronic device of the above-mentioned embodiment is used to implement the corresponding data analysis and data model building method in any of the above-mentioned embodiments, and has the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0145] Based on the same technical concept, corresponding to any of the above-mentioned embodiment methods, the present application also provides a non-transitory computer-readable storage medium, wherein the non-transitory computer-readable storage medium stores computer instructions, and the computer instructions are used to enable the computer to execute the data analysis and data model building method described in any of the above embodiments.

[0146] The computer-readable medium of this embodiment includes permanent and non-permanent, removable and non-removable media, and information storage can be implemented by any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, read-only compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device.

[0147] The computer instructions stored in the storage medium of the above embodiments are used to enable the computer to execute the data analysis and data model building method described in any of the above embodiments, and have the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0148] Based on the same inventive concept, corresponding to the data parsing and data model building method described in any of the above embodiments, the present disclosure also provides a computer program product, which includes computer program instructions. In some embodiments, the computer program instructions can be executed by one or more processors of a computer so that the computer and / or the processor execute the data parsing and data model building method. Corresponding to the execution subject corresponding to each step in each embodiment of the data parsing and data model building method, the processor that executes the corresponding step may belong to the corresponding execution subject.

[0149] The computer program product of the above-mentioned embodiment is used to enable the computer and / or the processor to execute the data analysis and data model building method described in any of the above embodiments, and has the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0150] Those skilled in the art should understand that the discussion of any of the above embodiments is merely illustrative and is not intended to imply that the scope of the present application (including the claims) is limited to these examples. In line with the concept of the present application, the technical features in the above embodiments or different embodiments may be combined, the steps may be implemented in any order, and there are many other variations of the different aspects of the embodiments of the present application as described above, which are not provided in detail for the sake of simplicity.

[0151] In addition, to simplify the description and discussion, and in order not to make the embodiments of the present application difficult to understand, the known power supply / ground connection with the integrated circuit (IC) chip and other components may or may not be shown in the provided drawings. In addition, the device can be shown in the form of a block diagram to avoid making the embodiments of the present application difficult to understand, and this also takes into account the fact that the details of the implementation of these block diagram devices are highly dependent on the platform to be implemented in the embodiments of the present application (that is, these details should be fully within the scope of understanding of those skilled in the art). In the case of elaborating specific details (e.g., circuits) to describe exemplary embodiments of the present application, it is obvious to those skilled in the art that the embodiments of the present application can be implemented without these specific details or when these specific details are changed. Therefore, these descriptions should be considered to be illustrative rather than restrictive.

[0152] Although the present application has been described in conjunction with specific embodiments of the present application, many replacements, modifications and variations of these embodiments will be apparent to those skilled in the art from the foregoing description. For example, other memory architectures (e.g., dynamic RAM (DRAM)) may use the embodiments discussed.

[0153] The embodiments of the present application are intended to cover all such substitutions, modifications and variations that fall within the broad scope of the appended claims. Therefore, any omissions, modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the embodiments of the present application should be included in the scope of protection of the present application.

Claims

1. A method for data analysis and data model construction, characterized in that: include: Obtaining a target file and preliminarily identifying a target page of the target file; Traversing the target page, determining target data, and determining a data storage method of the target page according to the target data; Determine the indicator name of the target data according to the target data and the data storage method; Determine a first mapping relationship between the target data and the indicator name in any of the target files, and store the target data and the indicator name in a standardized manner using a preset format; Determining, according to the standardized storage, a second mapping relationship between the target data and the indicator name in different target files; According to the second mapping relationship, a visual relationship model between the target data and the indicator name in different target files is determined.

2. The method according to claim 1, characterized in that: The step of obtaining the target file and preliminarily identifying the target page of the target file includes: Determine the page number and page names of the target file; In response to the existence of the page name, fuzzy search all pages using the page name to determine that a page matching the page name is the target page; In response to the absence of the page name, all the pages are traversed, starting from the first cell of the page, and all cells are traversed, using the first and last digits plus an indefinite length wildcard to identify whether there is data in the current page. If there is data, the current page is determined to be the target page.

3. The method according to claim 1, characterized in that The traversing the target page, determining the target data, and determining the data storage method of the target page according to the target data includes: Starting from the first cell of the target page, traverse all cells of the target page along the row direction and the column direction respectively, determine the cell where the number is located, and determine that the number is the target data; In response to the cells where the target data are located being arranged in any row or any column, determining that the data storage mode is row or column storage, and locating the target data; In response to the cells where the target data are located occupying multiple rows and multiple columns, and the cells where the target data are located are continuous, determining that the data storage mode is row-column multi-dimensional storage, and locating the target data; In response to the cells where the target data are located occupying multiple rows and / or multiple columns, and the cells where the target data are located are not continuous, it is determined that the data storage mode is local cell storage, and the target data is located.

4. The method according to claim 3, characterized in that: Determining the indicator name of the target data according to the target data and the data storage method includes: In response to the data storage method being row-based storage, determining the column where the target data is located, starting from the column where the target data is located, sequentially acquiring data contents of all cells in the remaining columns in a first target direction until the accumulated data contents in the remaining columns are not empty, and determining that the data contents in the remaining columns are the indicator names; In response to the data storage method being column storage, determining the row where the target data is located, starting from the row where the target data is located, sequentially acquiring data contents of all cells in the remaining rows in a second target direction until the accumulated data contents in the remaining rows are not empty, and determining that the data contents in the remaining rows are the indicator names; In response to the data storage method being row-column multidimensional storage, determining a target data area according to the positioning of the target data, taking the row and column where the first target data in the target data area is located as a starting point, determining the data contents of all cells in the remaining rows and columns in accordance with the first target direction and the second target direction in sequence, until the data contents in the remaining rows and columns are all not empty, determining the data contents in the remaining rows and columns as the indicator name; In response to the data storage method being local cell storage, determine whether there is a character in front of the first bit of the target data. If so, the character is the indicator name of the target data. If not, check the surrounding cells of the target data in turn. If the surrounding cells have data content, determine that the data content is the indicator name.

5. The method according to claim 4, characterized in that Determining a first mapping relationship between the target data and the indicator name in any of the target files, and storing the target data and the indicator name in a standardized manner using a preset format, including: In response to the data storage method being row-based storage, determining an indicator name association with the same row of the target data; In response to the data storage method being column storage, determining an indicator name association with the same column as the target data; In response to the data storage mode being row-column multi-dimensional storage, determining an indicator name association with the same row and the same column as the target data; In response to the data storage mode being local cell storage, determining an indicator name association closest to the target data; According to the association, the indicator names are categorized by dimensions, the indicator names are filled in priority by rows or columns according to different dimensional categories, and the target data are filled in the rows or columns corresponding to the indicator names.

6. The method according to claim 5, characterized in that Determining, according to the standardized storage, a second mapping relationship between the target data in different target files and between the target data and the indicator name includes: Determine the sources of different target files, and determine the connection between different target files by taking the sources as entities and the relationships between the sources as edges; Determining a first relationship between the indicator names in different target files according to connections between the different target files; Determining, according to the relationship between the indicator names, a second relationship between the target data in different target files; According to the first relationship and the second relationship, a second mapping relationship between the target data and the indicator name in different target files is determined.

7. The method according to claim 1, characterized in that The determining, according to the second mapping relationship, a visual relationship model between the target data and the indicator name in different target files includes: Determine the relationship model between the indicator names and the target data in different target files by taking the indicator names and the target data in different target files as nodes and taking the second mapping relationship as edges; The nodes and edges are visualized.

8. A data analysis and data model building device, characterized in that: include: An acquisition module, used to acquire a target file and preliminarily identify a target page of the target file; A storage mode module traverses the target page, determines the target data, and determines the data storage mode of the target page according to the target data; An indicator name module, used to determine the indicator name of the target data according to the target data and the data storage method; A standardization module, used to determine a first mapping relationship between the target data and the indicator name in any target file, and to store the target data and the indicator name in a standardized manner using a preset format; A mapping relationship module, used for determining a second mapping relationship between the target data and the indicator name in different target files according to the standardized storage; A visualization model module is used to determine a visualization relationship model between the target data and the indicator name in different target files according to the second mapping relationship.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the method according to any one of claims 1 to 7 is implemented.

10. A non-transitory computer-readable storage medium storing computer instructions, characterized in that: The computer instructions are used to enable a computer to execute the method according to any one of claims 1 to 7.

Images