Data acquisition format updating method, device, computer equipment and storage medium

Abstract

The embodiment of the present application belongs to the field of data collection technology and can be applied to the field of smart government affairs. It relates to a method for updating the data collection format, including receiving a standard format document provided by a maintenance party, and extracting a format report in the standard format document, wherein the format report records the field information of the fields contained in the modified data table; according to the field information recorded in a format report, determine the table structure of a modified data table; compare the table structure of the modified data table with the corresponding original data table, and change the field information corresponding to the corresponding original data table to update the data collection format. The present application also provides a data collection format update device, a computer device and a storage medium. In addition, the present invention also relates to blockchain technology, and the format report can be stored in the blockchain. The present application can quickly and generally traverse the data table, and find the data table and field to be adjusted, and the data collection format update effect is good and the speed is fast.

Description

Technical Field

[0001] The present application relates to the field of data acquisition technology, and more particularly to a method, device, computer equipment, and storage medium for updating data acquisition formats. Background Art

[0002] When a system or data warehouse collects and stores information, it usually defines a standard format or gives a corresponding template. However, when the system is modified, the standard format of the data may need to be modified to meet new business needs. At this time, the data storage format or template must be updated. The conventional update method is to manually refer to the database table structure updated by the maintenance party and make corresponding adjustments to the original data table structure in the system or data warehouse. Usually, when the system is updated, the maintenance party will release an updated version of the standard format to announce the updated system standard format in the form of a table. By traversing and analyzing the table provided by the maintenance party, all standard format updates can be collected. There are many such database tables in a system, and they are complicated because of the mutual relationships between them. It is a huge workload to find the system's updates to the standard format through manual investigation, and it is easy to make mistakes, and the information collection plan is slow to change. Summary of the invention

[0003] The purpose of the embodiment of the present application is to propose a method that can update the data collection format according to the database table structure provided by the maintainer.

[0004] In order to solve the above technical problems, the present application embodiment provides a data acquisition format update method, which adopts the following technical solution:

[0005] A data collection format update method comprises the following steps: receiving a standard format document provided by a maintenance party, and extracting a format report in the standard format document, wherein the format report records field information of fields included in a modified data table;

[0006] Determine a table structure of a modified data table according to a format report, wherein the table structure of the modified data table is specifically determined according to the field information recorded in the format report; compare the table structure of the modified data table with the corresponding original data table, change the field information of the original data table according to the table structure of the modified data table, and update the data collection format.

[0007] Furthermore, the step of determining the table structure of a modified data table based on the field information recorded in the format report specifically includes: reading the first row of a format report, and judging the information of each column in the first row based on preset keywords to determine the field information corresponding to each column in the format report; reading the second row to the last row of the format report in sequence, and determining the name, attribute information and constraints of a field based on the field information corresponding to each column in each row to determine the table structure of a modified data table.

[0008] Furthermore, the step of sequentially reading the second to last rows of the format report and determining the name, attribute information and constraints of a field according to the field information corresponding to each column in each row to determine the table structure of a modified data table specifically includes:

[0009] According to the target row in the format report, the field information in the target row is extracted, and the field name and attribute information are determined according to the field information in the corresponding column; the target row is any row from the second row to the last row; the characteristic string of the field information in the corresponding column is extracted to match the constraints on the corresponding field; the field name, attribute information and constraints of each field are determined according to the second row to the last row in the format report in a loop to obtain the table structure of the modified data table; and a table creation script is generated according to the table structure of the modified data table.

[0010] Furthermore, the step of comparing the table structure of the modified data table with the corresponding original data table, and changing the corresponding field information of the original data table according to the table structure of the modified data table to update the data collection format specifically includes: traversing all original data tables according to the name of the format report to determine the original data table corresponding to the modified data table; establishing the modified data table according to the table creation script; calling the stored procedure for the original data table and the modified data table, and determining the original data table and fields that have changed relative to the modified data table; and collecting data content through the changed data tables and fields.

[0011] Furthermore, the step of traversing all original data tables according to the name of the format report to determine the original data table corresponding to the modified data table specifically includes: parsing the name of the modified data table recorded in the format report and matching it with the name of the original data table; if no match is successful, extracting at least one group of name feature characters in the format report; traversing all original data tables, matching the name feature characters with the name of the original data table, determining the matching degree between the original data table name and the name feature characters, and outputting the names of the original data table and the modified data table whose matching degree reaches a preset value.

[0012] Furthermore, the step of calling a stored procedure on the original data table and the modified data table to determine the original data table and fields that have changed relative to the modified data table specifically includes: calling a stored procedure, and matching the fields in the original data table with the fields in the modified data table based on the names and attribute information of the corresponding fields in the original data table under the stored procedure and the names and attribute information of the corresponding fields in the modified data table to correspond the fields in the original data table and the modified data table; comparing the attribute information and constraints in the fields to determine the changed fields and the corresponding data table.

[0013] Furthermore, the step of receiving a standard format document provided by a maintenance party and extracting a format report from the standard format document specifically includes: receiving a word format document through Java-poi; mobilizing the APII provided by Java-poi to extract the format report in the table format in the word format document; extracting the cell data and row and column information of the format report.

[0014] In order to solve the above technical problems, the embodiment of the present application further provides a data acquisition format updating device, which adopts the following technical solution:

[0015] A data acquisition format updating device, comprising:

[0016] A format report acquisition module, used for receiving a standard format document provided by a maintainer, and extracting a format report in the standard format document, wherein the format report records field information of fields included in the modified data table;

[0017] A table structure determination module, used to determine the table structure of a modified data table according to a format report, wherein the table structure of the modified data table is specifically determined according to the field information recorded in the format report;

[0018] The data table updating module is used to compare the table structure of the modified data table with that of the corresponding original data table, change the field information of the original data table according to the table structure of the modified data table, and update the data collection format.

[0019] In order to solve the above technical problems, the embodiment of the present application further provides a computer device, which adopts the following technical solution:

[0020] A computer device comprises a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the data acquisition format updating method as described above when executing the computer program.

[0021] In order to solve the above technical problems, the embodiment of the present application further provides a computer-readable storage medium, which adopts the following technical solution:

[0022] A computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the data acquisition format updating method as described above are implemented.

[0023] Compared with the prior art, the embodiments of the present application mainly have the following beneficial effects: by extracting the format report in the standard format document and extracting the field information in the format report, the table structure of the modified data table is determined, wherein each format report records the table structure of a modified data table, and then the table structure of the modified data table is compared with the table structure of the original data table, and the corresponding data table and fields are changed to update the data collection format of the data table. This solution can quickly and generally traverse the data table and find the data table and fields to be adjusted, and the data collection format is updated effectively and quickly. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] In order to more clearly illustrate the scheme in the present application, a brief introduction is given below to the drawings required for use in the description of the embodiments of the present application. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0025] Figure 1 is an exemplary system architecture diagram to which the present application may be applied;

[0026] Figure 2 A flowchart of an embodiment of a method for updating a data acquisition format according to the present application;

[0027] Figure 3 yes Figure 2 A flowchart of a specific implementation of step S200;

[0028] Figure 4 yes Figure 3 A flowchart of a specific implementation of step S202;

[0029] Figure 5 yes Figure 1 A flowchart of a specific implementation of step S300;

[0030] Figure 6 yes Figure 5 A flowchart of a specific implementation of step S301;

[0031] Figure 7 yes Figure 5 A flowchart of a specific implementation of step S303;

[0032] Figure 8 yes Figure 1 A flowchart of a specific implementation of step S100;

[0033] Fig. 9 is a structural schematic diagram of an embodiment of a data acquisition format updating device according to the present application;

[0034] Fig.10 is a schematic structural diagram of an embodiment of a computer device according to the present application;

[0035] Fig.11 An embodiment of a format report. DETAILED DESCRIPTION

[0036] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by technicians in the technical field of the present application; the terms used in the specification of the application herein are only for the purpose of describing specific embodiments and are not intended to limit the present application; the terms "including" and "having" and any variations thereof in the specification and claims of the present application and the above-mentioned drawings are intended to cover non-exclusive inclusions. The terms "first", "second", etc. in the specification and claims of the present application or the above-mentioned drawings are used to distinguish different objects, not to describe a specific order.

[0037] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0038] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the accompanying drawings.

[0039] like Figure 1 As shown, the system architecture 10 may include terminal devices 11, 12, 13, a network 14 and a server 15. The network 14 is used to provide a medium for communication links between the terminal devices 11, 12, 13 and the server 15. The network 14 may include various connection types, such as wired, wireless communication links or optical fiber cables, etc.

[0040] Users can use terminal devices 11, 12, 13 to interact with server 15 through network 14 to receive or send messages, etc. Various communication client applications can be installed on terminal devices 11, 12, 13, such as web browser applications, shopping applications, search applications, instant messaging tools, email clients, social platform software, etc.

[0041] The terminal devices 11, 12, and 13 can be various electronic devices with display screens and supporting web browsing, including but not limited to smart phones, tablet computers, e-book readers, MP3 players (Moving Picture Experts Group Audio Layer III), MP4 (Moving Picture Experts Group Audio Layer IV), laptop computers, desktop computers, etc.

[0042] The server 15 may be a server that provides various services, such as a background server that provides support for web pages displayed on the terminal devices 11 , 12 , and 13 .

[0043] It should be noted that the data acquisition format update method provided in the embodiment of the present application is generally composed of Clothes The data acquisition format updating device is generally arranged in the server / terminal device.

[0044] It should be understood that Figure 1 The number of terminal devices, networks and servers in the embodiment is only for illustration. Any number of terminal devices, networks and servers may be provided according to implementation requirements.

[0045] Continue to refer Figure 2 , shows a flow chart of an embodiment of a method for updating a data acquisition format according to the present application. The method for updating a data acquisition format comprises the following steps:

[0046] Step S100: receiving a standard format document provided by a maintainer, and extracting a format report in the standard format document, wherein the format report records field information of fields included in the modified data table.

[0047] In the case of needing to modify the data collection plan, the maintenance party uploads the standard format document to the system or data warehouse. The standard format document provided by the maintenance party stores the table structure of multiple data tables in the database. The table structure of these data tables is saved in the form of a format report, and the field information of the corresponding data table is stored in the format report. A platform often adjusts the structure of its database due to business adjustments, so it often generates standard format documents. By comparing the modified data table structure recorded in the standard format document with the original data table structure, the changes in the table structure can be determined, and the content of data collection can be changed accordingly.

[0048] Each format report corresponds to the table structure of a data table, and a corresponding number of modified data tables correspond to the extracted format reports.

[0049] In this embodiment, the data acquisition format update method is executed on the electronic device (eg Figure 1 The server / terminal device shown in the figure can interact via a wired connection or a wireless connection. It should be noted that the wireless connection method may include but is not limited to 3G / 4G connection, WiFi connection, Bluetooth connection, WiMAX connection, Zigbee connection, UWB (ultra wideband) connection, and other wireless connection methods currently known or to be developed in the future.

[0050] Step S200, determining a table structure of a modified data table according to a format report, wherein the table structure of the modified data table is specifically determined according to field information recorded in the format report.

[0051] The format report records the field information about the data table structure, including the field name, its attribute information and possible constraints. Based on the above field name, attribute information and constraints, a field in the modified data table can be determined, and based on all the field information recorded in the format report, the table structure of the modified data table can be determined.

[0052] For a system that implements a set of business functions, a database is usually used to store the corresponding data. A common relational database is usually composed of several data tables, as well as mappings and views generated based on the data tables. The core is the table structure of the database, including the field names contained in each data table, the attribute information of the fields, and the constraints that specify the field content or link the fields in the same table and different tables.

[0053] Step S300, comparing the table structure of the modified data table with that of the corresponding original data table, changing the field information of the original data table according to the table structure of the modified data table, and updating the data collection format.

[0054] The changed data table is determined by comparing the structure of the modified data table with the structure of the corresponding original data table, including the changed fields in the data table, and the changed attribute information and constraints in the corresponding fields. The table structure defined by the fields in the data table limits the access content of the data. When storing external data, the system or data warehouse of the maintenance party will collect external data, collect data that conforms to the format defined by each field in the data table, and store it in the data table. When the structure of the data table changes, the subsequent data collection is based on the table structure of the modified data table. By comparing the table structure of the modified data table with the original data table, the data table and fields that need to be updated can be found, updated, and data can be collected through the updated data collection format. This solution can quickly and completely adjust the data collection format.

[0055] Furthermore, the step S100 of receiving a standard format document provided by a maintenance party and extracting a format report from the standard format document specifically includes:

[0056] Step S101, receiving a word format document through Java-poi.

[0057] Standard format documents are edited and stored through office software, so they meet certain formats and can be recognized and read through Java-poi.

[0058] Step S102, calling the API provided by Java-poi to extract the format report in the table format in the word format document.

[0059] Java-poi is a tool library provided by Apache. Users can parse common office documents to collect data in documents by calling the API provided by Java-poi, and extract field information in structured format reports through Java-poi.

[0060] Step S103, extracting the cell data and row and column information of the format report.

[0061] Furthermore, the step S200, determining the table structure of a modified data table according to the field information recorded in the format report, specifically includes:

[0062] Step S201, reading the first line of a format report, and judging the information of each column in the first line according to preset keywords to determine the field information corresponding to each column in the format report.

[0063] The format report stores the field information of the modified data table, and the structures of various format reports cannot be guaranteed to be completely consistent.

[0064] For this embodiment, the first line of the format report needs to be read. The first line introduces what kind of field information is recorded in each column of the format report. Through character matching, it can be determined that each column in the second to the last line of the format report records the type of field-related information, such as field name, attribute or constraint. Fig. 9 In the format report shown in the figure, the content of the first line can be determined by character matching. Starting from the second line, the second column of each line stores the name of the field, the third column of each line stores the data type of the field, the fourth column of each line stores the length of the field, and the fifth column of each line stores field constraints and other information through notes. According to the information disclosed in the first line of the format report, the second to the last line in the format report respectively store all the field information of the modified data table, and the content of one line of records corresponds to one field. For a field, its related information includes at least the field name, data type, field length, and the primary key constraint, foreign key constraint, non-empty tag, field format and other constraint contents stored in the note information, and also includes the number used for information management but not related to the disclosure of field content.

[0065] Step S202, sequentially read the second to last rows of the format report, and determine the name, attribute information and constraints of a field according to the field information corresponding to each column in each row, so as to determine the table structure of a modified data table.

[0066] Traverse and extract the data from the second to the last row in the format report, where each row of data corresponds to the relevant information of a field in the modified data table, and the combination of the relevant information of all fields can reflect the table structure of the modified data table. This solution can quickly and completely traverse and obtain the table structure of the modified data table. The attribute information includes the data type and data length of the field.

[0067] Furthermore, the step S202 of sequentially reading the second to last rows of the format report and determining the name, attribute information and constraints of a field according to the field information corresponding to each column in each row to determine the table structure of a modified data table specifically includes:

[0068] Step S2021, extracting field information in the target row according to the target row in the format report, and determining the field name and attribute information according to the field information in the corresponding column; the target row is any row from the second row to the last row.

[0069] According to any row of the format report as the target row, extract the column representing the field name and the column expressing various attribute information of the field in the target row, determine the name of the field and related attribute information respectively, and specifically determine the data type of the field by the column where the field information of the field data type is located, and determine the data length of the field by the column where the field information of the data length is located.

[0070] Step S2022: extracting characteristic strings of the field information in the corresponding column to match the constraints on the corresponding field.

[0071] Extract the string from the field information in the column corresponding to the constraint information in the format report, and match the corresponding feature string to match the constraint on the field. Fig. 9 In the described embodiment, the field information extracted in the last column corresponds to the constraints of the field, and the field information is extracted in the form of a character string, and then a feature string is matched to determine the constraints of the corresponding field.

[0072] Step S2023, looping from the second line to the last line in the format report, determining the field name, attribute information and constraints of each field to obtain the table structure of the modified data table.

[0073] The field names, attributes and constraints in a row of the format report together constitute the information of a field in the data table. By bringing together all the fields and the corresponding attribute information and constraints, all the relevant information of a modified data table is determined, revealing the table structure of the data table.

[0074] exist Fig. 9 In the embodiment shown, the insurance confirmation code field is determined by extracting the information of the second line in the format report, the field type is a string, the field length is 30, and the constraint is not empty. The contracting company code field is determined by extracting the information of the third line in the format report, the field type is a character, the field length is 4, and the constraint is not empty. The content of a field is determined according to the content of each line in the format report until the information of the field corresponding to the format report is determined.

[0075] Step S2024, generating a table creation script according to the modified table structure of the data table.

[0076] Based on the combination of field information including field name, attribute information and constraints, and in accordance with standard SQL language rules, a table creation script is generated. This solution can fully obtain the relevant information of each field in the modified data table.

[0077] Furthermore, the step S300 of comparing the table structure of the modified data table with the corresponding original data table and changing the field information corresponding to the corresponding original data table according to the table structure of the modified data table to update the data collection format specifically includes:

[0078] Step S301, traverse all original data tables according to the name of the format report to determine the original data table corresponding to the modified data table.

[0079] The name of the data table remains consistent before and after the modification; if it is a newly added data table, the original data table does not exist, the entire data table is treated as the changed data table, and all fields in the data table are changed fields.

[0080] First, the original data tables in the database are traversed, and the name of the format report is matched with the original data table in the database to find the original data table corresponding to the modified data table, and then the two are compared to determine the modified data table and fields. In one embodiment, the name of the format report is consistent with the corresponding original data table, and the format report and the original data table can directly correspond. In another embodiment, the format report changes the name of the data table, corresponds to the original data table through character matching or other forms, and outputs the modified data table and the original data table corresponding to the character matching for the user to confirm whether the two are corresponding. In one embodiment, the data table provided in the format report is a brand new data table, and the format report does not need to be matched with the original data table.

[0081] Step S302: Create a modified data table according to the table creation script.

[0082] In the previous extraction process, the modified data table only stays in the data recorded in the format report, and has not yet formed a real data table. At this time, it is necessary to use the SQL table creation language to establish the data table, where the establishment of the data table includes generating standard SQL table creation language based on the fields of the modified data table, its attribute information and constraints, and then establishing the modified data table.

[0083] Step S303, calling a stored procedure for the original data table and the modified data table, and determining the original data table and fields that have been changed relative to the modified data table.

[0084] The stored procedure is a SQL statement that is compiled once and used permanently for the table structure of the data table, and the stored procedure is implemented for the corresponding data table. The data storage procedure is called for the modified data table and the corresponding original data table respectively, and the fields, attribute information and constraints between the two are compared according to the SQL statement generated by calling the stored procedure, and the changed data table and fields are determined.

[0085] Step S304, collecting data content through the changed data tables and fields.

[0086] Summarize the changes between the modified data table and the table structure of the original data table, including the data table name and fields, as well as the attribute information and constraints that have changed in the fields. When performing business activities, the system usually collects corresponding data based on the structure of the data table, and modifies part of the business logic through the summarized changes to achieve the collection of corresponding data. During the operation of the system, changed data tables and fields will be generated after a preset time period. In one embodiment, the system periodically generates the above-mentioned changed data tables and fields. After the changed data tables and fields are generated, the system automatically changes the data collection content. This solution is efficient in updating data collection content.

[0087] Furthermore, the step S301 of traversing all original data tables according to the name of the format report to determine the original data table corresponding to the modified data table specifically includes:

[0088] S3011, obtaining the name of the modified data table recorded in the format report through regular expression analysis, and matching it with the name of the original data table.

[0089] The description of the modified data table name in the format report is obtained through regular expression parsing, and the name of the modified data table obtained is matched with the name of the original data table to make the modified data table correspond to the original data table.

[0090] S3012, if no match is successful, extract at least one set of name feature characters in the format report;

[0091] In one embodiment, if the table name of the modified data table obtained by regular expression parsing does not correspond to any original data table name, the modified data table may be a newly added data table, or the table name of the original data table has been modified. The name feature characters in the table name of the modified data table are extracted and compared with the original data table to determine the original data table that is suspected to be matched.

[0092] S3013, traverse all original data tables, match the name characteristic characters with the names of the original data tables, determine the matching degree between the original data table names and the name characteristic characters, and output the original data tables and modified data tables whose matching degrees reach a preset value.

[0093] By matching the name characteristic characters with the name of the original data table, the original data table and the modified data table with similar names are determined. In one embodiment, the matching degree is determined by the number or weight of the matched name characteristic characters, and the original data table and the modified data table with a higher matching degree are output for the user to manually determine the corresponding relationship between the original data table and the modified data table. This solution can fully match the modified data table.

[0094] Furthermore, the step S303 of calling a stored procedure on the original data table and the modified data table to determine the original data table and fields that have been changed relative to the modified data table specifically includes:

[0095] Step S3031, calling the stored procedure, and matching the fields in the original data table with the fields in the modified data table according to the names and attribute information of the corresponding fields in the original data table and the names and attribute information of the corresponding fields in the modified data table under the stored procedure, so as to correspond the fields in the original data table and the modified data table.

[0096] After matching the modified data table with the original data table, the fields are matched by comparing the field names and attribute information. The field names are usually not modified, and some fields are usually added or deleted. By comparing the field names, the fields in the modified data table can be matched with the fields in the original data table.

[0097] Step S3032, comparing the attribute information and constraints in the field to determine the changed field and the corresponding data table.

[0098] Compare the fields in the modified data table with the fields in the corresponding original data table to determine whether the field has any changes in name, attribute information, and constraints, so as to finally determine the changed fields and corresponding data tables, and extract this information. If the field is a newly added field, then the field is obviously the changed field, and the definition content of the entire field is considered as the changed content. This solution is highly efficient in changing fields and data tables.

[0099] Afterwards, the database table structure that conforms to the standard format document can be established through the extracted field information. This solution can completely extract the field information to improve the accuracy of field information extraction.

[0100] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program, and the computer program can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, the aforementioned storage medium can be a non-volatile storage medium such as a disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM).

[0101] It should be emphasized that in order to further ensure the privacy and security of the above-mentioned format report information, the above-mentioned format report can also be stored in a blockchain node.

[0102] This solution can be applied in the field of smart government affairs, thereby promoting the construction of smart cities.

[0103] It should be understood that, although the steps in the flowchart of the accompanying drawings are displayed in sequence as indicated by the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least a part of the steps in the flowchart of the accompanying drawings may include multiple sub-steps or multiple stages, and these sub-steps or stages are not necessarily executed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be executed in turn or alternately with other steps or at least a part of the sub-steps or stages of other steps.

[0104] Further references Fig. 9 , as a response to the above Figure 2 In order to realize the method shown in the figure, the present application provides an embodiment of a data acquisition format updating device, and the device embodiment is similar to Figure 2 Corresponding to the method embodiment shown, the device can be specifically applied to various electronic devices.

[0105] like Fig. 9 As shown, the data collection format updating device described in this embodiment includes: a format report acquisition module, a table structure determination module and a data table update module. Among them:

[0106] The format report acquisition module is used to receive the standard format document provided by the maintainer and extract the format report in the standard format document, wherein the format report records the field information of the fields contained in the modified data table.

[0107] The table structure determination module is used to determine the table structure of a modified data table according to the field information recorded in the format report.

[0108] The data table updating module is used to compare the table structure of the modified data table with the corresponding original data table, and change the field information corresponding to the corresponding original data table according to the table structure of the modified data table to update the data collection format.

[0109] When the structure of the data table changes, the subsequent data collection is based on the modified data table structure. By comparing the modified data table with the original data table structure, the data table and fields that need to be updated are found, updated, and data is collected using the updated data collection format. This solution can quickly and completely adjust the data collection format.

[0110] Furthermore, the table structure determination module includes:

[0111] The field information determination submodule is used to read the first line of a format report, and to identify the information of each column in the first line according to the keywords, so as to determine the field information corresponding to each column in the format report;

[0112] The table structure traversal submodule is used to read the second to last rows of the format report in sequence, and determine the name, attribute information and constraints of a field according to the field information corresponding to each column in each row, so as to determine the table structure of a modified data table.

[0113] This solution can quickly and completely traverse the table structure of the modified data table.

[0114] Furthermore, the table structure traversal submodule includes:

[0115] The field attribute unit is used to extract the field information in a row according to any row from the second row to the last row in the format report, and determine the name of a field in the modified data table according to the field information corresponding to the field name, and determine the attribute information of the corresponding field according to the field information corresponding to the attribute information;

[0116] A constraint unit, used to extract the field information corresponding to the constraint in the row, and specifically extract the characteristic string in the field information to match the constraint on the corresponding field;

[0117] The field attribute unit and constraint unit are also used to loop through the second to last rows in the format report to determine the field name, attribute information and constraint of each field to obtain the table structure of the modified data table;

[0118] The script unit is used to generate a table creation script according to the modified table structure of the data table.

[0119] This solution can completely obtain the relevant information of each field in the modified data table.

[0120] Furthermore, the data table updating module specifically includes:

[0121] A data table corresponding submodule, used for traversing all original data tables according to the name of the format report to determine the original data table corresponding to the modified data table;

[0122] The data table creation submodule is used to create a modified data table according to the table creation script.

[0123] The data table comparison submodule is used to call the stored procedure on the original data table and the modified data table to determine the original data table and fields that have changed relative to the modified data table.

[0124] The collection submodule is used to collect data content through the changed data tables and fields.

[0125] The solution is highly efficient in updating data collection content.

[0126] Furthermore, the data table corresponds to a submodule, specifically including:

[0127] A name characteristic character extraction module, for extracting at least one set of name characteristic characters in the data report form if the modified data table does not successfully match the original data table;

[0128] The data table name matching module is used to traverse all original data tables, match the name characteristic characters with the names of the original data tables, determine the matching degree between the original data table names and the name characteristic characters, and output the names of the original data tables and the modified data tables whose matching degrees reach a preset value.

[0129] This solution can fully match the modified data table.

[0130] Furthermore, the data table comparison submodule specifically includes:

[0131] The name matching unit is used to call the stored procedure, and match the fields in the original data table with the fields in the modified data table according to the names and attribute information of the corresponding fields in the original data table under the stored procedure and the names and attribute information of the corresponding fields in the modified data table, so as to correspond the fields in the original data table and the modified data table.

[0132] The attribute constraint matching unit is used to compare the attribute information and constraints in the field to determine the changed field and the corresponding data table.

[0133] This solution improves the efficiency of table fields and data tables.

[0134] The format report acquisition submodule specifically includes:

[0135] The document acquisition submodule is used to receive word format documents through Java-poi;

[0136] A format report extraction submodule, used for mobilizing POI to extract the format report in the table format in the word format document;

[0137] The data extraction submodule is used to extract the cell data and row and column information of the format report.

[0138] This solution can completely extract field information to improve the accuracy of field information extraction.

[0139] To solve the above technical problems, the present application also provides a computer device. Figure 6 , Figure 8 This is a basic structural block diagram of the computer device in this embodiment.

[0140] The computer device 6 includes a memory 61, a processor 62, and a network interface 63 that are interconnected through a system bus. It should be noted that the figure only shows a computer device 6 with components 61-63, but it should be understood that it is not required to implement all the components shown, and more or fewer components can be implemented instead. Among them, those skilled in the art can understand that the computer device here is a device that can automatically perform numerical calculations and / or information processing according to pre-set or stored instructions, and its hardware includes but is not limited to microprocessors, application specific integrated circuits (Application Specific Integrated Circuit, ASIC), programmable gate arrays (Field-Programmable Gate Array, FPGA), digital processors (Digital Signal Processor, DSP), embedded devices, etc.

[0141] The computer device may be a computing device such as a desktop computer, a notebook, a PDA, a cloud server, etc. The computer device may interact with a user through a keyboard, a mouse, a remote controller, a touch pad, or a voice control device.

[0142] The memory 61 includes at least one type of readable storage medium, and the readable storage medium includes flash memory, hard disk, multimedia card, card-type memory (for example, SD or DX memory, etc.), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, magnetic disk, optical disk, etc. In some embodiments, the memory 61 can be an internal storage unit of the computer device 6, such as a hard disk or memory of the computer device 6. In other embodiments, the memory 61 can also be an external storage device of the computer device 6, such as a plug-in hard disk equipped on the computer device 6, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (FlashCard), etc. Of course, the memory 61 can also include both the internal storage unit of the computer device 6 and its external storage device. In this embodiment, the memory 61 is generally used to store the operating system and various application software installed on the computer device 6, such as the program code of the data acquisition format update method, etc. In addition, the memory 61 can also be used to temporarily store various types of data that have been output or are to be output.

[0143] The processor 62 may be a central processing unit (CPU), a controller, a microcontroller, a microprocessor, or other data processing chips in some embodiments. The processor 62 is generally used to control the overall operation of the computer device 6. In this embodiment, the processor 62 is used to run the program code stored in the memory 61 or process data, such as running the program code of the data acquisition format update method.

[0144] The network interface 63 may include a wireless network interface or a wired network interface. The network interface 63 is generally used to establish a communication connection between the computer device 6 and other electronic devices.

[0145] The present application also provides another implementation, namely, providing a computer-readable storage medium, wherein the computer-readable storage medium stores a data acquisition format update method, and the data acquisition format update method program can be executed by at least one processor to enable the at least one processor to perform the steps of the data acquisition format update method as described above.

[0146] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions for a terminal device (which can be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in each embodiment of the present application.

[0147] Obviously, the embodiments described above are only some embodiments of the present application, rather than all embodiments. The preferred embodiments of the present application are given in the accompanying drawings, but they do not limit the patent scope of the present application. The present application can be implemented in many different forms. On the contrary, the purpose of providing these embodiments is to make the understanding of the disclosure of the present application more thorough and comprehensive. Although the present application is described in detail with reference to the aforementioned embodiments, for those skilled in the art, it is still possible to modify the technical solutions recorded in the aforementioned specific implementation methods, or to perform equivalent replacement of some of the technical features therein. Any equivalent structure made using the contents of the specification and drawings of this application, directly or indirectly used in other related technical fields, is similarly within the scope of patent protection of this application.

[0148] The blockchain referred to in this invention is a new application mode of computer technologies such as distributed data storage, peer-to-peer transmission, consensus mechanism, encryption algorithm, etc. Blockchain is essentially a decentralized database, a string of data blocks generated by cryptographic methods. Each data block contains a batch of network transaction information, which is used to verify the validity of its information (anti-counterfeiting) and generate the next block. Blockchain can include the underlying blockchain platform, platform product service layer, and application service layer.

Claims

1. A data acquisition format updating method, characterized in that: The steps include: Receiving a standard format document provided by a maintainer, and extracting a format report in the standard format document, wherein the format report records field information contained in the modified data table; Determine a table structure of a modified data table according to one of the format reports, wherein the table structure of the modified data table is specifically determined according to the field information recorded in the format report; Comparing the table structure of the modified data table with that of the corresponding original data table, changing the field information of the original data table according to the table structure of the modified data table, and updating the data collection format; The step of determining the table structure of a modified data table according to the field information recorded in the format report specifically includes: Reading the first line of a format report, and judging the information of each column in the first line according to preset keywords to determine the field information corresponding to each column in the format report; Read the second to the last row of the format report in sequence, and determine the name, attribute information and constraints of a field according to the field information corresponding to each column in each row, so as to determine the table structure of a modified data table; The step of sequentially reading the second to last rows of the format report and determining the name, attribute information and constraints of a field according to the field information corresponding to each column in each row to determine the table structure of a modified data table specifically includes: According to the target row in the format report, extract the field information in the target row, and determine the field name and attribute information according to the field information in the corresponding column; the target row is any row from the second row to the last row; Extracting a characteristic string of the field information in the corresponding column to match the constraint on the corresponding field; Determine the table structure of the modified data table according to the field name, attribute information and constraints of each field in the second to last rows of the format report; Generate a table creation script according to the table structure of the modified data table; The step of comparing the table structure of the modified data table with the corresponding original data table, and changing the field information corresponding to the corresponding original data table according to the table structure of the modified data table to update the data collection format specifically includes: Traverse all original data tables according to the name of the format report to determine the original data table corresponding to the modified data table; Create a modified data table according to the table creation script; Calling the stored procedure on the original data table and the modified data table, and determining the original data table and fields that have been changed relative to the modified data table; Collect data content through changing data tables and fields.

2. The data acquisition format updating method according to claim 1, characterized in that: The step of traversing all original data tables according to the name of the format report to determine the original data table corresponding to the modified data table specifically includes: Parse the name of the modified data table recorded in the format report and match it with the name of the original data table; If no match is successful, extract at least one set of name feature characters in the format report; All original data tables are traversed, the name characteristic characters are matched with the names of the original data tables, the matching degree between the original data table names and the name characteristic characters is determined, and the names of the original data tables and the modified data tables whose matching degree reaches a preset value are output.

3. The data acquisition format updating method according to claim 1, characterized in that: The step of calling a stored procedure on the original data table and the modified data table to determine the original data table and fields that have been changed relative to the modified data table specifically includes: Calling a stored procedure, and matching the fields in the original data table with the fields in the modified data table according to the names and attribute information of the corresponding fields in the original data table and the names and attribute information of the corresponding fields in the modified data table under the stored procedure, so as to match the fields in the original data table with the fields in the modified data table; Compare the attribute information and constraints in the fields to determine the changed fields and the corresponding data tables.

4. The data acquisition format updating method according to claim 1, characterized in that: The step of receiving the standard format document provided by the maintainer and extracting the format report in the standard format document specifically includes: Receive word format documents through Java-poi; Mobilize the API provided by Java-poi to extract the format report in the table format in the word format document; The cell data and row and column information of the format report are extracted to obtain the format report.

5. A data acquisition format updating device, which, when executed, implements the steps of the data acquisition format updating method according to any one of claims 1 to 4, characterized in that: include: A format report acquisition module, used for receiving a standard format document provided by a maintainer, and extracting a format report in the standard format document, wherein the format report records field information of fields included in the modified data table; A table structure determination module, used to determine the table structure of a modified data table according to a format report, wherein the table structure of the modified data table is specifically determined according to the field information recorded in the format report; The data table updating module is used to compare the table structure of the modified data table with that of the corresponding original data table, change the field information of the original data table according to the table structure of the modified data table, and update the data collection format.

6. A computer device, comprising a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the data acquisition format updating method according to any one of claims 1 to 4 when executing the computer program.

7. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the data acquisition format updating method according to any one of claims 1 to 4 are implemented.

Images