A method and system for realizing statistical data standardization

Abstract

The present invention relates to the field of data processing technology, and specifically discloses a method and system for realizing statistical data standardization, which obtains data and builds a database, identifies nouns in the data, generates a number of object names, generates a unique corresponding code for the object name according to a library table setting instruction, and generates a library table according to the object name and the code. After obtaining a data query instruction, the noun in the data query instruction can be matched with the object name in the library table to determine the corresponding code, and then all the object names corresponding to the code can be selected, and then the query is performed in the database to obtain the query result, or all the object names corresponding to the code can be screened out according to the code in the data query instruction, and then the query is performed in the database according to the object name to obtain the query result. The present invention realizes statistical data standardization and accurate screening and query, which provides great convenience and accuracy for data analysis work.

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and in particular to a method and system for implementing statistical data standardization. Background Art

[0002] Before data analysis is carried out, even if people have obtained comprehensive statistical data and information, the names of indicators in various industries are not expressed in the same way in these materials, which often leads to omissions in data statistics, directly affecting the accuracy of data statistical analysis and bringing great difficulties to data statistical work.

[0003] Therefore, based on the above problems, how to standardize statistical data is an urgent problem to be solved. Summary of the invention

[0004] In view of the technical problems in the prior art, the present invention provides a method and system for realizing statistical data standardization.

[0005] The first aspect of the present invention is a method for implementing statistical data standardization, comprising the steps of:

[0006] Obtain data and build database;

[0007] Identify nouns in the data and generate a number of object names;

[0008] Generate a unique corresponding code for the object name according to the library table setting instruction, and generate a library table according to the object name and the code; the codes of the object names with the same meaning are the same;

[0009] Storing the library table in the database;

[0010] Obtaining a data query instruction, and matching a noun in the data query instruction with an object name in the library table to obtain a first object name;

[0011] Determine the code corresponding to the first object name according to the first object name, and then filter out all object names corresponding to the code according to the determined code to obtain a second object name;

[0012] Search the database according to the second object name to obtain a query result;

[0013] Or, obtain a data query instruction, and filter out all object names corresponding to the code according to the code in the data query instruction;

[0014] A query is performed in the database according to the object name to obtain a query result.

[0015] Furthermore, the method further comprises:

[0016] Generate dimension information for the object name according to the library table setting instruction, and generate a library table according to the object name, the code and the dimension information;

[0017] and, according to the dimension information in the data query instruction, filtering out object names with consistent dimension information from the second object names to obtain a third object name;

[0018] A query is performed in the database according to the third object name to obtain a query result.

[0019] Furthermore, the method further comprises:

[0020] Identifying time data contained in the data material and generating a time tag for the corresponding data material;

[0021] The time stamp is stored in the database.

[0022] Furthermore, the method further comprises:

[0023] Generate a time screening condition according to the time information in the data query instruction;

[0024] The data materials whose time tags meet the time screening conditions are screened out from the query results as the final query results.

[0025] Furthermore, the method further comprises:

[0026] Identifying the regional data contained in the data material and generating a regional label for the corresponding data material;

[0027] The region labels are stored in the database.

[0028] Furthermore, the method further comprises:

[0029] Generate a region screening condition according to the region information in the data query instruction;

[0030] The data materials whose region labels meet the region screening conditions are screened out from the query results as the final query results.

[0031] Furthermore, the method further comprises:

[0032] Modify and delete the object name and its corresponding code in the library table according to the library table modification instruction;

[0033] Or, modify and delete the object name in the library table and its corresponding encoding and dimension information according to the library table modification instruction.

[0034] Furthermore, the method further comprises:

[0035] Generate a unique corresponding code for the new object name in the library table according to the library table new addition instruction, and update the library table according to the newly added object name and the code;

[0036] Or, according to the library table new addition instruction, a corresponding unique code and dimension information are generated for the new object name in the library table, and the library table is updated according to the newly added object name, the code and the dimension information.

[0037] Furthermore, the nouns in the data are identified to generate several object names, including:

[0038] Retrieving the industry standard file corresponding to the field according to the field characteristics of the data; the industry standard file records the industry specific terms;

[0039] The nouns in the data are identified according to the industry standard document, and the nouns that are consistent with the industry-specific nouns are marked and used as the object names.

[0040] The second aspect of the present invention is a system for implementing statistical data standardization, which implements the above-mentioned method for implementing statistical data standardization.

[0041] The present invention provides a method and system for implementing statistical data standardization. First, data information is obtained and a database is constructed. Then, nouns in the data information are identified to generate a plurality of object names. Then, a unique corresponding code is generated for the object name according to a library table setting instruction. A library table is generated according to the object name and the code, and stored in the database. Thus, a database and a library table with complete content are constructed to standardize the statistical data. Therefore, after obtaining a data query instruction, the noun in the data query instruction can be matched with the object name in the library table to obtain a first object name, and then the corresponding code is determined according to the first object name, and then all object names corresponding to the code are screened out according to the determined code to obtain a second object name, and the query is performed in the database according to the second object name to obtain the query result; or all object names corresponding to the code can be screened out according to the code in the data query instruction, and then the query is performed in the database according to the object name to obtain the query result. In the present invention, the codes of object names with the same meaning are the same, so all data information that meets the requirements can be queried through the content of the data query instruction, thereby realizing statistical data standardization and accurate screening and query, which provides great convenience and accuracy for data analysis. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0043] Figure 1 A flowchart of a method for implementing statistical data standardization according to an embodiment of the present invention;

[0044] Figure 2 This is a flowchart of a method for implementing statistical data standardization according to another embodiment of the present invention. DETAILED DESCRIPTION

[0045] The following will be combined with the accompanying drawings in the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work belong to the protection scope of the present invention.

[0046] A method for implementing statistical data standardization in an embodiment of the present invention is implemented by running a computer, such as Figure 1 As shown, the steps include:

[0047] Step S01: Acquire data and build a database.

[0048] The data information in this embodiment refers to the relevant data information selected for a certain industry or for specific data analysis needs, involving various reports, information, articles, etc. The data information collection process can be manually searched on a search engine and downloaded to a server, or manually scanned and text-recognized some offline public materials and then uploaded to the server; it can also be automatically searched and scanned by a computer program on the network platform and the local storage area of ​​the computer, and the relevant files are stored in the server after being screened. The database in this embodiment is stored in the server, and the storage time and source can be marked when storing. After the database is built, step S20 is executed.

[0049] Step S02: Identify nouns in the data and generate several object names.

[0050] This step is to identify relevant nouns in the data. These nouns are some transfer nouns that can reflect the industry in which they are located, such as GDP, gross domestic product, regional gross product, public fiscal budget revenue, net fixed assets, etc. of the economic industry.

[0051] Specifically, this step includes:

[0052] Step S021: retrieve the industry standard file corresponding to the field according to the field characteristics of the data.

[0053] Step S022: Identify nouns in the data according to industry standard documents, mark nouns that are consistent with industry-specific nouns, and use them as object names.

[0054] The industry standard file in this embodiment records industry specific terms, so names that are consistent with industry specific terms can be found through text recognition and matching, and these terms are used as object names. Industry standard files can be obtained in the same way as data materials, or they can be manually combed and formed into files readable by computer programs. When the embodiment of the present invention is executed, the corresponding industry standard files are retrieved by the computer program to realize the recognition and generation of object names.

[0055] Step S03: Generate a unique corresponding code for the object name according to the library table setting instruction, and generate a library table according to the object name and the code.

[0056] The embodiment of the present invention does not limit the specific encoding generation method. Preferably, the encoding is generated in the following manner:

[0057] First, the format of the code is set, which can contain numbers, letters, and special symbols, such as brackets, slashes, underscores, etc. In order to make the code content consistent with the number of bits, a binary or hexadecimal number of several bits can be used. The embodiment of the present invention divides the code into a field identification part, a name identification part, and a custom part. The field identification part refers to the code of the field to which the object name belongs. It is used to distinguish between different fields where the text content of the object name is the same. For example, A and B represent different fields respectively. The name identification part corresponds to the text content of the object name. The embodiment of the present invention does not recommend directly generating the corresponding binary number according to the text content, which will greatly increase the number of bits of the code. After the object name is divided into the field to which it belongs, the possibility of all or part of the text content of the object name is very small, so it can be formed with a number with fewer bits to form a corresponding relationship with the text content, so as to ensure the uniqueness of the code. For example, 0001 corresponds to "whether it is regarded as a legal person" and 0012 corresponds to "unit scale". For the custom part, special marks or identifications of the object name can be reflected, such as underscores, brackets, etc. This part is set as needed.

[0058] According to the set encoding format, the encoding of the object name is generated accordingly. It should be emphasized that in the steps of the embodiment of the present invention, the encodings of the object names with the same meaning are the same.

[0059] The library table of the embodiment of the present invention includes object names and their corresponding codes. Preferably, the library table also includes serial numbers, code generation time, attributes of the object name (such as text, numbers, symbols, etc.), the name of the field to which the object name belongs, etc.

[0060] Step S04: Store the library table in the database.

[0061] After the library table is generated, it is stored in a database that stores data.

[0062] After that, you can query the relevant information through data query instructions. Specifically:

[0063] Step S05: Obtain a data query instruction, and match the noun in the data query instruction with the object name in the library table to obtain a first object name.

[0064] The data query instruction in this embodiment can be generated by the user by operating the computer. The data query instruction is the trigger condition for the data query and also the embodiment of the specific query demand. For example, the data query instruction in the embodiment of the present invention contains a noun to be queried, and the noun is matched with the object name in the library table, and the matched object name is called the first object name.

[0065] Step S06: determining the code corresponding to the first object name, and then filtering out all object names corresponding to the code according to the determined code to obtain a second object name.

[0066] In combination with the foregoing embodiment, there are situations where different object names have the same meaning. Since the codes of object names with the same meaning are the same, the first object name can be used to first determine its corresponding unique code, and then all other object names with the same meaning are filtered out based on the code, and all matching object names are called second object names.

[0067] Step S07: query the database according to the second object name to obtain a query result.

[0068] The second object name is indexed in the data in the database to find the query result. The query result of the embodiment of the present invention is not limited in the way it is presented. For example, it is a text information list of all data containing the second object name. The list may include the file name, file upload time, file content time, data corresponding to the second object name, file viewing link, etc. A picture information list can also be used to design a function to zoom in / out the picture. The picture is a screenshot of the file according to the set frame size. The specific screenshot position is the location area where the second object name appears in the file. In this way, a file viewing function is also designed to facilitate users to view and check the original file.

[0069] Or Figure 2 As shown, another implementation method of querying related information through data query instructions is as follows:

[0070] Step S01: Acquire data and build a database.

[0071] Step S02: Identify nouns in the data and generate several object names.

[0072] Step S03: Generate a unique corresponding code for the object name according to the library table setting instruction, and generate a library table according to the object name and the code.

[0073] Step S04: Store the library table in the database.

[0074] For the process of generating the library table and establishing the database, reference can be made to the description of the above-mentioned embodiment, which will not be repeated here. This embodiment focuses on another implementation process of data query.

[0075] Step S08: Obtain a data query instruction, and filter out all object names corresponding to the code according to the code in the data query instruction.

[0076] The difference from the previous embodiment is that the data query instruction in this embodiment includes a code, so all object names corresponding to the code can be filtered out through the code.

[0077] Step S09: query the database according to the object name to obtain the query result.

[0078] Then, all the object names corresponding to the selected codes are indexed in the data in the database to find the query result. The query result is also as described in the example of the previous embodiment.

[0079] This embodiment uses codes for direct indexing, which simplifies the data query process. This method is more suitable for users who are familiar with codes.

[0080] In addition, the code in the data query instruction of this embodiment can be either the code generated in step S30 and uniquely corresponding to the object name, or part of the code, which is set specifically according to the data retrieval requirements. For example, the code in the data query instruction can be the "name identification part" of the entire code, or both the "field identification part" and the "name identification part".

[0081] The implementation method of statistical data standardization of the embodiment of the present invention first obtains data and builds a database, then identifies the nouns in the data, generates several object names, and then generates a unique corresponding code for the object name according to the library table setting instruction, and generates a library table according to the object name and the code, and stores it in the database, so as to construct a database and library table with complete content, and standardize the statistical data. Therefore, after obtaining the data query instruction, the noun in the data query instruction can be matched with the object name in the library table to obtain the first object name, and then the corresponding code is determined according to the first object name, and then all the object names corresponding to the code are screened out according to the determined code to obtain the second object name, and the query is performed in the database according to the second object name to obtain the query result; or all the object names corresponding to the code can be screened out according to the code in the data query instruction, and then the query is performed in the database according to the object name to obtain the query result. In the present invention, the codes of object names with the same meaning are the same, so all data information that meets the requirements can be queried through the content of the data query instruction, thereby realizing statistical data standardization and accurate screening and query, which provides great convenience and accuracy for data analysis.

[0082] In addition, a method for implementing statistical data standardization in an embodiment of the present invention further includes:

[0083] Step S10: Generate dimension information for the object name according to the library table setting instruction, and generate a library table according to the object name, code and dimension information.

[0084] The dimension information in this embodiment can be understood as a dimension that can be defined by an object name, such as a region dimension, a year dimension, etc. This embodiment adds dimension information based on the previous embodiment, and also reflects the dimension information in the library table.

[0085] And, step S11: according to the dimension information in the data query instruction, filter out object names with consistent dimension information from the second object names to obtain a third object name.

[0086] If there is a retrieval demand for a dimension, after all object names that meet the requirement are queried, unread information is filtered to obtain a third object name.

[0087] Step S12: querying the database according to the third object name to obtain a query result.

[0088] Generally, when searching for data, it is not only necessary to query the object name, but also to limit the dimension. In this embodiment, a specific example is as follows:

[0089] Assuming that the object name contained in the data query instruction is "GDP", through the aforementioned steps S50 and S60, the second object name obtained is all object names including "GDP" and "gross domestic product". If the dimension information is a regional dimension, specifically "Shenzhen", then through step S11, the third object name screened out is "Shenzhen GDP" and "Shenzhen Gross Domestic Product". Then, through the third object name, data information related to "Shenzhen GDP" and "Shenzhen Gross Domestic Product" can be queried in all data materials.

[0090] In addition, based on the above-mentioned embodiment, a method for implementing statistical data standardization in an embodiment of the present invention further includes:

[0091] Step S13: Identify the time data contained in the data material and generate a time tag for the corresponding data material.

[0092] Step S14: Store the time tag in the database.

[0093] Based on this embodiment, it is possible to query information not only through the object name and code contained in the data query instruction, but also to use time as a query dimension in the data query instruction and match it with the time tag of the data material, thereby obtaining data results under more search conditions.

[0094] Specifically, a method for implementing statistical data standardization in an embodiment of the present invention further includes:

[0095] Step S15: Generate a time screening condition according to the time information in the data query instruction.

[0096] Step S16: Filter out data materials whose time tags meet the time screening conditions from the query results as the final query results.

[0097] In addition, based on the above-mentioned embodiment, a method for implementing statistical data standardization in an embodiment of the present invention further includes:

[0098] Step S17: Identify the regional data contained in the data material and generate a regional label for the corresponding data material.

[0099] Step S18: Store the region label in the database.

[0100] This embodiment is similar to the previous embodiment, and uses the region dimension as the region label, so there is no need to implement the screening of the region dimension in the manner of step S11 and step S12. Instead, the region data contained in the data is first identified and the region label is generated, so that the data with the corresponding region label is screened out in the query results. Specifically:

[0101] Step S19: Generate a region screening condition according to the region information in the data query instruction.

[0102] Step S20: Filter out data with region tags that meet the region screening conditions from the query results as the final query results.

[0103] In addition, based on the above-mentioned embodiment, a method for implementing statistical data standardization in an embodiment of the present invention further includes:

[0104] Step S21: modify and delete the object name and its corresponding code in the library table according to the library table modification instruction.

[0105] Or, step S22: modify and delete the object name in the library table and its corresponding code and dimension information according to the library table modification instruction.

[0106] After the library table is generated, it can be modified and deleted as needed to ensure that the latest version of the library table best meets the current data statistical analysis needs.

[0107] In addition, based on the above-mentioned embodiment, a method for implementing statistical data standardization in an embodiment of the present invention further includes:

[0108] Step S23: Generate a unique corresponding code for the new object name in the library table according to the library table new addition instruction, and update the library table according to the newly added object name and code.

[0109] Or, step S24: generating a corresponding unique code and dimension information for the new object name in the library table according to the library table new addition instruction, and updating the library table according to the newly added object name, code and dimension information.

[0110] This embodiment takes into account the addition of new contents in the library table. As the amount of data in the database increases, the statistical analysis requirements increase and become higher. It is necessary to continuously improve the object names and corresponding codes in the library table to support more massive data analysis work.

[0111] An embodiment of the present invention also includes a system for implementing statistical data standardization. The system of this embodiment includes at least a memory and a processor. The memory is used to store computer programs and data information, library tables, etc. of the aforementioned embodiments, and the processor is used to call and run computer programs to implement the methods of the aforementioned embodiments.

[0112] The present invention is further described above with the aid of specific embodiments. However, it should be understood that the specific description here should not be construed as limiting the essence and scope of the present invention. Various modifications made to the above embodiments by ordinary technicians in the field after reading this specification are all within the scope of protection of the present invention.

Claims

1. A method for implementing statistical data standardization, characterized in that: Includes steps: Obtain data and build database; Identify nouns in the data and generate a number of object names; Generate a unique corresponding code for the object name according to the library table setting instruction, and generate a library table according to the object name and the code; the codes of the object names with the same meaning are the same; the code is divided into a field identification part, a name identification part and a custom part; Storing the library table in the database; Obtaining a data query instruction, and matching a noun in the data query instruction with an object name in the library table to obtain a first object name; Determine the code corresponding to the first object name according to the first object name, and then filter out all object names corresponding to the code according to the determined code to obtain a second object name; Search the database according to the second object name to obtain a query result; Or, obtain a data query instruction, and filter out all object names corresponding to the code according to the code in the data query instruction; Search the database according to the object name to obtain a query result; The nouns in the data are identified to generate several object names, including: Retrieving the industry standard file corresponding to the field according to the field characteristics of the data; the industry standard file records the industry specific terms; Identify nouns in the data according to the industry standard document, mark nouns that are consistent with the industry specific nouns, and use them as the object names; The method further comprises: Generate dimension information for the object name according to the library table setting instruction, and generate a library table according to the object name, the code and the dimension information; and, according to the dimension information in the data query instruction, filtering out object names with consistent dimension information from the second object names to obtain a third object name; A query is performed in the database according to the third object name to obtain a query result.

2. A method for implementing statistical data standardization according to claim 1, characterized in that: The method further comprises: Identifying time data contained in the data material and generating a time tag for the corresponding data material; The time stamp is stored in the database.

3. A method for implementing statistical data standardization as claimed in claim 2, characterized in that: The method further comprises: Generate a time screening condition according to the time information in the data query instruction; The data materials whose time tags meet the time screening conditions are screened out from the query results as the final query results.

4. A method for implementing statistical data standardization according to claim 1, characterized in that: The method further comprises: Identifying the regional data contained in the data material and generating a regional label for the corresponding data material; The region labels are stored in the database.

5. A method for implementing statistical data standardization as claimed in claim 2, characterized in that: The method further comprises: Generate a region screening condition according to the region information in the data query instruction; The data materials whose region labels meet the region screening conditions are screened out from the query results as the final query results.

6. A method for implementing statistical data standardization according to claim 1, characterized in that: Also includes: Modify and delete the object name and its corresponding code in the library table according to the library table modification instruction; Or, modify and delete the object name in the library table and its corresponding encoding and dimension information according to the library table modification instruction.

7. A method for implementing statistical data standardization according to claim 1, characterized in that: Also includes: Generate a unique corresponding code for the new object name in the library table according to the library table new addition instruction, and update the library table according to the newly added object name and the code; Or, according to the library table new addition instruction, a corresponding unique code and dimension information are generated for the new object name in the library table, and the library table is updated according to the newly added object name, the code and the dimension information.

8. A statistical data standardization implementation system, characterized in that: The system implements the method for standardizing statistical data as described in any one of claims 1 to 7.

Images