Data product automatic management method and device based on data specifications

Abstract

The invention provides a data product automatic management method and device based on data specifications, relates to the technical field of data processing, and aims at solving the technical problem that a traditional data management platform is low in automation degree. The method comprises the steps that in response to receiving of a data product physical model uploaded externally, the data product physical model is analyzed, description information of a data product is obtained, and the data product physical model represents standard features of the data product; according to the description information, a library table needed by data storage management is created, and the library table is configured to be capable of storing data products in a structured mode according to specifications of the description information; creating a data storage program corresponding to the library table according to the library table; responding to a retrieval request initiated by a user, and generating a data query program corresponding to the retrieval condition according to the retrieval condition; and realizing automatic management of data products according to the data storage program and the data query program.

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and more specifically, to a method and device for automated management of data products based on data specifications. Background Art

[0002] With the development of information technology, the scale of data product production has rapidly increased, placing increasing pressure on data product management. However, traditional data management systems often rely heavily on manual operations, requiring human intervention at many steps, from data collection, cleaning, and conversion to final storage and query. This results in the economic costs of system construction and maintenance increasing exponentially with system scale. Furthermore, this manual involvement makes it difficult to ensure efficient development and agile response. Furthermore, manual operations inevitably lead to errors and omissions, posing risks to data security. Consequently, the current approach to building data management systems is gradually moving towards automation.

[0003] The related technology has implemented a visual configuration tool that supports manual configuration of data models and business statistics. The system automatically builds a data warehouse and generates data statistical query statements based on the configuration. The related technology also implements a system that completes the configuration of data resource management and governance through web pages, and builds a distributed data cleaning and governance program through secondary development on the back end. The above-mentioned related technologies have achieved a certain degree of automated construction of data management systems, but there is still a lot of manual participation. To solve the above problems, it is necessary to reduce the degree of manual participation in the data management system construction process as much as possible. Therefore, there is an urgent need for a fully automatic data management system construction method and a platform to achieve this function. Summary of the Invention

[0004] In view of this, the present invention provides a method and device for automated management of data products based on data specifications, aiming to solve the technical problem of low automation level of traditional data management platforms.

[0005] One aspect of the present invention provides a method for automated management of data products based on data specifications, which is applied to a data management system. The method includes: in response to receiving a physical model of a data product uploaded from an external source, parsing the physical model of the data product to obtain description information of the data product, wherein the physical model of the data product represents the specification characteristics of the data product; based on the description information, creating a library table required for data storage management, wherein the library table is configured to store data products in a structured manner in accordance with the specifications of the description information; based on the library table, creating a data entry program corresponding to the library table; in response to a search request initiated by a user, generating a data query program corresponding to the search conditions according to the search conditions; and based on the data entry program and the data query program, realizing automated management of data products.

[0006] According to an embodiment of the present invention, before creating the library table required for data storage management based on the description information, the method further includes: storing the description information in a data product specification parsing result table.

[0007] According to an embodiment of the present invention, storing the description information in the data product specification parsing result table includes: storing the product information in the description information in the product information table; and storing the product attribute information in the description information in the attribute information table.

[0008] According to an embodiment of the present invention, creating a library table required for data storage management based on description information includes: determining whether a corresponding product database table exists based on product information; in response to the corresponding product database table not existing, generating a first structured query language for creating the library table based on the product information and product attribute information; in response to the corresponding product database table existing, generating a second structured query language for updating the library table based on the product information and product attribute information; and creating the library table required for data storage management based on the first structured query language or the second structured query language.

[0009] According to an embodiment of the present invention, based on the library table, creating a data entry program corresponding to the library table includes: generating an entry request interface and an entry result feedback interface based on product information, and the IP and port in the data product physical model; generating a data verification program based on product attribute information and attribute verification rules; generating a data entry program based on product attribute information and a structured query language template library.

[0010] According to an embodiment of the present invention, in response to a user initiating a retrieval request, generating a data query program corresponding to the retrieval conditions according to the retrieval conditions includes: generating interface rendering logic according to product attribute information, wherein different attribute types render different logic; in response to a user initiating a retrieval request through an operation interface, generating a data query program according to the retrieval conditions.

[0011] According to an embodiment of the present invention, the method also includes: obtaining a test data product uploaded together with a data product physical model; obtaining corresponding test product attribute information, a test data storage program, and a test data query program based on the test product information of the test data product; testing the data management system using the test product information, the test product attribute information, the test data storage program, and the test data query program; in response to the test passing, updating the data management system and automatically going online.

[0012] Another aspect of the present application provides a data product automatic management device based on data specification, the device comprising: an analysis module configured to analyze a data product physical model received externally to obtain description information of the data product, wherein the data product physical model represents the specification characteristics of the data product; a first creation module configured to create a library table required for data storage management according to the description information, wherein the library table is configured to store the data product in a structured manner according to the specification of the description information; a second creation module configured to create a data loading program corresponding to the library table according to the library table; a generation module configured to generate a data query program corresponding to a retrieval condition according to the retrieval condition in response to a retrieval request initiated by a user; and a management module configured to implement automatic management of the data product according to the data loading program and the data query program.

[0013] Another aspect of the present application provides an electronic device comprising: one or more processors; a memory storing one or more programs, wherein the one or more programs, when executed by the one or more processors, cause the one or more processors to implement the method as described above.

[0014] Another aspect of the present application provides a computer-readable storage medium storing computer-executable instructions, which, when executed, implement the method as described above.

[0015] Another aspect of the present application provides a computer program product comprising computer-executable instructions, which, when executed, implement the method as described above.

[0016] Compared with the prior art, the data product automatic management method and device based on data specification provided by the embodiments of the present application have at least the following beneficial effects:

[0017] The data product automatic management method and device based on data specification provided by the embodiments of the present application import a pre-configured data product physical model (data product specification characteristics), which standardizes the description of product attributes, types, storage requirements, and query interfaces, etc. Based on these specifications, the system can automatically generate a configured loading scheme and query logic without human intervention, greatly reducing the labor cost and avoiding data security risks caused by human errors. The method and device can be widely applied to data product processing and production systems, and thus can realize fully automated data product management. BRIEF DESCRIPTION OF DRAWINGS

[0018] The above and other objects, features and advantages of the present application will become more apparent from the following description of embodiments of the present application, taken in conjunction with the accompanying drawings, in which:

[0019] Figure 1Schematically shows a flow chart of a method for automated management of data products based on data specifications according to an embodiment of the present invention;

[0020] Figure 2 Schematically shows a structural diagram of a data product specification parsing result table in a data product automatic management method based on data specifications according to an embodiment of the present invention;

[0021] Figure 3 Schematically shows a flow chart for automatically creating a library table required for data storage management based on description information according to an embodiment of the present invention;

[0022] Figure 4 Schematically shows a flowchart of automatically creating a data storage program corresponding to a library table according to an embodiment of the present invention;

[0023] Figure 5 Schematically shows a flow chart of generating a data storage request and storage result feedback interface according to an embodiment of the present invention;

[0024] Figure 6 Schematically shows a flow chart of automatically generating a data query program corresponding to a search condition according to a search condition according to an embodiment of the present invention;

[0025] Figure 7 Schematically shows a flow chart of a data management system automatically generating a test program and executing a test according to an embodiment of the present invention;

[0026] Figure 8 Schematically shows an interface diagram of a data management system according to an embodiment of the present invention;

[0027] Figure 9 A schematic diagram shows a structural block diagram of a data product automatic management device based on data specifications according to an embodiment of the present invention;

[0028] Figure 10 The structure block diagram of an electronic device suitable for implementing the data product automatic management method based on data specifications according to an embodiment of the present invention is schematically shown. DETAILED DESCRIPTION

[0029] Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of the present invention. In the following detailed description, for ease of explanation, many specific details are set forth to provide a comprehensive understanding of embodiments of the present invention. However, it is apparent that one or more embodiments may also be implemented without these specific details. In addition, in the following description, descriptions of known structures and technologies are omitted to avoid unnecessary confusion of the concept of the present invention.

[0030] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. As used herein, the terms "comprises", "comprising", "includes", "including" and the like are specifically intended to be open-ended terms meaning that other elements can be added.

[0031] All terms used herein including technical and scientific terms have the meanings commonly understood by one of ordinary skill in the art unless otherwise specified. It should be noted that the use of terms herein, such as should be construed to have meanings consistent with the context of the specification and relevant work in the field.

[0032] In the case using expressions similar to "at least one of A, B and C, etc.", it will be generally understood that one of ordinary skill in the art will recognize the context and interpret the description accordingly. For example, "a system having at least one of A, B, and C" can be interpreted as including, but not limited to, a system that includes A alone, a system that includes B alone, a system that includes C alone, a system that includes both A and B, a system that includes both A and C, a system that includes both B and C, and / or a system that includes all of A, B, and C, etc.

[0033] In embodiments of the present application, the collection, updating, analysis, processing, use, transmission, provision, disclosure, storage, etc. of the data involved (for example, including but not limited to user personal information) comply with relevant laws and regulations, are used for legal purposes, and do not violate public order and good customs. In particular, necessary measures are taken to prevent illegal access to user personal information data and to maintain user personal information security and network security.

[0034] With the development of information technology, the production scale of data products is rapidly increasing, and the pressure on data product management is gradually increasing. However, the traditional data management system construction method often uses manual methods and relies heavily on manual operation. From data collection, cleaning, conversion to final storage and query, many steps require manual intervention, which leads to exponential expansion of the economic cost of system construction and maintenance with the system scale. And due to human involvement, the efficiency of development and agile response is difficult to guarantee, and errors and omissions are inevitable due to manual operation, which poses a potential risk to data security. Therefore, the current data management system construction method has gradually moved towards the direction of automated construction.

[0035] The related technology has implemented a visual configuration tool that supports manual configuration of data models and business statistics. The system automatically builds a data warehouse and generates data statistical query statements based on the configuration. The related technology also implements a system that completes the configuration of data resource management and governance through web pages, and builds a distributed data cleaning and governance program through secondary development on the back end. The above-mentioned related technologies have achieved a certain degree of automated construction of data management systems, but there is still a lot of manual participation. To solve the above problems, it is necessary to reduce the degree of manual participation in the data management system construction process as much as possible. Therefore, there is an urgent need for a fully automatic data management system construction method and a platform to achieve this function.

[0036] Based on this, an embodiment of the present invention provides a data product automated management method based on data specifications, aiming to solve the technical problem of low automation level of traditional data management platforms.

[0037] An automated data product management method based on data specifications according to an embodiment of the present invention is applied to a data management system, the method comprising: in response to receiving a data product physical model uploaded from an external source, parsing the data product physical model to obtain description information of the data product, wherein the data product physical model represents the specification features of the data product; creating a library table required for data storage management based on the description information, wherein the library table is configured to store the data product in a structured manner in accordance with the specifications of the description information; creating a data entry program corresponding to the library table based on the library table; in response to a search request initiated by a user, generating a data query program corresponding to the search conditions based on the search conditions; and realizing automated management of the data product based on the data entry program and the data query program.

[0038] In order to make the objectives, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with specific embodiments and with reference to the accompanying drawings.

[0039] Figure 1 The flowchart of the data product automatic management method based on data specifications according to an embodiment of the present invention is schematically shown.

[0040] like Figure 1 As shown, the data product automatic management method based on data specifications of this embodiment may include, for example, operations S1 to S5, and the method may be applied to a data management system.

[0041] In operation S1 , in response to receiving a data product physical model uploaded from outside, the data product physical model is parsed to obtain description information of the data product, wherein the data product physical model represents the specification features of the data product.

[0042] In operation S2, a library table required for data storage management is created according to the description information, wherein the library table is configured to store data products in a structured manner according to the specification of the description information.

[0043] In operation S3, a data storage program corresponding to the library table is created according to the library table.

[0044] In operation S4, in response to a search request initiated by the user, a data query program corresponding to the search condition is generated according to the search condition.

[0045] In operation S5, automatic management of data products is achieved according to the data storage program and the data query program.

[0046] The data product automated management method and device based on data specifications provided by the embodiments of the present invention import a pre-configured data product physical model (data product specification features). The data product physical model describes the product attributes, type, storage requirements, query interface and other information in a standardized manner. Based on these specifications, the system can automatically generate configured warehousing plans and query logic without human intervention, greatly reducing labor costs and avoiding data security risks caused by human omissions. It can be widely used in data product processing and production systems, and therefore can achieve fully automated data product management.

[0047] In this embodiment, the data product can be, for example, a typical remote sensing data product. The data product physical model can be obtained based on a pre-written data product specification file. The data product specification file describes each type of data product, including the number of attributes in the data product, the type of each attribute, its value range, and the rules that attribute values ​​must conform to based on regular expressions.

[0048] According to an embodiment of the present invention, in response to receiving a data product physical model uploaded from an external source, operation S1 parses the data product physical model to obtain description information of the data product, which may include:

[0049] After receiving the uploaded data product specification file package, define the data product specification as R, define the test data as F, and define the product identification function as f(x).

[0050] The input file R obtains the set P=f(R) and stores the standard parsing results. The set P can be further expressed as P={P1, P2, P3,…,P n}.

[0051] Get the key parameters in set P and define P i For product specification version information, define P j Identifies the product.

[0052] Determine product identification P j Check whether a data management system generated with a different version already exists. If the system is running, suspend it.

[0053] Define the product specification parsing function as , further automatically parse the data product specification file to obtain the data product description information set M, which is further expressed as M={M1,M2,M3,…,M n}.

[0054] According to an embodiment of the present invention, before operation S2 creates the library table required for data storage management according to the description information, it may further include:

[0055] The description information is stored in the data product specification parsing result table. The data product specification parsing result table is as follows: Figure 2 shown.

[0056] Figure 2 The structure of the data product specification parsing result table in the data specification-based data product automatic management method according to an embodiment of the present invention is schematically shown.

[0057] like Figure 2 As shown, in the data product specification parsing result table, the product information in the description information is stored in the product information table, and the product attribute information in the description information is stored in the attribute information table. Specifically:

[0058] Define product information as A, and , stored in the product information table, specifically including fields such as product identification, name, version, and status.

[0059] Define product attribute information as B, and , stored in the attribute information table, specifically including fields such as attribute identifier, name, type, verification rule, product identifier, etc., where the product identifier field is the unique identifier in the product table, and the two tables have a many-to-one relationship.

[0060] Figure 3 The flowchart of automatically creating the library tables required for data storage management according to description information according to an embodiment of the present invention is schematically shown.

[0061] like Figure 3 As shown, according to an embodiment of the present invention, operation S2 may include, for example, creating a library table required for data storage management based on the description information:

[0062] Determine whether the corresponding product database table exists based on product information;

[0063] In response to the corresponding product database table not existing, generating a first structured query language for creating a database table according to the product information and product attribute information;

[0064] In response to the existence of the corresponding product database table, generating a second structured query language for updating the database table according to the product information and product attribute information;

[0065] Create a library table required for data storage management according to the first structured query language or the second structured query language.

[0066] In this embodiment, the product information A in the description information is used i , corresponding product attribute information B i , automatically create the database tables required for data storage management, specifically:

[0067] Get product information A from the product information table i , where the product identifier is defined as , get all the attributes of the product from the attribute information table B α ;

[0068] Use the product identifier α to determine whether the corresponding product data table exists. If not, set α, B α Bring in table creation statement to generate function Generate the first SQL (Structured Query Language) statement for table creation ; If it exists, change α, B α Generate a function by introducing a table update statement Generate the second SQL statement for table update ;

[0069] Finally, the first SQL statement or the second SQL statement is brought into the executor to complete the operation of the product library table.

[0070] Figure 4 The flowchart of automatically creating a data storage program corresponding to a library table according to an embodiment of the present invention is schematically shown.

[0071] like Figure 4 As shown, according to an embodiment of the present invention, operation S3 may include, for example, creating a data storage program corresponding to the library table based on the library table:

[0072] Generate the warehousing request interface and warehousing result feedback interface based on product information, as well as the IP and port in the data product physical model;

[0073] Generate data verification program based on product attribute information and attribute verification rules;

[0074] Generate a data entry program based on product attribute information and structured query language template library.

[0075] In this embodiment, the obtained product identification α and product attribute specification B are used. α Automatically create a data entry program, including an interface for receiving data entry notifications, a data file verification program, and a data management and entry program. Specifically:

[0076] Use the product identifier α and the parameters such as IP and port in the configuration file to generate the storage request interface and storage result feedback interface, such as Figure 5 As shown;

[0077] Using product attribute specification B α , and obtain the attribute verification rules , loop all attributes and generate data verification program;

[0078] Using product attribute specification B α , SQL template library, and obtain key information such as attribute name and type , generate data storage SQL, and further generate data storage program.

[0079] Figure 5 The flowchart of generating a data storage request and a storage result feedback interface according to an embodiment of the present invention is schematically shown.

[0080] like Figure 5 As shown, since the data storage process mode is fixed in this embodiment, it is only necessary to read the configuration information such as the interface IP and port to generate the data storage request and storage result feedback interface.

[0081] Subsequently, the attribute value range and the verification rules described by the regular expression in the data product specification parsing result are read and converted into an executable program. After receiving the data writing completion notification interface, it is executed. Then, a storage SQL statement template is generated according to the data product repository table. The SQL statement is filled in and executed according to the specific attribute data of the product. The data product information is stored in the library table, and finally the result is returned to the user.

[0082] Figure 6 The flowchart of automatically generating a data query program corresponding to the retrieval conditions according to the retrieval conditions is schematically shown in accordance with an embodiment of the present invention.

[0083] like Figure 6 As shown, according to an embodiment of the present invention, in response to a user initiating a search request, operation S4 generates a data query program corresponding to the search condition according to the search condition, for example, including:

[0084] Generate interface rendering logic based on product attribute information, where different attribute types render different logic;

[0085] In response to a search request initiated by a user through an operation interface, a data query program is generated according to the search conditions.

[0086] In this embodiment, the obtained product attribute specification B α , generate interface rendering logic, and automatically create a data query program when receiving search conditions generated according to interface specifications. Specifically:

[0087] Using product attribute specification B α , loop through all product attribute specifications , get the type corresponding to the attribute. Different attribute types require different rendering logic. For example, the Date rendering logic is TimePicker (time selector), the String rendering logic is Input (input box), etc. All conversion results are rendered as an interface and returned to the user;

[0088] When the user operation interface generates the retrieval condition β, the condition conversion function τ(x) is introduced to convert it into a query statement, ε=τ(β) is defined, and the SQL statement ε is passed into the database to generate a data query program.

[0089] It can be seen that the data product automated management method based on data specifications provided by the embodiment of the present invention can automatically generate configured warehousing plans and query logic based on data specifications, thereby realizing fully automated data product management.

[0090] Figure 7 The flowchart of the data management system automatically generating a test program and executing a test according to an embodiment of the present invention is schematically shown.

[0091] like Figure 7 As shown, according to an embodiment of the present invention, the data product automated management method based on data specifications further includes:

[0092] Obtain the test data product uploaded together with the data product physical model;

[0093] According to the test product information of the test data product, obtain the corresponding test product attribute information, test data storage program and test data query program;

[0094] Test the data management system using test product information, test product attribute information, test data storage procedures, and test data query procedures;

[0095] In response to the test passing, the data management system is updated and automatically put into operation.

[0096] In this embodiment, the product attribute specification library B is parsed by using the test data product F uploaded at the same time. α , automatically generate data product repository, parse storage program, query program ε, automatically create test process, run the test process and give test report, specifically:

[0097] Using product identifier α, obtain product attribute specification B α , data entry management program , data query program ε;

[0098] Substitute all parameters into the automatic test function ω(x,y,z), and further express the test result ξ=ω(α, B α ,ε), in the test function ω(x,y,z), it is defined that if all test items pass, the test passes, otherwise the test fails, and finally a test report is given;

[0099] If the test result is passed, the system is updated and automatically put into operation; otherwise, the process is terminated and a prompt is given that the system build / update has failed.

[0100] Figure 8 The following schematically shows an interface diagram of a data management system according to an embodiment of the present invention.

[0101] like Figure 8 As shown, the data management system of the embodiment of the present invention clearly displays product grouping, search conditions, search results, etc., giving users a visual display.

[0102] Figure 9 The structure block diagram of the data product automatic management device based on data specifications according to an embodiment of the present invention is schematically shown.

[0103] like Figure 9 As shown, the data product automatic management device 900 based on data specifications according to the embodiment of the present invention includes: a parsing module 910 , a first creation module 920 , a second creation module 930 , a generation module 940 and a management module 950 .

[0104] The parsing module 910 is used to parse the data product physical model in response to receiving an externally uploaded data product physical model to obtain description information of the data product, wherein the data product physical model represents the standard features of the data product.

[0105] In this embodiment, the parsing module 910 is configured to format and parse the input data product specification file to obtain attribute descriptions of the data product. The data product specification file describes each type of data product, including the number of attributes in the data product, the type of each attribute, its value range, and the rules that attribute values ​​expressed based on regular expressions must comply with. The parsing results are stored in the data product specification parsing result table.

[0106] The first creation module 920 is used to create a library table required for data storage management according to the description information, wherein the library table is configured to store data products in a structured manner according to the specifications of the description information.

[0107] In this embodiment, the first creation module 920 is configured to monitor the data product specification parsing result table. When the table changes, that is, when the data product changes, the data product specification parsing result table is read, and fields such as data category identifier, data category name, attribute identifier, attribute name, attribute type, etc. are extracted from it, and it is determined whether the database table already exists. If so, the current structure of the database table is extracted, and database table creation and update statements are generated based on the above information, and the database table creation or update statements are executed to create or update the data product storage management library table.

[0108] The second creation module 930 is used to create a data storage program corresponding to the library table according to the library table.

[0109] In this embodiment, the second creation module 930 includes sub-functions such as warehousing process interface generation, data file verification program generation, and data management program generation. Regarding the warehousing process interface generation sub-function, since the data warehousing process mode is fixed in this embodiment, it is only necessary to read the configuration information such as the interface IP and port to generate the data warehousing request and warehousing result feedback interface. Regarding the automatic generation sub-function of the data file verification program, the attribute value domain range and the verification rules described by the regular expression in the data product specification parsing result are read, and converted into an executable program, which is executed after receiving the data write completion notification interface. Regarding the data management program generation sub-function, it is responsible for generating a warehousing SQL statement template based on the data product storage library table, filling in the SQL statement according to the specific attribute data of the product and executing it, and storing the data product information in the library table.

[0110] The generation module 940 is used to generate a data query program corresponding to the search condition in response to a search request initiated by a user.

[0111] In this embodiment, the generation module 940 is configured to automatically create data retrieval query SQL statements and data retrieval query interfaces, wherein the query SQL statements are encapsulated in a unified template and support multi-attribute combination queries, and the data retrieval query interface automatically calls the corresponding components to generate data based on the attributes and attribute types of the data.

[0112] The management module 950 is used to realize the automated management of data products according to the data storage program and the data query program.

[0113] In this embodiment, in addition to being used to realize automated management of data products according to the data warehousing procedure and data query procedure, the management module 950 is also used to confirm the data products uploaded together with the data product specifications after the system is set up, and then create one or more test processes according to the number of test data products, use each test data product to test, and verify the system's data warehousing, verification, management, query and other functions in turn. For each test process, first generate the results of the database and interface after the test process is executed through logical calculation, and then execute the test process to check whether the program reports an error during operation. If no error is reported, compare the expected result with the result after execution. If they are the same, the test process passes. Finally, combine the test results of all test processes to give a test report. When all test processes pass, the test as a whole passes.

[0114] Any number of the modules, submodules, units, and subunits according to embodiments of the present invention, or at least part of the functionality of any number of these units, can be implemented in a single module. Any one or more of the modules, submodules, units, and subunits according to embodiments of the present invention can be split into multiple modules for implementation. Any one or more of the modules, submodules, units, and subunits according to embodiments of the present invention can be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on a chip, a system on a substrate, a system on a package, an application-specific integrated circuit (ASIC), or can be implemented in hardware or firmware using any other reasonable method of circuit integration or packaging, or can be implemented in any one of the three implementation methods of software, hardware, and firmware, or any appropriate combination of any of these. Alternatively, one or more of the modules, submodules, units, and subunits according to embodiments of the present invention can be at least partially implemented as a computer program module that, when executed, can perform the corresponding functionality.

[0115] For example, any number of the parsing module 910, the first creation module 920, the second creation module 930, the generation module 940, and the management module 950 can be combined into a single module / unit / sub-unit, or any one of these modules / units / sub-units can be split into multiple modules / units / sub-units. Alternatively, at least part of the functionality of one or more of these modules / units / sub-units can be combined with at least part of the functionality of other modules / units / sub-units and implemented in a single module / unit / sub-unit. According to an embodiment of the present invention, at least one of the parsing module 910, the first creation module 920, the second creation module 930, the generation module 940, and the management module 950 can be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on a chip, a system on a substrate, a system on a package, an application-specific integrated circuit (ASIC), or can be implemented in hardware or firmware through any other reasonable means of circuit integration or packaging, or can be implemented in any one of software, hardware, and firmware, or any suitable combination of any of these. Alternatively, at least one of the parsing module 910 , the first creation module 920 , the second creation module 930 , the generation module 940 and the management module 950 may be at least partially implemented as a computer program module, which may perform corresponding functions when executed.

[0116] It should be noted that the part of the data product automated management device based on data specifications in the embodiment of the present invention corresponds to the part of the data product automated management method based on data specifications in the embodiment of the present invention. The description of the part of the data product automated management device based on data specifications specifically refers to the part of the data product automated management method based on data specifications, which will not be repeated here.

[0117] Figure 10 The structure block diagram of an electronic device suitable for implementing the data product automatic management method based on data specifications according to an embodiment of the present invention is schematically shown. Figure 10 The electronic device shown is only an example and should not limit the functions and scope of use of the embodiments of the present invention.

[0118] like Figure 10As shown, an electronic device 1000 according to an embodiment of the present invention includes a processor 1001, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 1002 or a program loaded from a storage unit 1008 into a random access memory (RAM) 1003. The processor 1001 may include, for example, a general-purpose microprocessor (e.g., a CPU), an instruction set processor and / or a related chipset and / or a special-purpose microprocessor (e.g., an application-specific integrated circuit (ASIC)), etc. The processor 1001 may also include onboard memory for caching purposes. The processor 1001 may include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of the present invention.

[0119] The storage unit 1008 stores various programs and data required for the operation of the electronic device 1000. The processor 1001, ROM 1002, and storage unit 1008 are interconnected via a bus 1004. The processor 1001 executes the programs stored in the ROM 1002 and / or storage unit 1008 to perform the various operations of the method flow according to the embodiment of the present invention. It should be noted that the programs may also be stored in one or more memories other than the ROM 1002 and storage unit 1008. The processor 1001 may also execute the programs stored in the one or more memories to perform the various operations of the method flow according to the embodiment of the present invention.

[0120] According to an embodiment of the present invention, electronic device 1000 may further include an input / output (I / O) interface 1005, which is also connected to bus 1004. Electronic device 1000 may also include one or more of the following components connected to I / O interface 1005: an input section 1006 including a keyboard, mouse, etc.; an output section 1007 including devices such as a cathode ray tube (CRT), liquid crystal display (LCD), and speakers; a storage section 1008 including a hard disk; and a communication section 1009 including a network interface card such as a LAN card or modem. Communication section 1009 performs communication processing via a network such as the Internet. A drive 1010 is also connected to I / O interface 1005 as needed. Removable media 1011, such as a magnetic disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed in drive 1010 as needed, so that computer programs read from the removable media can be installed into storage section 1008 as needed.

[0121] According to an embodiment of the present invention, the method flow according to an embodiment of the present invention can be implemented as a computer software program. For example, an embodiment of the present invention includes a computer program product, which includes a computer program carried on a computer-readable storage medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 1009, and / or installed from the removable medium 1011. When the computer program is executed by the processor 1001, the above-mentioned functions defined in the system of the embodiment of the present invention are performed. According to an embodiment of the present invention, the system, device, apparatus, module, unit, etc. described above can be implemented by a computer program module.

[0122] The present invention also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiments, or may exist independently and not incorporated into the device / apparatus / system. The computer-readable storage medium carries one or more programs, which, when executed, implement the method according to the embodiments of the present invention.

[0123] According to embodiments of the present invention, a computer-readable storage medium may be a non-volatile computer-readable storage medium. Examples include, but are not limited to, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present invention, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0124] For example, according to an embodiment of the present invention, the computer-readable storage medium may include the ROM 1002 and / or the storage portion 1008 described above and / or one or more memories other than the ROM 1002 and the storage portion 1008 .

[0125] An embodiment of the present invention also includes a computer program product, which includes a computer program, which contains program code for executing the method provided by the embodiment of the present invention. When the computer program product runs on an electronic device, the program code is used to enable the electronic device to implement the method provided by the embodiment of the present invention.

[0126] When the computer program is executed by the processor 1001, the above functions defined in the system / device of the embodiment of the present invention are performed. According to the embodiment of the present invention, the above-described systems, devices, modules, units, etc. can be implemented by computer program modules.

[0127] In one embodiment, the computer program may be stored on a tangible storage medium such as an optical storage device or a magnetic storage device. In another embodiment, the computer program may be transmitted and distributed in the form of a signal on a network medium, downloaded and installed via the communication portion 1009, and / or installed from the removable medium 1011. The program code contained in the computer program may be transmitted using any appropriate network medium, including but not limited to wireless, wired, or any suitable combination thereof.

[0128] According to an embodiment of the present invention, the program code for executing the computer program provided by the embodiment of the present invention can be written in any combination of one or more programming languages. Specifically, these computer programs can be implemented using high-level procedural and / or object-oriented programming languages, and / or assembly / machine languages. Programming languages ​​include, but are not limited to, languages ​​such as Java, C++, Python, "C" or similar programming languages. The program code can be executed entirely on the user computing device, partially on the user device, partially on a remote computing device, or entirely on a remote computing device or server. In the case of a remote computing device, the remote computing device can be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computing device (for example, using an Internet service provider to connect via the Internet).

[0129] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architectures, functions, and operations of the systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each box in the flowchart or block diagram may represent a module, program segment, or portion of code, which contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the boxes may occur in an order different from that marked in the accompanying drawings. For example, two boxes shown in succession may actually be executed substantially in parallel, or they may sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flowchart, as well as the combination of boxes in the block diagram or flowchart, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or may be implemented using a combination of dedicated hardware and computer instructions. It will be understood by those skilled in the art that the features described in the various embodiments of the present invention may be combined and / or coupled in various ways, even if such combinations or couplings are not explicitly described in the present invention. In particular, the features described in the various embodiments of the present invention may be combined and / or coupled in various ways without departing from the spirit and teachings of the present invention. All such combinations and / or combinations fall within the scope of the present invention.

[0130] The above describes embodiments of the present invention. However, these embodiments are for illustrative purposes only and are not intended to limit the scope of the present invention. Although each embodiment has been described separately above, this does not mean that the measures in each embodiment cannot be advantageously used in combination. Without departing from the scope of the present invention, those skilled in the art may make various substitutions and modifications, which should all fall within the scope of the present invention.

Claims

1. A data product automation management method based on data specifications, applied to a data management system, characterized in that: The method comprises: In response to receiving a data product physical model uploaded from an external source, parsing the data product physical model to obtain description information of the data product, wherein the data product physical model represents specification features of the data product; Creating a library table required for data storage management based on the description information, wherein the library table is configured to store data products in a structured manner according to the specifications of the description information; According to the library table, creating a data storage program corresponding to the library table; In response to a search request initiated by a user, generating a data query program corresponding to the search condition according to the search condition; According to the data storage program and the data query program, automatic management of data products is achieved.

2. The method according to claim 1, characterized in that Before creating the library table required for data storage management according to the description information, the method further includes: The description information is stored in the data product specification parsing result table.

3. The method according to claim 2, characterized in that The storing of the description information into the data product specification parsing result table includes: Storing the product information in the description information in the product information table; The product attribute information in the description information is stored in the attribute information table.

4. The method according to claim 3, characterized in that The step of creating the database table required for data storage management according to the description information includes: Determine whether a corresponding product database table exists according to the product information; In response to the corresponding product database table not existing, generating a first structured query language for creating a database table according to the product information and the product attribute information; In response to the existence of the corresponding product database table, generating a second structured query language for updating the database table according to the product information and the product attribute information; Create a library table required for data storage management according to the first structured query language or the second structured query language.

5. The method according to claim 3, characterized in that The step of creating a data storage program corresponding to the library table according to the library table includes: Generate an import request interface and an import result feedback interface based on the product information and the IP and port in the data product physical model; Generate a data verification program based on the product attribute information and attribute verification rules; A data entry program is generated based on the product attribute information and a structured query language template library.

6. The method according to claim 3, characterized in that In response to a user initiating a search request, generating a data query program corresponding to the search condition according to the search condition includes: Generate interface rendering logic based on the product attribute information, wherein different attribute types render different logics; In response to a search request initiated by a user through an operation interface, a data query program is generated according to the search conditions.

7. The method according to claim 1, characterized in that The method further comprises: Obtaining the test data product uploaded together with the data product physical model; According to the test product information of the test data product, obtain the corresponding test product attribute information, test data storage program and test data query program; Testing the data management system using the test product information, the test product attribute information, the test data storage program, and the test data query program; In response to the test passing, the data management system is updated and automatically put into operation.

8. A data product automation management device based on data specifications, characterized in that: The device comprises: a parsing module, configured to, in response to receiving a data product physical model uploaded from an external source, parse the data product physical model to obtain description information of the data product, wherein the data product physical model represents specification features of the data product; A first creation module is configured to create a library table required for data storage management based on the description information, wherein the library table is configured to store data products in a structured manner according to the specifications of the description information; A second creation module is used to create a data storage program corresponding to the library table according to the library table; A generating module, configured to generate a data query program corresponding to a search condition in response to a search request initiated by a user; The management module is used to realize the automatic management of data products according to the data storage program and the data query program.

9. An electronic device, characterized in that: include: one or more processors; a storage device for storing one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors are enabled to execute the method according to any one of claims 1 to 7.

10. A computer-readable storage medium having executable instructions stored thereon, characterized in that: When the instruction is executed by a processor, the processor executes the method according to any one of claims 1 to 7.

Images