Data product design system and method based on artificial intelligence and storage medium

Abstract

The invention discloses a data product design system and method based on artificial intelligence, and a storage medium, which are applied to the technical fields of artificial intelligence, big data and data product development, improve the design level of data products, accelerate the iteration process of the data products, and facilitate the improvement of the efficiency of the data products entering the market for transaction and circulation. Comprising a metadata rule management module used for storing and managing metadata rules of data products; the metadata design management module is used for extracting a plurality of demand characteristics from the obtained product demands through a preset artificial intelligence model on the basis of following a data product metadata rule, and defining a data product metadata design specification of each level; the data product design module is used for receiving and cooperatively processing multiple types of data product designs to obtain a data product design file; the data product development module is used for carrying out validity analysis on the data product design file by utilizing the data product configuration list, the data product metadata rule and the data product metadata design specification and combining a preset artificial intelligence model; and the data product test module is used for generating a data product specification file by using the data product design file when the data product design file passes the validity analysis, and releasing the data product specification file when the verification is passed.

Description

Technical Field

[0001] The present invention relates to the technical fields of artificial intelligence, big data and data product development, and in particular to a data product design system, method and storage medium based on artificial intelligence. Background Art

[0002] The data factor market refers to a place where data products or services are exchanged and the demands of all parties are realized through circulation. Data, as a new production factor born in the context of the information and intelligent technology revolution, is an indispensable part of economic development, just like traditional production factors such as land, labor, and capital. The cultivation and development of the data factor market is of great significance to promoting the high-quality development of the digital economy. The data factor market has a series of specific requirements for the design of data products, which are aimed at ensuring that data products can meet market demand and achieve compliance, efficient circulation and use of data.

[0003] The data factor market emphasizes high-quality data supply, so the design of data products needs to ensure the accuracy, completeness, timeliness and interpretability of data. This requires that when designing data products, data should be sourced from reliable and authoritative channels, inaccurate or misleading data should be avoided, data should be cleaned and integrated, duplicate, erroneous and invalid data should be removed, data quality and availability should be improved, and a data update and maintenance mechanism should be established to ensure that data can reflect market changes and business needs in a timely manner. The data factor market requires that data can circulate efficiently, which requires that standardized data formats and protocols be used when designing data products, data interoperability and compatibility should be improved, efficient and stable data interfaces should be designed, and data can be smoothly circulated and exchanged. The data factor market emphasizes the high-value application of data, and requires that data product design can tap and release the potential value of data. This requires that when designing data products, the potential value of data should be tapped through data analysis, mining and modeling, to provide support for business decisions, data should be applied to multiple fields such as production, life, and governance, data application scenarios and scope should be expanded, intuitive and easy-to-use user interfaces and interaction methods should be designed, and user experience of data products should be enhanced. At present, with the promotion and application of artificial intelligence technology, artificial intelligence technology has brought unprecedented innovation and optimization to data products with its powerful data processing, analysis and generation capabilities, and has increasingly broad application prospects in data product design.

[0004] At present, in the data product design stage, the project team needs to invest a lot of human resources to compile the metadata and content of the data product, and design and develop the data product in accordance with the data factor market requirements such as format standardization and content correctness. However, in the data product design process, there is a lack of intelligent tools or business systems that can support the project team to efficiently carry out the design of data products. Therefore, there is an urgent need for a data product design system and method based on artificial intelligence to solve the above problems. Summary of the invention

[0005] The purpose of the present invention is to solve the problems of the prior art and proposes a data product design system and method based on artificial intelligence. The present invention also discloses a computer-readable storage medium device that can effectively connect data product design, product verification, product generation and product use processes, improve the design level of data products, speed up the data product iteration process, and help improve the efficiency of data products entering the market for trading and circulation.

[0006] A data product design system based on artificial intelligence, comprising:

[0007] Metadata rule management module, used to store and manage data product metadata rules;

[0008] A metadata design management module is used to extract a number of demand features from the acquired product requirements through a preset artificial intelligence model on the basis of following the data product metadata rules, and define the data product metadata design specifications at each level;

[0009] The data product design module is used to receive and collaboratively process multiple types of data product designs to obtain data product design files, input the data product requirements into the preset artificial intelligence model to propose product keywords, and then perform multi-dimensional mapping of the product keywords and the preset standard feature data set to obtain a file format that conforms to the data product design, thereby improving the efficiency of subsequent data matching. The data product design is determined based on the data product metadata design specifications and data product requirements at each level, including data products and input and output data flows; it is also used to receive and collaboratively process multiple types of data product design changes to obtain updated data product design files.

[0010] The data product development module is used to establish a data product configuration list based on the data products and input and output data flows in the data product design; use the data product configuration list, the data product metadata rules and the data product metadata design specifications, and combine with a preset artificial intelligence model to perform validity analysis on the data product design file; and is also used to perform data lineage and business impact analysis based on the data product configuration list when the data product design file fails the validity analysis, the data product design file fails the verification, or when a user's product change request is received, to locate and output the data product design and change content that need to be changed; based on the data product design change, update the data product configuration list, and use the updated data product configuration list, the data product metadata rules and the data product metadata design specifications, and combine with a preset artificial intelligence model to perform validity analysis on the updated data product design file; the data product development module is specifically used to establish a data product lineage relationship graph database based on the data product design, with data products at each level as nodes and input and output data flows as edges. The data product development module is specifically used to verify the content of the data product design file in XML format and check the data product compliance through the XSD method based on the data product metadata rules and the data product metadata design specifications; based on the input-output relationship in the data product configuration list, verify the source Origin element in the data product design file in XML format.

[0011] The data product testing module is used to generate a data product specification file using the data product design file when the data product design file passes the validity analysis. It is also used to verify the data product metadata format standardization and content correctness of the generated data product specification file using the data product design file in combination with a preset artificial intelligence model, and publish the data product specification file when the verification passes. The data product specification file and the data product design file have different formats. The format of the data product design is the extensible markup language XML format; the data product metadata design specification is defined in the form of a document structure description XSD file.

[0012] A data product design method based on artificial intelligence, comprising:

[0013] Receive and collaboratively process multiple types of data product designs, and use a preset artificial intelligence model to obtain a data product design file, wherein the data product design is determined based on the data product metadata design specifications and data product requirements at each level, including data products and input and output data flows, and the data product metadata design specifications are defined based on the data product metadata rules;

[0014] Establishing a data product configuration list based on the data products and input and output data flows in the data product design;

[0015] The data product configuration list, the data product metadata rules and the data product metadata design specifications are used in combination with a preset artificial intelligence model to perform validity analysis on the data product design file; when the data product design file passes the validity analysis, the data product design file is used to generate a data product specification file, and the data product specification file and the data product design file have different formats; the data product design file is used to verify the data product metadata format standardization and content correctness of the data product specification file, and when the verification passes, the data product specification file is released.

[0016] As an improvement of the above method, the data product design method based on artificial intelligence further includes:

[0017] When the data product design file fails the validity analysis, fails the verification, or receives a product change request from the user, perform data lineage and business impact analysis based on the data product configuration list, locate and output the data product design and change content that need to be changed; collaboratively process multiple types of data product design changes to obtain updated data product design files; update the data product configuration list based on the data product design changes; use the updated data product configuration list, the data product metadata rules, and the data product metadata design specifications, and combine with the preset artificial intelligence model to perform validity analysis on the updated data product design file.

[0018] A storage medium having instructions stored thereon, which, when executed on a computer, execute the above-mentioned artificial intelligence-based data product design method. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 This is a schematic diagram of the structure of a data product design system based on artificial intelligence in Example 1 of the present invention;

[0020] Figure 2 This is a flow chart of a data product design method based on artificial intelligence according to Embodiment 2 of the present invention;

[0021] Figure 3 This is a schematic diagram of the structure of a logistics data product design system based on artificial intelligence according to Embodiment 3 of the present invention;

[0022] Figure 4 This is a flow chart of a logistics data product design method based on artificial intelligence according to Embodiment 3 of the present invention; DETAILED DESCRIPTION

[0023] The technical solution of the present invention is described in detail below with reference to the accompanying drawings and embodiments.

[0024] Example 1

[0025] like Figure 1 As shown, it is a schematic diagram of the structure of a data product design system based on artificial intelligence according to an embodiment of the present invention, comprising: a metadata rule management module 101, a metadata design management module 102, a data product design module 103, a data product development module 104 and a data product testing module 105;

[0026] The metadata rule management module 101 is used to store and manage data product metadata rules;

[0027] The metadata design management module 102 is used to extract a number of demand features from the obtained product requirements through a preset artificial intelligence model on the basis of following the data product metadata rules, and define the data product metadata design specifications at each level;

[0028] The data product design module 103 is used to receive and collaboratively process multiple types of data product designs to obtain data product design files, input the data product requirements into a preset artificial intelligence model to propose product keywords, and then perform multi-dimensional mapping of the product keywords and the preset standard feature data set to obtain a file format that conforms to the data product design, thereby improving the efficiency of subsequent data matching. The data product design is determined based on the data product metadata design specifications and data product requirements at each level, including data products and input and output data flows; the data product design module 103 is also used to receive and collaboratively process multiple types of data product design changes to obtain updated data product design files.

[0029] The data product development module 104 is used to establish a data product configuration list based on the data products and input and output data flows in the data product design; use the data product configuration list, the data product metadata rules and the data product metadata design specifications, and combine with the preset artificial intelligence model to perform validity analysis on the data product design file; in the process of using the data product, it is necessary to continuously adjust the format and content of the data product according to the feedback from use to meet the data product requirements in different data analysis and usage scenarios. In order to solve the product change requirements, preferably, the data product development module 104 is also used to perform data lineage and business impact analysis based on the data product configuration list when the data product design file fails the validity analysis, the data product design file fails the verification, or when the user's product change requirements are received, locate and output the data product design and change content that need to be changed; based on the updated data product design file, update the data product configuration list, use the updated data product configuration list, the data product metadata rules and the data product metadata design specifications, and combine with the preset artificial intelligence model to perform validity analysis on the updated data product design file;

[0030] The data product testing module 105 is used to generate a data product specification file using the data product design file when the data product design file passes the validity analysis. The data product testing module 105 is also used to verify the data product metadata format standardization and content correctness of the data product specification file using the data product design file, and release the data product specification file when the verification passes.

[0031] Of course, in actual use, the present invention can define what kind of data product the data product is based on actual needs, and define various levels and multiple types, and the present invention is not limited here.

[0032] The blood relationship database can be a graph database or a database in other organizational forms, as long as it can represent data products at each level and their blood relationship.

[0033] In an embodiment of the present invention, a graph database is used. In this case, the data product development module 104 is specifically used to establish a data product blood relationship graph database based on the data product design, with data products at each level as nodes and input and output data flows as edges.

[0034] Preferably, the format of the data product design is the Extensible Markup Language (XML) format; the data product metadata design specification is defined in the form of a document structure description (XSD) file; the data product development module 104 is specifically used to verify the content of the data product design file in XML format in XSD mode based on the data product metadata rules and the data product metadata design specification, and to perform a data product compliance check; based on the input-output relationship in the data product configuration list, the source Origin element in the data product design file in XML format is verified.

[0035] In the scheme of the embodiment of the present invention, based on the metadata rules, the data product design specifications in XSD format and the data product design in XML are defined to achieve rapid iteration of the data product process. The data product model is established based on the graph database method, which can efficiently carry out data product design lineage analysis and product design change business impact analysis. Based on the data product design, data product specifications are produced to support the verification of the data product design, and the final product design can be output to the data operation system as input, which can effectively connect the various links in the data product design, verification, production and use process.

[0036] Example 2

[0037] Based on the same inventive concept as that of Embodiment 1, Embodiment 2 of the present invention provides a data product design method based on artificial intelligence, and its flow chart is as follows: Figure 2 As shown, including:

[0038] Step S201: Receive and collaboratively process multiple types of data product designs to obtain data product design files, input the data product requirements into a preset artificial intelligence model to propose product keywords, and then perform multi-dimensional mapping of the product keywords and preset standard feature data sets to obtain a file format that conforms to the data product design, thereby improving the efficiency of subsequent data matching.

[0039] Among them, the data product design is determined based on the data product metadata design specifications and data product requirements at each level, including data products and input and output data flows. The data product metadata design specifications are defined on the basis of following the data product metadata rules.

[0040] In this step S201, the user's data product design changes may also be received. At this time, multiple types of data product design changes are collaboratively processed to obtain updated data product design files.

[0041] Step S202: Create a data product configuration list based on the data products and input and output data flows in the data product design.

[0042] In this step S202, if the data product design is to be updated, the data product configuration list is updated based on the data product and input and output data flows in the data product design update.

[0043] Step S203: Utilizing the data product configuration list, the data product metadata rules and the data product metadata design specifications, and combining with a preset artificial intelligence model, a validity analysis is performed on the data product design file;

[0044] If the data product configuration list is updated in step S202, then in step S203, the updated data product configuration list, the data product metadata rules and the data product metadata design specifications can be used in combination with a preset artificial intelligence model to perform validity analysis on the updated data product design file.

[0045] Step S204, determine whether the validity analysis is passed, if passed, execute step S205; if not, execute step S210.

[0046] Step S205: Generate a data product specification file using the data product design file.

[0047] Preferably, the data product specification file and the data product design file are in different formats, and after step S205, the following steps are further included:

[0048] Step S206: Use the data product design file to verify the data product specification file for the standardization of the data product metadata format and the correctness of the content.

[0049] Step S207: Determine whether the verification is passed. If the verification is passed, execute step S208; if the verification is not passed, jump to step S210.

[0050] Step S208: Release the data product specification file.

[0051] Step S209: Generate a data product using the data product specification file, and verify whether the data product using the data product specification complies with the metadata design specifications. If not, execute step S210; if compliant, end, indicating that the generated data product specification file has withstood the verification of the real data product.

[0052] In this step S209, unlike the verification of an empty data product specification file, the data product itself that uses the data product specification file is verified for compliance with the metadata design specification, and the real data product is used to check whether the data product specification file meets the requirements.

[0053] Step S210: Perform data lineage and business impact analysis based on the data product configuration list, locate and output the data product design and change content that need to be changed, and jump to step S201.

[0054] In addition, based on the same inventive concept, an embodiment of the present invention also provides a storage medium, on which instructions are stored. When the instructions are run on a computer, the above-mentioned artificial intelligence-based data product design method is executed.

[0055] Example 3

[0056] The embodiment of the present invention takes a logistics data product as an example to illustrate the solution of the present invention in detail.

[0057] This embodiment 3 provides a logistics data product design system based on artificial intelligence, which is used to complete the logistics data product design and testing, and output data product specifications, including the following: Figure 3 The five modules shown are a logistics data product metadata rule management module 301 , a logistics data product metadata design management module 302 , a logistics data product design module 303 , a logistics data product development module 304 and a logistics data product testing module 305 .

[0058] The logistics data product metadata rule management module 301 is mainly responsible for storing and managing general metadata rules. It supports the automatic generation of data product metadata files in XML format by importing other domestic and foreign data products. It can be used as the input and reference of metadata required for data product design through unique ID blood relationship reference.

[0059] The logistics data product metadata design management module 302 supports the definition of metadata specifications in the form of XSD (XML schema Definition) files based on data product metadata rules.

[0060] The logistics data product design module 303 is mainly responsible for supporting the design of data products, defining the data product design in XML format based on the comprehensive needs of all parties, supporting the editing and confirmation of the data product design, and performing logical verification of the data product definition in XML format through the defined data product specification XSD file.

[0061] The logistics data product development module 304 is mainly responsible for designing data products based on XML format and establishing data product models, storing the relationship between data products based on a graph database, and conducting data product lineage analysis and product effectiveness analysis based on the data product lineage information in combination with preset artificial intelligence models, and supports business impact analysis of changes based on user demand changes as input.

[0062] The logistics data product testing module 305 supports the automatic generation of data product specifications in CDF, FITS, CSV and other formats based on data product design in XML format, and supports the verification of the data product metadata format standardization and content correctness of data products in CDF, FITS, CSV and other formats based on data product design files in XML format.

[0063] The above-mentioned artificial intelligence-based logistics data product design system is mainly oriented to the data product metadata rules in XSD format. It takes the metadata rules as the specification and combines the data product usage requirements to define the data product format in XML format, and establishes a data product model based on the XML format. It supports users to analyze the validity of data product design, data lineage and business impact of product design changes based on the data product model. After the data product design is completed, it supports the verification of the data product design and the output of product specifications. Finally, the output data product specifications are used as input to the data operation system.

[0064] This embodiment 3 also provides a logistics data product design method based on artificial intelligence, such as Figure 4 As shown, the following steps are included:

[0065] Step S401: Collect, import and manage logistics data product metadata rules.

[0066] In step S401, before starting the data product design, in addition to referring to the general data metadata rules, it is also necessary to use the preset artificial intelligence model to collect and import the data product metadata rules in the logistics industry, as well as the metadata information of other similar data products at home and abroad, in order to adapt to and meet the usage needs and usage habits of users in the industry. Example: Usually, data products need to include scenario information. By defining a standard XML schema, the basic scenario information design principles of each data product are agreed upon, including the scenario name (name), scenario unit (agency), goal (objective), and the knowledge field (discipline) corresponding to the scenario. At the same time, a limited scope of the knowledge field can be agreed upon. The following is part of meta.xsd.

[0067] <? xml version="1 .0" encoding="UTF-8"? >

[0068] <xs:schemaxmlns:xs="http: / / www .w3 .org / 2001 / XMLSchema"xmlns:scenarios="http: / / www .example .com / scenarios"targetNamespace="http: / / www.example .com / scenarios"elementFormDefault="qualified">

[0069] <xs:simpleType name="DisciplineType">

[0070] <xs:restriction base="xs:string">

[0071] <xs:enumeration value="Traffic" / >

[0072] <xs:enumeration value="Freight" / >

[0073]

[0074]

[0075] <xs:complexType name="ScenariosType">

[0076] <xs:sequence>

[0077] <xs:element name="Name"type="xs:string" / >

[0078] <xs:element name="Agency"type="xs:string" / >

[0079] <xs:element name="Objective"type="xs:string" / >

[0080] <xs:element name="Discipline"type="scenarios:DisciplineType" / >

[0081] <xs:element name="DesignUnit"type="xs:string" / >

[0082] < / xs:sequence>

[0083]

[0084]

[0085] Step S402: Define logistics data product design specifications.

[0086] In step S402, the specific technical method is to define an XSD file in a standard format. The data product design specification can usually include two levels. On the one hand, the data product metadata design specification at the scenario level is defined. The specification is applicable to all data product designs within the scope of the scenario. The main content defines unified scenario information (Scenarios), including: name, number, design unit (DesignUnit), time, application information (ApplicationInfo), etc.; on the other hand, the data product metadata design specification at each subject level is performed. The design specification at this level is applicable to all data products generated based on the subject. Usually, the data product of a subject will be divided into level 0, level 1, level 2 and level 3 or more data according to the data processing procedures and the organization method of the data product. According to the organization method, it will be divided into time segmentation and organization or event segmentation and organization, etc. The data product design specification at the subject level usually includes subject name (SubjectName), subject code, subject scope, subject content (SC) and data category (DataCategory) and other contents. Taking the LOGISTICS logistics scenario as an example, it is necessary to define the schema data product design specification of the LOGISTICS logistics scenario level. This specification needs to follow the metadata rules defined in step S401. In addition, the data product design specification of the subject level is defined. This specification needs to follow the design specification of the scenario level. The following is part of the LOGISTICS .xsd, which needs to reference the meta .xsd defined in step S401:

[0087] <? xml version="1 .0" encoding="UTF-8"? >

[0088] <xs:schemaxmlns:xs="http: / / www .w3.org / 2001 / XMLSchema"

[0089] xmlns:scenarios="http: / / www .example .com / scenarios"

[0090] xmlns:logistics="http: / / www .example .com / logistics"

[0091] xmlns:subject="http: / / www .example .com / subject"

[0092] elementFormDefault="qualified"

[0093] targetNamespace="http: / / www .example .com / scenarios">

[0094] <!--Introduce the namespace and components defined in meta.xsd-->

[0095] <xs:import namespace="http: / / www .example .com / scenarios"schemaLocation ="meta .xsd" / >

[0096] <!-- Define scene type -->

[0097] <xs:complexType name="ScenariosType">

[0098] <xs:complexcontent>

[0099] <xs:restriction base="scenarios:ScenariosType">

[0100] <xs:sequence>

[0101] <xs:element name="Name"type="xs:string"fixed="LOGISTICS" / >

[0102] <xs:element name="Agency"type="xs:string" / >

[0103] <xs:element name="Objective"type="xs:string" / >

[0104] <xs:elementname="Discipline"type="scenarios:DisciplineType"fixed="Freight" / >

[0105] <xs:element name="DesignUnit"type="xs:string"fixed="ZHONGJINYUNLIAN" / >

[0106] < / xs:sequence>

[0107]

[0108] < / xs:complexcontent>

[0109]

[0110] <!-- Define the type of product category -->

[0111] <xs:simpleType name="ProductCategoryType">

[0112] <xs:restriction base="xs:string">

[0113] <xs:enumeration value="Category0" / >

[0114] <xs:enumeration value="Category1" / >

[0115] <xs:enumeration value="Category2" / >

[0116] <xs:enumeration value="Category3" / >

[0117]

[0118]

[0119] <!-- Define the attribute elements of product categories -->

[0120] <xs:element name="ProductCategory"type="ProductCategoryType" / >

[0121] <!--Define the Subject element-->

[0122] <xs:element name="Subject"type="subject:SubjectType" / >

[0123] <!--Define the complex type of Subject-->

[0124] <xs:complexType name="SubjectType"xmlns="http: / / www .example .com / subject">

[0125] <xs:sequence>

[0126] <!-- Reference the product category in the logistics namespace -->

[0127] <xs:element name="ProductCategory"type="logistics:ProductCategoryType" / >

[0128] <!--define name-->

[0129] <xs:element name="Name"type="xs:string" / >

[0130] <!-- Define subject content (SC) -->

[0131] <xs:element name="PrincipalInvestigator"type="xs:string" / >

[0132] < / xs:sequence>

[0133]

[0134]

[0135] In step S402, a subject-level design specification is defined, such as GTW_subject.xsd. The subject name is GTW, and the subject's SC is defined as Developer. The design specification follows LOGISTICS.xsd and is only applicable to data products of all levels produced by the GTW subject, and is not applicable to data product specifications of other subjects.

[0136] <? xml version="1 .0" encoding="UTF-8"? >

[0137] <xs:schemaxmlns:xs="http: / / www .w3.org / 2001 / XMLSchema"

[0138] xmlns:logistics="http: / / www .example .com / scenarios"

[0139] targetNamespace="http: / / www .example .com / subject"

[0140] elementFormDefault="qualified">

[0141] <!-- Reference components in LOGISTICS.xsd -->

[0142] <xs:import namespace="http: / / www .example .com / scenarios"schemaLocation ="LOGISTICS.xsd" / >

[0143] <!--Define SubjectType complex type-->

[0144] <xs:complexType name="SubjectType">

[0145] <xs:sequence>

[0146] <xs:element name="ProductCategory"type="logistics:ProductCategoryType" / >

[0147] <xs:element name="Name"type="xs:string" / >

[0148] <xs:element name="PrincipalInvestigator"type="xs:string"fixed="Developer" / >

[0149] < / xs:sequence>

[0150]

[0151] <!-- Define GTW elements -->

[0152] <xs:element name="GTW"type="subject:SubjectType" / >

[0153]

[0154] Step S403: Carry out logistics data product design.

[0155] In step S403, the user refers to the product design specifications at the logistics scenario level and the subject level and the requirements of the data product, and uses the preset artificial intelligence model to design the data product in XML format. The design content covers the file name, file type, file naming rules and data product metadata information of the data product: attribute name, attribute type, data specification (length limit, threshold limit, enumeration type limit, etc.), attribute description, attribute default value, attribute information value source and attribute information calculation method. When designing the data product, it is also necessary to analyze the information lineage relationship with the logistics original data information library. Example: The following is a part of the data product design of Category0 level data (Category0_product.xml). In this data product design, LOGISTICS .XSD and GTW_subject.xsd are followed. For the convenience of explanation, only one data element REGISTER is added, and the size, description, type, unit and source of the element are explained, and a unique ID is defined for each element, where Origin is used to define the source of the element. In this example, the ID of REGISTER is GTW_Category0_618.

[0156] <? xml version="1 .0" encoding="UTF-8"? >

[0157] <Category0Product xmlns="http: / / www .example .com / Category0"

[0158] xmlns:xsi="http: / / www .w3.org / 2001 / XMLSchema-instance"

[0159] xsi:schemaLocation="http: / / www .example .com / scenariosLOGISTICS.xsd

[0160] http: / / www.example .com / subject GTW_subject.xsd">

[0161] <productid> 88888< / productid>

[0162] <creationdate> 2024-01-16T12:00:00Z< / creationdate>

[0163] <headerentry>

[0164] http: / / www.example .com / subject GTW_subject.xsd">

[0165] <headerentry>

[0166] <key> REGISTER< / key>

[0167] <value> 1< / value>

[0168] <id> GTW_Category1_618< / id>

[0169] <comment> Required value< / comment>

[0170] <type> Integer< / type>

[0171] <unit> ---< / unit>

[0172] <origin> GTW_Category0_618< / origin>

[0173] < / headerentry>

[0174] <headerentry>

[0175] <key> SUM< / key>

[0176] <value> 5< / value>

[0177] <id> GTW_Category1_326< / id>

[0178] <comment> File count for Category1 product< / comment>

[0179] <type> Integer< / type>

[0180] <unit> ---< / unit>

[0181] <origin> UVI_Category1_326< / origin>

[0182] < / headerentry>

[0183] <logistics:scenarios>

[0184] <name> LOGISTICS< / name>

[0185] <agency> Enterprise Technology Center< / agency>

[0186] <objective> Intelligent Freight Dispatch< / objective>

[0187] <discipline> Transport< / discipline>

[0188] <designunit> ZHONGJINYUNLIAN< / designunit>

[0189] < / logistics:scenarios>

[0190] <subject:subject>

[0191] <productcategory> Category1< / productcategory>

[0192] <name> GTW< / name>

[0193] <principalinvestigator> Developer< / principalinvestigator>

[0194] < / subject:subject>

[0195]

[0196] Step S404: Establish a logistics data product lineage relationship model.

[0197] In step S404, normally the processing of level 0 data products only relies on the original data, but the contents of level 1, 2, and 3 data products usually associate input data from many different sources for comprehensive processing. Therefore, a logistics data product lineage model based on graph database storage is constructed with data products at each level as nodes and input and output data flows as edges.

[0198] Step S405: Conduct data product design effectiveness analysis based on the logistics data product lineage model.

[0199] In step S405, the validity analysis is mainly divided into two aspects. On the one hand, the XML content can be verified by XSD to carry out the data product design standardization check. Based on the XSD method, it can automatically check whether the format in the data product design XML file complies with the specification, whether the content of the data product definition complies with the requirements of the logistics data metadata rules, etc.; on the other hand, it is verified by the Origin element defined in the data product design file XML. When the validity analysis passes, jump to step S406, and when the validity analysis fails, jump to step S408.

[0200] Step S406: Generate a logistics data product specification file based on the data product design file in XML format.

[0201] In step S406, the file name, file type and attribute name, attribute type and attribute default value in the product metadata defined in XML format are read, and a product specification file of type FITS, CDF or CSV is generated based on the data product design information and related data information.

[0202] Step S407: Perform logistics data product verification.

[0203] In step S407, based on the data product specification, the file name, file type in the data product specification file and the attribute name, attribute type, attribute value range, attribute description, attribute default value and other information in the product metadata are extracted and verified with the information in the data product design file in XML format to confirm that the generated data product specification can meet the requirements of the data product design in terms of attribute type, attribute value range, attribute description, attribute default value and other aspects. If the verification is passed, jump to step S409. If the verification is not passed, jump to step S408.

[0204] In step S407, in the link of data product verification, the data product design that has not passed the verification needs to be updated and iterated. In addition, the design process of the data product needs to be continuously iterated and updated multiple times according to the needs of the scenario and the needs of data analysis in order to form a reliable data product design version. Therefore, in the process of data product design changes, data lineage and business impact analysis is performed based on the logistics data product lineage model to quickly locate and output to the user the data product design that needs to be changed and the content of the change. Example: As defined in step S403, the Category-1 data product of the GTW theme is processed by the Category0 data product of the GTW theme and the Category1 data product of the UVI theme. When the design is updated, if the REGISTER data element of the Category0 data product of the GTW theme or the SUM data element of the Category1 of the UVI theme changes, the specific impact on the Category-1 data product of the GTW theme can be quickly located through the graph database.

[0205] Step S408: Conduct logistics data product relevance and business impact analysis, locate and output to the user the data product design that needs to be changed and the content of the change, and then jump to step S403.

[0206] Step S409: Output the data product specification file required by the logistics and transportation data operation system. The data product specification file can be used to support the production of logistics data products at all levels.

[0207] Step S410: After the external transportation data operation system produces the data product using the data product specification file, it verifies whether the data product complies with the metadata design specification. If not, it jumps to step S408. If it complies, it ends.

[0208] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit the present invention. Although the present invention is described in detail with reference to the embodiments, it should be understood by those skilled in the art that any modification or equivalent replacement of the technical solutions of the present invention does not depart from the spirit and scope of the technical solutions of the present invention and should be included in the scope of the claims of the present invention.< / headerentry>

Claims

1. A data product design system based on artificial intelligence, characterized in that: include: Metadata rule management module, used to store and manage data product metadata rules; The metadata design management module is used to extract a number of demand features from the obtained product requirements through a preset artificial intelligence model on the basis of following the data product metadata rules, and define the data product metadata design specifications at each level; data product design module; data product development module; data product testing module.

2. The data product design system based on artificial intelligence as claimed in claim 1, characterized in that: The data product design module is used to receive and collaboratively process multiple types of data product designs to obtain data product design files, input the data product requirements into a preset artificial intelligence model to propose product keywords, and then perform multi-dimensional mapping of the product keywords and the preset standard feature data set to obtain a file format that conforms to the data product design, thereby improving the efficiency of subsequent data matching. The data product design is determined based on the data product metadata design specifications and data product requirements at each level, including data products and input and output data flows; it is also used to receive and collaboratively process multiple types of data product design changes to obtain updated data product design files.

3. The data product design system based on artificial intelligence as claimed in claim 1, characterized in that: The data product development module is used to establish a data product configuration list based on the data product and input and output data flows in the data product design; use the data product configuration list, the data product metadata rules and the data product metadata design specifications, and combine with a preset artificial intelligence model to perform validity analysis on the data product design file; It is also used to perform data lineage and business impact analysis based on the data product configuration list, locate and output the data product design and change content that need to be changed when the data product design file fails the validity analysis, fails the verification, or receives the user's product change request; Based on the data product design changes, the data product configuration list is updated, and the updated data product configuration list, the data product metadata rules and the data product metadata design specifications are used in combination with the preset artificial intelligence model to analyze the validity of the updated data product design file; the data product development module is specifically used to establish a data product blood relationship graph database based on the data product design, with each level of data products as nodes and input and output data flows as edges. The data product development module is specifically used to verify the content of the data product design file in XML format and the data product normativeness through the XSD method based on the data product metadata rules and the data product metadata design specifications; based on the input and output relationship in the data product configuration list, the source Origin element in the data product design file in XML format is verified.

4. The data product design system based on artificial intelligence as claimed in claim 1, characterized in that: The data product testing module is used to generate a data product specification file using the data product design file when the data product design file passes the validity analysis. It is also used to verify the data product metadata format standardization and content correctness of the generated data product specification file using the data product design file in combination with a preset artificial intelligence model, and publish the data product specification file when the verification passes. The data product specification file and the data product design file have different formats. The format of the data product design is the extensible markup language XML format; the data product metadata design specification is defined in the form of a document structure description XSD file.

5. A data product design method based on artificial intelligence, characterized in that: include: Receive and collaboratively process multiple types of data product designs, and use a preset artificial intelligence model to obtain a data product design file, wherein the data product design is determined based on the data product metadata design specifications and data product requirements at each level, including data products and input and output data flows, and the data product metadata design specifications are defined on the basis of following the data product metadata rules; establish a data product configuration list based on the data products and input and output data flows in the data product design; use the data product configuration list, the data product metadata rules and the data product metadata design specifications, and combine with the preset artificial intelligence model to perform validity analysis on the data product design file; When the data product design file passes the validity analysis, the data product design file is used to generate a data product specification file, and the data product specification file and the data product design file have different formats; the data product design file is used to verify the data product metadata format standardization and content correctness of the data product specification file, and when the verification passes, the data product specification file is released.

6. The data product design method based on artificial intelligence as claimed in claim 5, characterized in that: When the data product design file fails the validity analysis, fails the verification, or receives the product change request from the user, perform data lineage and business impact analysis based on the data product configuration list, locate and output the data product design and change content that need to be changed; collaboratively process multiple types of data product design changes to obtain updated data product design files; Based on the data product design changes, the data product configuration list is updated; the updated data product configuration list, the data product metadata rules and the data product metadata design specifications are used, and the validity of the updated data product design files is analyzed in combination with a preset artificial intelligence model.

7. A storage medium storing instructions, which, when executed on a computer, executes the artificial intelligence-based data product design method as described in any one of claims 5 to 6.

Images