Internet of vehicles metadata management method, device and equipment and readable storage medium

Abstract

The application relates to a vehicle networking metadata management method and device, equipment and a readable storage medium, and relates to the technical field of metadata management. The application comprises the following steps: extracting public attribute fields of metadata in vehicle networking and importing the metadata into a metadata management system, extracting data source fields and importing the data source fields into the metadata management system; creating an SQL table creation statement based on the data source fields; creating a data table based on the SQL table creation statement; analyzing the SQL table creation statement to obtain field information of metadata to be managed on the data table, which comprises a first name field and a first description field; matching the first name field and the first description field with name fields and description fields in the metadata management system; when a second name field similar to the first name field and a second description field similar to the first description field are matched, updating the first name field and the first description field based on the second name field and the second description field, realizing the management of the metadata, and improving the quality of the metadata.

Description

Technical Field

[0001] The present application relates to the field of metadata management technology, and in particular to a method, apparatus, device and readable storage medium for managing metadata of an Internet of Vehicles. Background Art

[0002] Metadata is used to describe data within an enterprise, and metadata management is an effective means of organizing and inventorying data assets. Metadata not only represents information such as data type, name, and value, but also provides contextual information about the data, such as its domain, value range, relationships between data, business rules, and even its origin. In data analysis, metadata helps data warehouse developers and senior data analysts easily find the data they are interested in. Metadata can provide a detailed diagram of data structure definitions and elements, as well as data context and relationships, making application development more efficient and providing data referenceability, citation, lineage analysis, impact analysis, and change analysis. Therefore, metadata management is essential.

[0003] However, in the absence of strong constraints within the company, different departments and developers within the company may have different definitions and names for attributes with the same meaning. This is especially true in the Internet of Vehicles, where the amount of data is much larger and the above phenomenon is more likely to occur, which can easily cause ambiguity and lead to poor metadata quality, which is extremely unfriendly to metadata development and exploration. Summary of the Invention

[0004] The present application provides a method, apparatus, device and readable storage medium for managing metadata of an Internet of Vehicles (IoV) to solve the problem of poor metadata quality caused by different definitions and naming of attributes with the same meaning in metadata in related technologies.

[0005] In a first aspect, a method for managing metadata of an Internet of Vehicles is provided, comprising the following steps:

[0006] Extracting common attribute fields of metadata in the Internet of Vehicles and importing the common attribute fields into a metadata management system, wherein the common attribute fields include a name field and a description field;

[0007] Extracting data source fields and importing the data source fields into the metadata management system, wherein the data source fields include one or more of a database type field, a database name field, a database instance field, a database address field, a database port field, and a database description field;

[0008] Create an SQL table creation statement based on the data source fields;

[0009] Creating a data table based on the SQL table creation statement, and parsing the SQL table creation statement to obtain field information of metadata to be managed on the data table, the field information including a first name field and a first description field;

[0010] matching the first name field and the first description field with the name field and the description field in a metadata management system;

[0011] When a second name field similar to the first name field and a second description field similar to the first description field are matched in the metadata management system, the first name field is updated based on the second name field, and the first description field is updated based on the second description field to achieve metadata management.

[0012] In some embodiments, after the step of matching the first name field and the first description field with the name field and the description field in the metadata management system, the method further includes:

[0013] When a second name field similar to the first name field is not matched in the metadata management system or a second description field similar to the first description field is not matched in the metadata management system, the first name field and the first description field are imported into the metadata management system.

[0014] In some embodiments, after the step of updating the first description field based on the second description field, the method further includes:

[0015] Conduct compliance checks on dataX data processing scripts;

[0016] When the dataX data processing script meets the requirements, execute the dataX data processing script and obtain the configuration information of the dataX data processing script;

[0017] Parsing the configuration information to obtain first basic information of the output table and second basic information of the input table, wherein the first basic information includes a data source field of the output table, a table name of the output table, and field information of metadata on the output table; and the second basic information includes a data source field of the input table, a table name of the input table, and field information of metadata on the input table;

[0018] When there exists in the metadata management system a first table entity corresponding to the data source field of the output table and the table name of the output table, and a second table entity corresponding to the data source field of the input table and the table name of the input table, a table-level lineage relationship between the output table and the input table is generated based on the table name of the output table and the table name of the input table;

[0019] The table-level kinship relationship is imported into a metadata management system.

[0020] In some embodiments, after the step of: when the metadata management system has a first table entity corresponding to the data source field of the output table and the table name of the output table, and a second table entity corresponding to the data source field of the input table and the table name of the input table, the method further includes:

[0021] Querying the metadata management system for first identifiers corresponding to column name fields of each column in the output table according to the first basic information;

[0022] Querying the metadata management system for second identifiers corresponding to column name fields of each column in the input table according to the second basic information;

[0023] A column-level lineage relationship is generated based on the first identifier and the second identifier, and the column-level lineage relationship is imported into a metadata management system.

[0024] In some embodiments, before the step of: when the metadata management system has a first table entity corresponding to the data source field of the output table and the table name of the output table, and a second table entity corresponding to the data source field of the input table and the table name of the input table, the method further includes:

[0025] Match the metadata field information on the input table with the public attribute fields in the metadata management system;

[0026] When no common attribute field similar to the field information of the metadata on the input table is matched in the metadata management system, the field information of the metadata on the input table is imported into the metadata management system.

[0027] In some embodiments, before the step of creating a data table based on the SQL table creation statement, the method further includes:

[0028] Determine the accuracy and executability of the SQL table creation statement;

[0029] If the accuracy and executability of the SQL table creation statement meet the requirements, then executing the step of creating a data table based on the SQL table creation statement;

[0030] If the accuracy and executability of the SQL table creation statement do not meet the requirements, the SQL table creation statement is updated and modified to obtain an updated SQL table creation statement, and the step of creating a data table based on the SQL table creation statement is executed based on the updated SQL table creation statement.

[0031] In some embodiments, matching a second name field similar to the first name field in the metadata management system includes:

[0032] Calculating string edit distances between the first name field and each name field in the metadata management system;

[0033] Normalize each string edit distance separately to obtain multiple similarities;

[0034] When one of the similarities is greater than or equal to the similarity threshold, it is determined that a second name field similar to the first name field exists in the common attribute field in the metadata management system.

[0035] In a second aspect, a vehicle network metadata management device is provided, comprising:

[0036] an extraction unit configured to extract common attribute fields from metadata in the Internet of Vehicles and import the common attribute fields into a metadata management system, wherein the common attribute fields include a name field and a description field; extract data source fields and import the data source fields into the metadata management system, wherein the data source fields include one or more of a database type field, a database name field, a database instance field, a database address field, a database port field, and a database description field;

[0037] a processing unit configured to create an SQL table creation statement based on the data source field; create a data table based on the SQL table creation statement, and parse the SQL table creation statement to obtain field information of metadata to be managed on the data table, the field information including a first name field and a first description field;

[0038] a matching unit, configured to match the first name field and the first description field with a name field and a description field in a metadata management system;

[0039] A management unit, which is used to update the first name field based on the second name field and update the first description field based on the second description field when a second name field similar to the first name field and a second description field similar to the first description field are matched in the metadata management system, so as to realize the management of metadata.

[0040] In a third aspect, a vehicle network metadata management device is provided, comprising: a memory and a processor, wherein the memory stores at least one instruction, and the at least one instruction is loaded and executed by the processor to implement the aforementioned vehicle network metadata management method.

[0041] In a fourth aspect, a computer-readable storage medium is provided, wherein the computer storage medium stores a computer program, and when the computer program is executed by a processor, the aforementioned vehicle network metadata management method is implemented.

[0042] The present application provides a method, apparatus, device and readable storage medium for managing metadata of an Internet of Vehicles, comprising extracting common attribute fields of metadata in the Internet of Vehicles and importing the common attribute fields into a metadata management system, wherein the common attribute fields include a name field and a description field; extracting a data source field and importing the data source field into the metadata management system, wherein the data source field includes one or more of a database type field, a database name field, a database instance field, a database address field, a database port field and a database description field; creating an SQL table creation statement based on the data source field; creating a data table based on the SQL table creation statement, and parsing the SQL table creation statement to obtain field information of metadata to be managed on the data table, wherein the field information includes a first name field and a first description field; matching the first name field and the first description field with the name field and the description field in the metadata management system; and when a second name field similar to the first name field and a second description field similar to the first description field are matched in the metadata management system, updating the first name field based on the second name field and updating the first description field based on the second description field to achieve metadata management. Through this application, the relevant fields of the metadata to be managed on the data table created according to the SQL table creation statement are checked and updated based on the common attribute fields, so as to achieve the standardization of the metadata fields and improve the quality of the metadata. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0044] Figure 1 A flowchart of a method for managing metadata for an Internet of Vehicles (IoV) according to an embodiment of the present application is provided;

[0045] Figure 2 A schematic diagram of the structure of a vehicle network metadata management device provided in an embodiment of the present application;

[0046] Figure 3 A schematic diagram of the structure of a vehicle network metadata management device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0047] To make the purpose, technical solutions, and advantages of the embodiments of this application more clear, the technical solutions in the embodiments of this application will be clearly and completely described below in conjunction with the drawings in the embodiments of this application. Obviously, the described embodiments are part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0048] Embodiments of the present application provide a method, apparatus, device, and readable storage medium for managing metadata for an Internet of Vehicles (IoV), which can solve the problem of poor metadata quality in related technologies caused by different definitions and naming of attributes with the same meaning in metadata.

[0049] Figure 1 The embodiment of the present application provides a method for managing metadata of an Internet of Vehicles, comprising the following steps:

[0050] Step S10: extracting common attribute fields of metadata in the Internet of Vehicles, and importing the common attribute fields into a metadata management system, wherein the common attribute fields include a name field and a description field;

[0051] For example, metadata management has many benefits, but data governance is a gradual process, rather than always waiting until data is generated to adapt to existing standards. Therefore, once you have your own data standards, you need a set of effective processes and methods to allow subsequent projects to follow the standards in order to achieve the purpose of regularizing data.

[0052] Since the source of metadata in the Internet of Vehicles is often protocol documents based on national or industry standards, or enterprise-level protocols defined based on these standard protocols, there are often many common attributes from protocol documents to data parsing. Therefore, in this embodiment, the fields of common attributes are extracted, and then the data structure can be customized based on the common attributes and generated into a SQL (Structured Query Language) table structure and other data structures. Only after the customized data structure passes the inspection will it be entered into the final metadata management system.

[0053] Among them, this embodiment can extract common attribute fields in vertical business areas according to business types and enter them into the metadata management system. Common attribute fields include name field, data type field, description field, remark field, owner field, category field, and data interval field, etc. Common attribute fields can be classified according to vehicle type. For example, common attribute fields of Internet of Vehicles positioning data include frame number field, chassis number field, SIM number field, license plate number field, vehicle model field, and enterprise field, etc.; common attribute fields of CAN signals include speed field, latitude and longitude field, elevation field, azimuth field, throttle opening field, brake pedal opening field, total mileage field, elevation field, acceleration field, headlight status field, door status field, window status field, air conditioning status field, and gear field, etc.; CAN signal fuel vehicle attribute fields include instantaneous fuel consumption field, total fuel consumption field, and fuel tank level field, etc.; The attribute fields of CAN signal for new energy vehicles include total current field, total voltage field, SOC remaining power field, battery status field, charging status field, motor speed field, battery status field and battery temperature field, etc.; the attribute fields of CAN signal for autonomous driving include driving mode field, x-axis speed field, y-axis speed field, z-axis speed field, x-axis acceleration field, y-axis acceleration field and z-axis acceleration field; the sensor fields include identification data field, identification target number field, traffic light color field, etc.; the public indicator fields include operating mileage field, operating time field, fuel consumption field, power consumption field and order number field, etc.

[0054] For example, take the vehicle frame number as an example: its name field is vin, the data type field is string, the description field is the vehicle frame number, the remarks field is the basic attributes of the vehicle, the owner field is public, and the category field is the basic attributes of the vehicle.

[0055] Step S20: extracting data source fields and importing the data source fields into the metadata management system, wherein the data source fields include one or more of a database type field, a database name field, a database instance field, a database address field, a database port field, and a database description field;

[0056] Step S30: creating an SQL table creation statement based on the data source fields;

[0057] For example, in this embodiment, data source fields are maintained in the metadata management system, where the data source fields include a database type field, a database name field, a database instance field, a database address field, a database port field, and a database description field. The applicant can select the desired fields from the database type field, database name field, database instance field, database address field, database port field, and database description field. After receiving the user-selected desired fields, this embodiment will create an SQL table creation statement based on the selected desired fields. It should be noted that the SQL table creation statement can also be automatically created by the applicant based on the selected desired fields, and then this embodiment directly receives the SQL table creation statement.

[0058] Furthermore, before the step of creating a data table based on the SQL table creation statement, the method further includes:

[0059] Determine the accuracy and executability of the SQL table creation statement;

[0060] If the accuracy and executability of the SQL table creation statement meet the requirements, then executing the step of creating a data table based on the SQL table creation statement;

[0061] If the accuracy and executability of the SQL table creation statement do not meet the requirements, the SQL table creation statement is updated and modified to obtain an updated SQL table creation statement, and the step of creating a data table based on the SQL table creation statement is executed based on the updated SQL table creation statement.

[0062] For example, in this embodiment, the accuracy and executable nature of the SQL table creation statement are automatically determined by parsing and trial running the SQL table creation statement. If the SQL table creation statement is accurate and executable, the SQL table creation statement is executed and a data table is obtained. If the SQL table creation statement is inaccurate or unexecutable, the reason for the inaccuracy or unexecutability is recorded and output, and at this time, the SQL table creation statement needs to be modified based on the reason for the inaccuracy or unexecutability and the SQL table creation statement needs to be resubmitted.

[0063] Step S40: creating a data table based on the SQL table creation statement, and parsing the SQL table creation statement to obtain field information of metadata to be managed on the data table, the field information including a first name field and a first description field;

[0064] For example, in this embodiment, when the SQL table creation statement is accurate and executable, the data table will be created by executing the SQL table creation statement, and the SQL table creation statement will be parsed using the druid tool to obtain the name field and description field of the metadata to be managed in the data table.

[0065] Step S50: matching the first name field and the first description field with the name field and the description field in the metadata management system;

[0066] Furthermore, after the step of matching the first name field and the first description field with the name field and the description field in the metadata management system, the method further includes:

[0067] When a second name field similar to the first name field is not matched in the metadata management system or a second description field similar to the first description field is not matched in the metadata management system, the first name field and the first description field are imported into the metadata management system.

[0068] Step S60: When a second name field similar to the first name field and a second description field similar to the first description field are matched in the metadata management system, the first name field is updated based on the second name field, and the first description field is updated based on the second description field to achieve metadata management.

[0069] Furthermore, matching a second name field similar to the first name field in the metadata management system includes:

[0070] Calculating string edit distances between the first name field and each name field in the metadata management system;

[0071] Normalize each string edit distance separately to obtain multiple similarities;

[0072] When one of the similarities is greater than or equal to the similarity threshold, it is determined that a second name field similar to the first name field exists in the common attribute field in the metadata management system.

[0073] For example, in this embodiment, string similarity matching is performed between the name field and description field of the metadata to be managed in the data table and the name field and description field in the public attribute field. Since the principles and methods of string similarity matching are the same, for the sake of simplicity, this embodiment uses the string similarity matching between the name field of the metadata to be managed in the data table and the name field in the public attribute field as an example: the string edit distance between the name field of the metadata to be managed in the data table and each name field in the public attribute field is calculated, and then the logarithm of each string edit distance is normalized to obtain the similarity corresponding to each string edit distance;

[0074] If one of the similarities is greater than or equal to the similarity threshold, it means that there is a name field in the public attribute field that is similar to the name field of the metadata to be managed in the data table. At this time, it can be prompted that it already exists and the corresponding field can be marked, and the name field and description field of the metadata to be managed in the data table are kept consistent with the name field and description field in the public attribute field; however, if all the similarities are less than the similarity threshold, it means that there is no name field in the public attribute that is similar to the name field of the metadata to be managed in the data table. At this time, the name field of the metadata to be managed in the data table is automatically added to the public attribute field in the metadata management system.

[0075] It can be seen that this embodiment checks, updates and modifies the relevant fields of the metadata to be managed on the data table created according to the SQL table creation statement based on the common attribute fields, realizes the standardization of metadata fields, improves the quality of metadata, and makes the maintenance of metadata simpler, more effective and convenient, thereby reducing the communication cost during the subsequent data development and use.

[0076] Furthermore, after the step of updating the first description field based on the second description field, the method further includes:

[0077] Conduct compliance checks on dataX data processing scripts;

[0078] When the dataX data processing script meets the requirements, execute the dataX data processing script and obtain the configuration information of the dataX data processing script;

[0079] Parsing the configuration information to obtain first basic information of the output table and second basic information of the input table, wherein the first basic information includes a data source field of the output table, a table name of the output table, and field information of metadata on the output table; and the second basic information includes a data source field of the input table, a table name of the input table, and field information of metadata on the input table;

[0080] When there exists in the metadata management system a first table entity corresponding to the data source field of the output table and the table name of the output table, and a second table entity corresponding to the data source field of the input table and the table name of the input table, a table-level lineage relationship between the output table and the input table is generated based on the table name of the output table and the table name of the input table;

[0081] The table-level kinship relationship is imported into a metadata management system.

[0082] Furthermore, after the step of: when the metadata management system has a first table entity corresponding to the data source field of the output table and the table name of the output table, and a second table entity corresponding to the data source field of the input table and the table name of the input table, the method further includes:

[0083] Querying the metadata management system for first identifiers corresponding to column name fields of each column in the output table according to the first basic information;

[0084] Querying the metadata management system for second identifiers corresponding to column name fields of each column in the input table according to the second basic information;

[0085] A column-level lineage relationship is generated based on the first identifier and the second identifier, and the column-level lineage relationship is imported into a metadata management system.

[0086] Furthermore, before the step of: when there are first table entities corresponding to the data source fields of the output table and the table name of the output table, and second table entities corresponding to the data source fields of the input table and the table name of the input table, the method further includes:

[0087] Match the metadata field information on the input table with the public attribute fields in the metadata management system;

[0088] When no common attribute field similar to the field information of the metadata on the input table is matched in the metadata management system, the field information of the metadata on the input table is imported into the metadata management system.

[0089] For example, the mainstream ETL tools currently used for data extraction, transformation, and loading include dataX, Sqoop, and Hive. Sqoop and Hive use Apache Atlas hooks to maintain lineage relationships, but they perform lineage update checks each time the script runs. DataX, on the other hand, lacks a robust data lineage management tool and must maintain data lineage through SQL parsing. This embodiment, however, avoids repeated lineage updates by performing a compliance check on the dataX data processing scripts before metadata maintenance. After the dataX data processing scripts pass the compliance check, the dataX configuration files are analyzed to establish lineage relationships for the corresponding data.

[0090] Specifically, after the applicant submits the dataX data processing script, this embodiment will detect whether the dataX data processing script complies with the specifications based on relevant specifications and conduct a trial run. After confirming that the dataX data processing script meets the requirements and is executable, the dataX data processing script will be automatically deployed and executed; at the same time, the configuration information of dataX will be obtained and parsed to obtain the data source field of the input table, the table name of the input table, and the fields of the metadata on the input table, as well as the data source field of the output table, the table name of the output table, and the fields of the metadata on the output table.

[0091] At this point, the metadata fields on the input table need to be matched with the public attribute fields in the metadata management system. If no public attribute fields similar to the metadata field information on the input table are found in the metadata management system, the metadata field information on the input table is imported into the metadata management system. Since the method and principle for matching the metadata fields on the input table with the public attribute fields in the metadata management system are similar to the method and principle for matching the first name field and the first description field with the name field and description field in the metadata management system, they will not be repeated here for the sake of brevity.

[0092] According to the data source fields and table names of the output table and input table obtained by parsing, the metadata management system is queried to determine whether there are table entities corresponding to the output table and input table respectively; if not, all fields on the output table and input table are read according to the data source fields and table names of the output table and input table and complete table metadata information is generated and imported into the metadata management system; if it exists, the GUID (globally unique identifier) ​​of the corresponding table entity is obtained, and a table-level lineage relationship entity between the output table and the input table is generated according to the table name corresponding to the GUID; and the GUID of each column name field is queried in the metadata management system according to the data source field of the input table, the table name of the input table and the field of metadata on the input table, as well as the data source field of the output table, the table name of the output table and the field of metadata on the output table, and the column-level lineage relationship entity is generated according to the GUID, and finally the table-level lineage relationship and the column-level lineage relationship are imported into the metadata management system. As can be seen from this, this embodiment performs metadata and lineage relationship maintenance after the dataX data processing script is standardized, so as to avoid the problem of repeatedly updating metadata each time when the hive hook or dataX plug-in is embedded.

[0093] See also Figure 2 As shown, the embodiment of the present application also provides a vehicle network metadata management device, including:

[0094] an extraction unit configured to extract common attribute fields from metadata in the Internet of Vehicles and import the common attribute fields into a metadata management system, wherein the common attribute fields include a name field and a description field; extract data source fields and import the data source fields into the metadata management system, wherein the data source fields include one or more of a database type field, a database name field, a database instance field, a database address field, a database port field, and a database description field;

[0095] a processing unit configured to create an SQL table creation statement based on the data source field; create a data table based on the SQL table creation statement, and parse the SQL table creation statement to obtain field information of metadata to be managed on the data table, the field information including a first name field and a first description field;

[0096] a matching unit, configured to match the first name field and the first description field with a name field and a description field in a metadata management system;

[0097] A management unit, which is used to update the first name field based on the second name field and update the first description field based on the second description field when a second name field similar to the first name field and a second description field similar to the first description field are matched in the metadata management system, so as to realize the management of metadata.

[0098] Furthermore, the management unit is further configured to:

[0099] When a second name field similar to the first name field is not matched in the metadata management system or a second description field similar to the first description field is not matched in the metadata management system, the first name field and the first description field are imported into the metadata management system.

[0100] Furthermore, the management unit is further configured to:

[0101] Conduct compliance checks on dataX data processing scripts;

[0102] When the dataX data processing script meets the requirements, execute the dataX data processing script and obtain the configuration information of the dataX data processing script;

[0103] Parsing the configuration information to obtain first basic information of the output table and second basic information of the input table, wherein the first basic information includes a data source field of the output table, a table name of the output table, and field information of metadata on the output table; and the second basic information includes a data source field of the input table, a table name of the input table, and field information of metadata on the input table;

[0104] When there exists in the metadata management system a first table entity corresponding to the data source field of the output table and the table name of the output table, and a second table entity corresponding to the data source field of the input table and the table name of the input table, a table-level lineage relationship between the output table and the input table is generated based on the table name of the output table and the table name of the input table;

[0105] The table-level kinship relationship is imported into a metadata management system.

[0106] Furthermore, the management unit is further configured to:

[0107] Querying the metadata management system for first identifiers corresponding to column name fields of each column in the output table according to the first basic information;

[0108] Querying the metadata management system for second identifiers corresponding to column name fields of each column in the input table according to the second basic information;

[0109] A column-level lineage relationship is generated based on the first identifier and the second identifier, and the column-level lineage relationship is imported into a metadata management system.

[0110] Furthermore, the matching unit is further used to match the field information of the metadata on the input table with the common attribute fields in the metadata management system;

[0111] The management unit is further configured to import the field information of the metadata on the input table into the metadata management system when no common attribute field similar to the field information of the metadata on the input table is matched in the metadata management system.

[0112] Furthermore, the processing unit is further configured to:

[0113] Determine the accuracy and executability of the SQL table creation statement;

[0114] If the accuracy and executability of the SQL table creation statement meet the requirements, then executing the step of creating a data table based on the SQL table creation statement;

[0115] If the accuracy and executability of the SQL table creation statement do not meet the requirements, the SQL table creation statement is updated and modified to obtain an updated SQL table creation statement, and the step of creating a data table based on the SQL table creation statement is executed based on the updated SQL table creation statement.

[0116] Furthermore, the matching unit is specifically configured to:

[0117] Calculating string edit distances between the first name field and each name field in the metadata management system;

[0118] Normalize each string edit distance separately to obtain multiple similarities;

[0119] When one of the similarities is greater than or equal to the similarity threshold, it is determined that a second name field similar to the first name field exists in the common attribute field in the metadata management system.

[0120] It should be noted that those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described devices and units can refer to the corresponding processes in the aforementioned embodiment of the vehicle network metadata management method, and will not be repeated here.

[0121] The vehicle network metadata management device provided in the above embodiment can be implemented in the form of a computer program. The computer program can be used in Figure 3 The shown vehicle network metadata management device runs on it.

[0122] An embodiment of the present application also provides a vehicle network metadata management device, including: a memory, a processor and a network interface connected via a system bus, the memory storing at least one instruction, and the at least one instruction being loaded and executed by the processor to implement all or part of the steps of the aforementioned vehicle network metadata management method.

[0123] Among them, the network interface is used for network communication, such as sending assigned tasks, etc. Those skilled in the art will understand that Figure 3 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0124] The processor may be a CPU, or other general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. A general-purpose processor may be a microprocessor, or any conventional processor. The processor is the control center of a computer device, connecting various parts of the entire computer device using various interfaces and lines.

[0125] The memory can be used to store computer programs and / or modules. The processor implements various functions of the computer device by running or executing the computer programs and / or modules stored in the memory, and calling the data stored in the memory. The memory can mainly include a program storage area and a data storage area. The program storage area can store an operating system, at least one application required for a function (such as a video playback function, an image playback function, etc.), etc.; the data storage area can store data created based on the use of the mobile phone (such as video data, image data, etc.). In addition, the memory can include high-speed random access memory and non-volatile memory, such as a hard disk, internal memory, a plug-in hard disk, a smart memory card (SmartMediaCard, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), at least one disk storage device, a flash memory device, or other volatile solid-state storage device.

[0126] An embodiment of the present application also provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, all or part of the steps of the aforementioned vehicle network metadata management method are implemented.

[0127] The embodiments of the present application implement all or part of the aforementioned processes, and may also be completed by instructing related hardware through a computer program. The computer program may be stored in a computer-readable storage medium, and the computer program, when executed by a processor, may implement the steps of each of the above methods. Among them, the computer program includes computer program code, and the computer program code may be in source code form, object code form, executable file or some intermediate form, etc. The computer-readable medium may include: any entity or device capable of carrying computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal and software distribution medium, etc. It should be noted that the content contained in the computer-readable medium may be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media do not include electric carrier signals and telecommunication signals.

[0128] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, servers, or computer program products. Thus, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage and optical storage) containing computer-usable program code.

[0129] It should be noted that, in this document, the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, article, or system comprising a series of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, article, or system. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, method, article, or system comprising the element.

[0130] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems) and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowcharts and / or block diagrams, as well as combinations of processes and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowcharts and / or block diagrams. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0131] The foregoing is merely a list of specific embodiments of the present application, intended to enable those skilled in the art to understand or implement the present application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application is not limited to the embodiments shown herein, but is intended to conform to the broadest scope consistent with the principles and novel features of the present application.

Claims

1. A method for managing metadata of an Internet of Vehicles, characterized in that: The following steps are involved: Extracting common attribute fields of metadata in the Internet of Vehicles and importing the common attribute fields into a metadata management system, wherein the common attribute fields include a name field and a description field; Extracting data source fields and importing the data source fields into the metadata management system, wherein the data source fields include one or more of a database type field, a database name field, a database instance field, a database address field, a database port field, and a database description field; Create an SQL table creation statement based on the data source fields; Creating a data table based on the SQL table creation statement, and parsing the SQL table creation statement to obtain field information of metadata to be managed on the data table, the field information including a first name field and a first description field; matching the first name field and the first description field with the name field and the description field in a metadata management system; When a second name field similar to the first name field and a second description field similar to the first description field are matched in the metadata management system, the first name field is updated based on the second name field, and the first description field is updated based on the second description field, so as to achieve metadata management; Wherein, after the step of updating the first description field based on the second description field, the method further includes: Conduct compliance checks on dataX data processing scripts; When the dataX data processing script meets the requirements, execute the dataX data processing script and obtain the configuration information of the dataX data processing script; Parsing the configuration information to obtain first basic information of the output table and second basic information of the input table, wherein the first basic information includes a data source field of the output table, a table name of the output table, and field information of metadata on the output table; and the second basic information includes a data source field of the input table, a table name of the input table, and field information of metadata on the input table; When there exists in the metadata management system a first table entity corresponding to the data source field of the output table and the table name of the output table, and a second table entity corresponding to the data source field of the input table and the table name of the input table, a table-level lineage relationship between the output table and the input table is generated based on the table name of the output table and the table name of the input table; The table-level kinship relationship is imported into a metadata management system.

2. The method for managing metadata of an Internet of Vehicles according to claim 1, wherein: After the step of matching the first name field and the first description field with the name field and the description field in the metadata management system, the method further includes: When a second name field similar to the first name field is not matched in the metadata management system or a second description field similar to the first description field is not matched in the metadata management system, the first name field and the first description field are imported into the metadata management system.

3. The method for managing metadata of the Internet of Vehicles according to claim 1, wherein: After the step of: when a first table entity corresponding to the data source field of the output table and the table name of the output table respectively exists in the metadata management system, and a second table entity corresponding to the data source field of the input table and the table name of the input table respectively exists in the metadata management system, the method further includes: Querying the metadata management system for first identifiers corresponding to column name fields of each column in the output table according to the first basic information; Querying the metadata management system for second identifiers corresponding to column name fields of each column in the input table according to the second basic information; A column-level lineage relationship is generated based on the first identifier and the second identifier, and the column-level lineage relationship is imported into a metadata management system.

4. The method for managing metadata of an Internet of Vehicles according to claim 1, wherein: Before the step of: when the metadata management system has a first table entity corresponding to the data source field of the output table and the table name of the output table, and a second table entity corresponding to the data source field of the input table and the table name of the input table, the method further includes: Match the metadata field information on the input table with the public attribute fields in the metadata management system; When no common attribute field similar to the field information of the metadata on the input table is matched in the metadata management system, the field information of the metadata on the input table is imported into the metadata management system.

5. The method for managing metadata of the Internet of Vehicles according to claim 1, wherein: Before the step of creating a data table based on the SQL table creation statement, the method further includes: Determine the accuracy and executability of the SQL table creation statement; If the accuracy and executability of the SQL table creation statement meet the requirements, then executing the step of creating a data table based on the SQL table creation statement; If the accuracy and executability of the SQL table creation statement do not meet the requirements, the SQL table creation statement is updated and modified to obtain an updated SQL table creation statement, and the step of creating a data table based on the SQL table creation statement is executed based on the updated SQL table creation statement.

6. The method for managing metadata of the Internet of Vehicles according to claim 1, wherein: The matching of a second name field similar to the first name field in the metadata management system includes: Calculating string edit distances between the first name field and each name field in the metadata management system; Normalize each string edit distance separately to obtain multiple similarities; When one of the similarities is greater than or equal to the similarity threshold, it is determined that a second name field similar to the first name field exists in the common attribute field in the metadata management system.

7. A vehicle network metadata management device, characterized in that: include: An extraction unit, configured to extract common attribute fields of metadata in the Internet of Vehicles and import the common attribute fields into a metadata management system, wherein the common attribute fields include a name field and a description field; Extracting data source fields and importing the data source fields into the metadata management system, wherein the data source fields include one or more of a database type field, a database name field, a database instance field, a database address field, a database port field, and a database description field; a processing unit configured to create an SQL table creation statement based on the data source field; create a data table based on the SQL table creation statement, and parse the SQL table creation statement to obtain field information of metadata to be managed on the data table, the field information including a first name field and a first description field; a matching unit, configured to match the first name field and the first description field with a name field and a description field in a metadata management system; a management unit configured to, upon matching a second name field similar to the first name field and a second description field similar to the first description field in the metadata management system, update the first name field based on the second name field and update the first description field based on the second description field, thereby implementing metadata management; Wherein, after the step of updating the first description field based on the second description field, the method further includes: Conduct compliance checks on dataX data processing scripts; When the dataX data processing script meets the requirements, execute the dataX data processing script and obtain the configuration information of the dataX data processing script; Parsing the configuration information to obtain first basic information of the output table and second basic information of the input table, wherein the first basic information includes a data source field of the output table, a table name of the output table, and field information of metadata on the output table; and the second basic information includes a data source field of the input table, a table name of the input table, and field information of metadata on the input table; When there exists in the metadata management system a first table entity corresponding to the data source field of the output table and the table name of the output table, and a second table entity corresponding to the data source field of the input table and the table name of the input table, a table-level lineage relationship between the output table and the input table is generated based on the table name of the output table and the table name of the input table; The table-level kinship relationship is imported into a metadata management system.

8. A vehicle network metadata management device, characterized in that: include: A memory and a processor, wherein the memory stores at least one instruction, and the at least one instruction is loaded and executed by the processor to implement the vehicle network metadata management method according to any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that: The computer storage medium stores a computer program, and when the computer program is executed by a processor, the vehicle network metadata management method according to any one of claims 1 to 6 is implemented.

Images