Asset data management method, system and device and storage medium

Abstract

The invention discloses an asset data management method, system and device and a storage medium, and the method comprises the steps: collecting IT asset data according to a configuration file corresponding to a data source type, and distributing the IT asset data to different interface tables based on a preset classification rule; normalizing the change records of the source data and the interface table data, and writing the change records of the source data and the interface table data into a model table according to fields according to a normalized processing result; and aggregating the change logs of the model table according to the primary keys corresponding to the change logs of the model table, and updating the target library by the aggregated change logs according to the configuration file for establishing the relationship. The IT asset data is collected according to the configuration file corresponding to the data source type, the change logs of the model table are aggregated according to the primary key corresponding to the change logs of the model table, and then the target library is updated according to the configuration file for establishing the relationship, so that compared with the prior art, the efficiency and the safety of IT asset management are improved.

Description

Technical Field

[0001] The present invention relates to the field of big data management technology, and in particular to an asset data management method, system, device and storage medium. Background Art

[0002] Currently, there are general data extraction tools that can support the processing of multi-source data, but in the IT asset data management scenario, there are still problems such as data fluctuation, data pollution, difficulty in tracing and lack of monitoring. On the one hand, when the upstream system changes, especially when the source sends a large amount of key-value duplicate data, it will cause the entire data processing chain to be overloaded, resulting in a large number of meaningless checks and modifications, thus causing efficiency problems; on the other hand, these tools lack the necessary conversion rules and rejection rules for data management, and cannot perform data cleaning and standardization; in addition, existing products usually do not care about field-level lineage analysis, and for assets affected by multiple source data, it is difficult to trace the specific attribute reporting system; finally, the data processing process does not leave enough traces, and the alarm configuration is not flexible enough, which will also lead to difficulties in process monitoring or post-event tracing.

[0003] Therefore, there is an urgent need for an asset data management method that can improve the efficiency and security of IT asset management. Summary of the invention

[0004] The main purpose of the present invention is to provide an asset data management method, system, device and storage medium, aiming to solve the technical problems of low efficiency and security of IT asset management in the prior art.

[0005] To achieve the above object, the present invention provides an asset data management method, the method comprising the following steps:

[0006] Collect IT asset data according to the configuration file corresponding to the data source type, and distribute the IT asset data to different interface tables based on preset classification rules;

[0007] Normalize the change records of source data and interface table data, and write the change records of source data and interface table data into the model table by field according to the normalization processing results;

[0008] The change logs of the model table are aggregated according to the primary keys corresponding to the change logs of the model table, and the aggregated change logs are used to update the target database according to the configuration files for establishing the relationship.

[0009] Optionally, after the step of aggregating the change log of the model table according to the primary key corresponding to the change log of the model table, and updating the target library with the aggregated change log according to the configuration file for establishing the relationship, the method further includes:

[0010] Monitoring the operating frequency of the IT asset data collection, the normalization processing, and the change log aggregation to obtain monitoring results;

[0011] Based on the monitoring results and the timeliness requirements corresponding to the IT asset data, the operating frequencies of collecting the IT asset data, the normalization processing, and aggregating the change log are adjusted respectively.

[0012] Optionally, the step of collecting IT asset data according to a configuration file corresponding to a data source type, and dividing the IT asset data into different interface tables based on a preset classification rule includes:

[0013] Determine a data source type, and determine parameters to be configured based on the data source type;

[0014] Generate a data access configuration file according to the parameters to be configured, parse the configuration file through Connector, and extract a data list according to the parsing result;

[0015] IT asset data is collected based on the data list, and the IT asset data is diverted to different interface tables based on preset classification rules, and the interface tables are designed for posting source.

[0016] Optionally, after the step of collecting IT asset data based on the data list and dividing the IT asset data into different interface tables based on preset classification rules, the method further includes:

[0017] Determine the key fields to be compared, compare the source data with the interface table data based on the key fields to be compared, and obtain a comparison result;

[0018] According to the comparison result, a change record of the source data is determined, wherein the change record includes an interface table name, a change attribute, an updated value, and a data source code.

[0019] Optionally, the step of normalizing the change records of the source data and the interface table data, and writing the change records of the source data and the interface table data into the model table by field according to the normalization processing result includes:

[0020] Generate a change log of the source data based on the change records of the source data and the interface table data;

[0021] Determine whether each field in the change log of the source data complies with standard data according to preset verification rules;

[0022] If the field does not conform to the data specification, the non-conforming data is rejected and an alarm message is generated;

[0023] If the field meets the standard data, the format of the corresponding content of the field is converted according to the preset conversion rules and then written into the model table.

[0024] Optionally, the step of aggregating the change log of the model table according to the primary key corresponding to the change log of the model table, and updating the target library with the aggregated change log according to the configuration file for establishing the relationship includes:

[0025] When the record of the model table is changed, the change log of the model table is updated, and the change log of the model table is aggregated according to the primary key corresponding to the change log of the model table to obtain the aggregated change log;

[0026] Analyze the mapping relationship between the model table and the target library, and determine the mapping rule and relationship strategy between the model table and the target library according to the relationship analysis result;

[0027] Based on the mapping rule and the relationship strategy, the aggregated change log is used to update the target database according to the configuration file for establishing the relationship.

[0028] Optionally, after the step of aggregating the change log of the model table according to the primary key corresponding to the change log of the model table, and updating the target library with the aggregated change log according to the configuration file for establishing the relationship, the method further includes:

[0029] Monitor the connectivity of the interfaces corresponding to each of the data sources, the backlog of KAFKA messages, and the changes in data traffic;

[0030] When the connectivity of the interface corresponding to the data source, the backlog of KAFKA messages, and the change in data flow are abnormal, an alarm message is generated;

[0031] When updating the target database, if the target node search in the target database fails, resulting in an abnormal relationship building, a relationship building abnormality log is generated and failure retries are performed regularly.

[0032] In addition, to achieve the above-mentioned purpose, the present invention also proposes an asset data management system, the system comprising:

[0033] A collection module, used to collect IT asset data according to the configuration file corresponding to the data source type, and to distribute the IT asset data to different interface tables based on preset classification rules;

[0034] The summary module is used to normalize the change records of the source data and the interface table data, and write the change records of the source data and the interface table data into the model table according to the normalization processing results by field;

[0035] The storage module is used to aggregate the change logs of the model table according to the primary key corresponding to the change logs of the model table, and update the target library with the aggregated change logs according to the configuration file for establishing the relationship.

[0036] In addition, to achieve the above-mentioned purpose, the present invention also proposes an asset data management device, which includes: a memory, a processor, and an asset data management program stored in the memory and executable on the processor, and the asset data management program is configured to implement the steps of the asset data management method described above.

[0037] In addition, to achieve the above-mentioned purpose, the present invention also proposes a storage medium, on which an asset data management program is stored, and when the asset data management program is executed by a processor, the steps of the asset data management method described above are implemented.

[0038] The present invention discloses collecting IT asset data according to a configuration file corresponding to a data source type, and shunting the IT asset data to different interface tables based on preset classification rules; normalizing the change records of source data and interface table data, and writing the change records of source data and interface table data into a model table by field according to the normalization processing result; aggregating the change log of the model table according to the primary key corresponding to the change log of the model table, and updating the target library with the aggregated change log according to the configuration file for establishing a relationship. Since the present invention collects IT asset data according to the configuration file corresponding to the data source type, shunting the IT asset data to different interface tables based on preset classification rules, writing the change records of source data and interface table data into the model table by field according to the normalization processing result, and finally aggregating the change log of the model table according to the primary key corresponding to the change log of the model table, and updating the target library with the aggregated change log according to the configuration file for establishing a relationship, compared with the prior art, the present invention effectively improves the efficiency and security of IT asset management. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 This is a flow chart of the first embodiment of the asset data management method of the present invention;

[0040] Figure 2 A schematic diagram of the collection and management process in the asset data management method of the present invention;

[0041] Figure 3 This is a flow chart of a second embodiment of the asset data management method of the present invention;

[0042] Figure 4 It is a schematic diagram of the overall framework of the system in the asset data management method of the present invention;

[0043] Figure 5This is a flow chart of a third embodiment of the asset data management method of the present invention;

[0044] Figure 6 This is a structural block diagram of the first embodiment of the asset data management system of the present invention;

[0045] Figure 7 It is a structural diagram of an asset data management device in a hardware operating environment involved in an embodiment of the present invention.

[0046] The realization of the purpose, functional features and advantages of the present invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0047] It should be understood that the specific embodiments described herein are only used to explain the present invention, and are not used to limit the present invention.

[0048] The embodiment of the present invention provides a method for managing asset data. Figure 1 , Figure 1 This is a flow chart of the first embodiment of the asset data management method of the present invention.

[0049] In this embodiment, the asset data management method includes steps S10 to S30:

[0050] Step S10: collecting IT asset data according to the configuration file corresponding to the data source type, and dividing the IT asset data into different interface tables based on preset classification rules.

[0051] It should be noted that the execution subject of this embodiment can be a computer server device with data processing, network communication and program running functions applied to the IT asset data management scenario, such as a personal computer, a laptop computer, a mobile phone, a smart watch, etc., or an electronic device capable of realizing the above functions, an asset data management device, etc. The following takes the asset data management device system (hereinafter referred to as the system) as an example to illustrate this embodiment and the following embodiments.

[0052] It should be explained that the data source types in this embodiment include: (1) external API: the centralized configuration management platform (hereinafter referred to as CMDB) actively calls the external API to collect data; (2) KAFKA reporting: CMDB receives data from KAFKA to complete the collection; (3) CMDB API: CMDB provides an API for external systems to report data.

[0053] It should be understood that IT asset data refers to data information of various assets related to enterprise information technology (IT). These asset data may involve hardware equipment, software applications, data files, network infrastructure, service contracts and other aspects.

[0054] It is understandable that the parameters in the configuration file corresponding to the data source type include: the system platform code of the source party, the person in charge, the acquisition method, the authentication configuration and the extraction path.

[0055] It should be noted that the corresponding configuration parameters can be determined according to the different types of data sources, and the configuration file for data access, that is, the configuration file corresponding to the data source type, can be generated according to the configuration parameters. The content of the configuration file, taking the CMDB receiving data from KAFKA as an example, clusterName is the name of the KAFKA cluster, topicName indicates the TOPIC to be consumed, and groupId, username and password define the consumer group, username and password of this consumer end.

[0056] In a specific implementation, the data source type can be determined, and the parameters to be configured can be determined based on the data source type; a data access configuration file can be generated based on the parameters to be configured, the configuration file can be parsed through the Connector, and a data list can be extracted based on the parsing result; IT asset data can be collected based on the data list, and the IT asset data can be diverted to different interface tables based on preset classification rules, and the interface table is designed for source attachment.

[0057] It should be noted that after reading the data (after collecting IT asset data based on the data list), the preset classification rules are used to determine which interface table the data should be placed in. The interface table here is designed to be source-attached to record the true situation of the source data, which is convenient for subsequent tracing of the data situation. A data source usually contains data that affects multiple models. According to the preset classification rules, the data is diverted to different interface tables. Finally, the source data is compared with the interface table data, and the changes are recorded as logs. The logs contain the interface table name, changed attributes, updated values, and data source codes. Logs will not be generated for unchanged or duplicate information.

[0058] In a specific implementation, after the step of collecting IT asset data based on the data list and diverting the IT asset data to different interface tables based on preset classification rules, it also includes: determining key fields to be compared, comparing the source data with the interface table data based on the key fields to be compared, and obtaining a comparison result; based on the comparison result, determining a change record of the source data, the change record including the interface table name, change attributes, updated value and data source code.

[0059] Step S20: normalize the change records of the source data and the interface table data, and write the change records of the source data and the interface table data into the model table according to the normalization processing results by field.

[0060] It should be noted that the above-mentioned normalization processing includes field-level verification, merging, regularization and other processing of the data, and recording the changes of the source data and the interface table data as logs.

[0061] It is understandable that a model table may refer to a structured data table after business logic and data processing, which is used to support specific analysis, reporting or application requirements.

[0062] It should be explained that in the process of writing the change records of the source data and the interface table data into the model table according to the normalization processing results by field, the system rejects the data that does not meet the specifications according to the preset verification rules and issues an alarm. For the attributes that need to be written after standardization, they are calculated and processed according to the preset conversion rules before being written into the model table. The table structure of the model table is consistent with that of the target library. When the model table records are changed, logs are recorded.

[0063] Step S30: Aggregate the change logs of the model table according to the primary key corresponding to the change logs of the model table, and update the target database with the aggregated change logs according to the configuration file for establishing the relationship.

[0064] It should be noted that the above-mentioned aggregating the change log of the model table according to the primary key corresponding to the change log of the model table can merge duplicate data, solve the problem of duplicate consumption, and effectively reduce the frequency of changes.

[0065] When the change log of the model table is written to the target database, a certain batch (for example, 1,000) of log data will be read first, and then the final result of a property of a certain instance in the batch will be updated to the target database through the aggregation model and primary key. For example, the "logonStatus" property of the CMBAGENT instance "A2DEC516-C473-4D07-96D0-E92418475FC5" frequently changes in a short period of time. During the storage stage, multiple change logs will be merged, and only the final status will be updated once in the target database, which effectively reduces the frequency of changes.

[0066] It should be explained that since the target library stores data in a graph structure and stores the node and relationship data of the instance, the change log of the model table needs to be implemented in the target library according to certain mapping rules and relationship strategies. The attribute mapping rules can be saved as a JSON structure, and the model table fields correspond to the library attributes one by one.

[0067] It is understandable that the configuration file for establishing a relationship involves multiple steps and different configuration files, including entity mapping, relationship mapping, and database connection configuration. For example, in the configuration file for establishing a relationship in this embodiment, "sTableField" indicates the attribute field for establishing a relationship in this instance; "fieldRestrictRegex" is a verification rule, and only the attributes that meet this rule can establish a relationship; "fieldExtractRegex" is an attribute field extraction rule, such as ".+(?= / )" means extracting the characters before the "." separator in the string; "remoteModelId" indicates the association model; "remoteModelAttr" indicates the association model attribute; "relationId" indicates the relationship ID; "createRelationType" indicates the type of relationship establishment, which has multiple types such as default type and one-to-one. The meaning of the following configuration is to extract the content before the "." of the "primaryAdmin" field of the current model CDDMNODE record as the primary key value of the USER model, find the USER model data with "code" equal to the primary key value, and establish the "CDDMNODE_USERS_primary" relationship between the current instance and the USER model. When data is deleted, the deletion flag is recorded in the model table and logical deletion is performed. When the data is landed in the target database, actual deletion is performed based on the deletion flag.

[0068] For example, refer to Figure 2 , Figure 2The present invention is a schematic diagram of the collection and management process in the asset data management method. In the figure, ACSHOST, ACSCLUSTER and ACSPUBLICIP are models for collection and management, etl_acshost_interface, etl_acscluster_interface and etl_acspublicip_interface are interface tables, and etl_s_model_acshost, etl_s_model_acscluster and etl_s_model_acspublicip are summary tables. The ACSHOST (ACS physical machine) model collection and management with ACS metadata (KAFKA) as the source is taken as an example to illustrate the scheme. (1) Collection service: Start the service to obtain a unique identification code, and filter and obtain the data source to be processed according to the identification code. According to the definition of the preset data source definition method, the data source is monitored to obtain data; in the field mapping definition of the data source and the interface table, according to the matching of the identifier, when the "serviceType" in a certain data is "VMHost", the data is disassembled into the interface table: etl_acshost_interface, and the interface table field is defined according to the data source, and the timestamp and processing flag are added. (2) Summarization service: read the interface table log, and write the interface table changes to the summary table: etl_s_model_acshost according to the summary definition field mapping table configuration table. After the service is started, the list of summaries to be monitored is obtained, and threads are obtained from the thread pool. Each thread obtains the interface table to be monitored from the configuration table, checks the records with a processing identifier of 0 from the interface table log, and puts them into the summary table after processing according to the specified rules according to the field mapping table. (3) Storage service: read the summary table and write the data to the target database.

[0069] It should be noted that the data in the collection stage, summary stage, and storage stage are related and are completed by multi-threaded programs. The main program mainly checks which data needs to be stored, that is, checks the summary schedule (processing table), finds the summary data that needs to be stored, and completes the storage according to the configuration (mapping). Because the collection and summary are running uninterruptedly, the storage data is automatically prepared.

[0070] In a specific implementation, when a record in a model table changes, the change log of the model table is updated, and the change log of the model table is aggregated according to the primary key corresponding to the change log of the model table to obtain an aggregated change log; the mapping relationship between the model table and the target library is analyzed, and the mapping rules and relationship strategies between the model table and the target library are determined according to the relationship analysis results; based on the mapping rules and the relationship strategies, the aggregated change log is used to update the target library according to the configuration file for establishing the relationship.

[0071] This embodiment discloses collecting IT asset data according to the configuration file corresponding to the data source type, and diverting the IT asset data to different interface tables based on preset classification rules; normalizing the change records of source data and interface table data, and writing the change records of source data and interface table data into the model table by field according to the normalization processing result; aggregating the change log of the model table according to the primary key corresponding to the change log of the model table, and updating the target library with the aggregated change log according to the configuration file for establishing the relationship. Since this embodiment collects IT asset data according to the configuration file corresponding to the data source type, diverts IT asset data to different interface tables based on preset classification rules, writes the change records of source data and interface table data into the model table by field according to the normalization processing result, and finally aggregates the change log of the model table according to the primary key corresponding to the change log of the model table and updates the target library according to the configuration file for establishing the relationship, compared with the prior art, this embodiment effectively improves the efficiency and security of IT asset management.

[0072] refer to Figure 3 , Figure 3 This is a flow chart of the second embodiment of the asset data management method of the present invention.

[0073] Based on the first embodiment, in this embodiment, after step S30, steps S40 to S50 are further included:

[0074] Step S40: monitoring the operating frequency of the IT asset data collection, the normalization processing and the change log aggregation to obtain a monitoring result.

[0075] Step S50: Based on the monitoring result and the timeliness requirement corresponding to the IT asset data, the operating frequencies of collecting the IT asset data, the normalization processing and the aggregating the change log are adjusted respectively.

[0076] It should be noted that the execution of the above steps S10, S20 and S30 is implemented by scheduling tasks. Three scheduled tasks are defined in three stages, and operating parameters and operating frequencies are set in each stage, thereby realizing the start and stop of management, as well as the storage rate control of data with different timeliness and data volume models.

[0077] The collection phase (i.e., step S10) connects to multiple systems, and network connectivity, service changes, and message queue congestion can cause consumption anomalies. Abnormal changes such as interface connectivity, KAFKA message backlogs, and sudden increases and decreases in data traffic will be monitored here, and the administrator will be notified in a timely manner. The consumption and aggregation process (i.e., step S20) can reject attributes that do not conform to the agreed format, and will record abnormal logs and feedback to developers. When updating the target library (i.e., step S30), due to the instability of the target library, the target node search fails, resulting in abnormal relationship building. At this time, the log will be recorded for developers to track and analyze, and failure retries will be automatically performed at regular intervals.

[0078] In a specific implementation, after the step of aggregating the change log of the model table according to the primary key corresponding to the change log of the model table, and updating the target library with the aggregated change log according to the configuration file for establishing the relationship, it also includes: monitoring the connectivity of the interfaces corresponding to each of the data sources, the backlog of KAFKA messages, and the change in data flow; generating an alarm message when the connectivity, the backlog of KAFKA messages, and the change in data flow of the interfaces corresponding to the data sources are abnormal; when updating the target library, if the target node search of the target library fails, resulting in an abnormal relationship establishment, a relationship establishment abnormality log is generated, and failure retries are performed regularly.

[0079] It needs to be explained that data is gradually landed from the data source to the interface table, summary table, and finally to the target database. During this period, the data has undergone rule-defined conversion, standard data is landed, and abnormal data is rejected, solving the problem of data verification; after the change log is aggregated, duplicate data is merged, solving the problem of duplicate consumption; according to the configuration, it enters the landing database at a fixed frequency, solving the problem of high-frequency storage; this ensures that the landing database is safe, stable, and efficient OLAP service in the case of massive data and high-frequency retransmission.

[0080] It is understandable that the administrator can view the rejection data, verification errors, summary errors, storage errors and update logs generated during the system operation phase. In combination with the logs, this embodiment also provides: rejection alarm, KAFKA congestion alarm, data source connectivity alarm, and data volume fluctuation alarm. These measures leave enough time and clues for managers, provide strong data guarantee for auditing and troubleshooting, and further improve the efficiency and security of IT asset management.

[0081] For example, refer to Figure 4 , Figure 4It is a schematic diagram of the overall framework of the system in the asset data management method of the present invention. In the figure, the overall framework is divided into four modules: acquisition module, summary module, storage module and monitoring module. Among them, the acquisition module includes data source definition (in the data source definition stage, the parameters that need to be configured include: the system platform code of the source, the person in charge, the acquisition method, the authentication configuration and the extraction path, etc.), Connector parsing configuration file and preset classification rules. The acquisition module integrates multiple data sources and supports the consumption of source data in the form of KAFKA, API, and external API active reporting. The summary module includes data verification, preset conversion rules and record logs. The core function of this is to perform field-level verification, merging, regularization and other normalization processing on the data, and record field changes as logs. The storage module mainly aggregates the logs by primary key (i.e., change log compression), reduces the number of changes, and updates the target library according to the configuration file, including attribute writing, relationship creation and data deletion (i.e., the storage strategy includes attribute rules, relationship rules, deletion rules and storage inspection). The monitoring module is responsible for the scheduling of various tasks such as collection, aggregation, and warehousing. By adjusting the running frequency of each task, it meets the timeliness requirements of different asset types and relieves database pressure. At the same time, the monitoring module also monitors data source interface anomalies, KAFKA consumption backlogs, abnormal data, etc., and notifies developers and reporters. The monitoring module includes functions such as task scheduling, consumption monitoring, abnormal alarms, and error retry.

[0082] This embodiment discloses monitoring the operating frequency of the IT asset data collection, the normalization processing and the change log aggregation to obtain monitoring results; based on the monitoring results and the timeliness requirements corresponding to the IT asset data, the operating frequencies of the IT asset data collection, the normalization processing and the change log aggregation are adjusted respectively. Compared with the prior art, this embodiment adjusts the operating frequencies of the IT asset data collection, the normalization processing and the change log aggregation respectively based on the monitoring results and the timeliness requirements corresponding to the IT asset data, solves the problem of high-frequency warehousing, and further ensures that the landing library is safe, stable and efficient OLAP service under the condition of massive data and high-frequency retransmission influx.

[0083] refer to Figure 5 , Figure 5 It is a flowchart of the third embodiment of the asset data management method of the present invention.

[0084] Based on the above embodiments, in this embodiment, step S20 includes steps S201 to S204:

[0085] Step S201: Generate a change log of the source data based on the change records of the source data and the interface table data.

[0086] Step S202: judging whether each field in the change log of the source data complies with standard data according to preset verification rules.

[0087] Step S203: If the field does not conform to the data specification, the data that does not conform to the specification is rejected and an alarm message is generated.

[0088] Step S205: If the field meets the standard data, the format of the corresponding content of the field is converted according to the preset conversion rules and then written into the model table.

[0089] It should be noted that the preset validation rules and preset conversion rules have built-in common data processing logic, such as judging whether the data is empty, verifying the standard format of personnel information, removing leading and trailing spaces, extracting JSON attributes, etc., which meet most of the data validation and conversion logic. If special rules are required, customized development can be carried out.

[0090] In the specific implementation, the change records of source data and interface table data are written into the model table by field. During this period, according to the verification rules, data that does not meet the specifications is rejected and an alarm is issued. For attributes that need to be written after standardization, they are calculated and processed according to the configured conversion rules before being written into the model table. The table structure of the model table is consistent with that of the target database. When the model table records are changed, logs are recorded.

[0091] This embodiment discloses generating a change log of the source data based on the change records of the source data and the interface table data; judging whether each field in the change log of the source data complies with the standard data according to the preset verification rules; if the field does not comply with the standard data, rejecting the non-standard data and generating an alarm message; if the field complies with the standard data, format conversion is performed on the corresponding content of the field according to the preset conversion rules and then written into the model table. Compared with the prior art, the data in this embodiment has been converted according to the rules, the standard data has been implemented, and the abnormal data has been rejected, which solves the problem of data verification and further improves the security of IT asset management.

[0092] In addition, an embodiment of the present invention further proposes a storage medium, on which an asset data management program is stored. When the asset data management program is executed by a processor, the steps of the asset data management method described above are implemented.

[0093] Reference Figure 6 , Figure 6 This is a structural block diagram of the first embodiment of the asset data management system of the present invention.

[0094] like Figure 6 As shown, the asset data management system proposed in the embodiment of the present invention includes: a collection module 601, a summary module 602 and a warehousing module 603.

[0095] The collection module 601 is used to collect IT asset data according to the configuration file corresponding to the data source type, and to distribute the IT asset data to different interface tables based on preset classification rules.

[0096] The summary module 602 is used to normalize the change records of the source data and the interface table data, and write the change records of the source data and the interface table data into the model table according to the normalization processing results by field.

[0097] The storage module 603 is used to aggregate the change logs of the model table according to the primary key corresponding to the change logs of the model table, and update the target database with the aggregated change logs according to the configuration file for establishing the relationship.

[0098] The acquisition module 601 is also used to determine the data source type, and determine the parameters to be configured based on the data source type; generate a data access configuration file based on the parameters to be configured, parse the configuration file through the Connector, and extract a data list based on the parsing result; collect IT asset data based on the data list, and divert the IT asset data to different interface tables based on preset classification rules, and the interface table is a source-attached design.

[0099] The acquisition module 601 is also used to determine the key fields to be compared, compare the source data with the interface table data based on the key fields to be compared, and obtain a comparison result; based on the comparison result, determine the change record of the source data, and the change record includes the interface table name, change attributes, updated value and data source code.

[0100] The warehousing module 603 is also used to update the change log of the model table when the record of the model table is changed, and aggregate the change log of the model table according to the primary key corresponding to the change log of the model table to obtain the aggregated change log; analyze the mapping relationship between the model table and the target library, and determine the mapping rules and relationship strategy between the model table and the target library according to the relationship analysis result; based on the mapping rules and the relationship strategy, update the target library with the aggregated change log according to the configuration file for establishing the relationship.

[0101] The present system embodiment discloses collecting IT asset data according to the configuration file corresponding to the data source type, and diverting the IT asset data to different interface tables based on preset classification rules; normalizing the change records of source data and interface table data, and writing the change records of source data and interface table data into the model table by field according to the normalization processing result; aggregating the change log of the model table according to the primary key corresponding to the change log of the model table, and updating the target library with the aggregated change log according to the configuration file for establishing the relationship. Since the present system embodiment collects IT asset data according to the configuration file corresponding to the data source type, diverts the IT asset data to different interface tables based on preset classification rules, writes the change records of source data and interface table data into the model table by field according to the normalization processing result, and finally aggregates the change log of the model table according to the primary key corresponding to the change log of the model table and updates the target library according to the configuration file for establishing the relationship, compared with the prior art, the present system embodiment effectively improves the efficiency and security of IT asset management.

[0102] Based on the above-mentioned first embodiment of the asset data management system of the present invention, a second embodiment of the asset data management system of the present invention is proposed.

[0103] In this embodiment, the asset data management system further includes: a monitoring module.

[0104] The monitoring module is used to monitor the operating frequency of the IT asset data collection, the normalization processing and the change log aggregation to obtain monitoring results; based on the monitoring results and the timeliness requirements corresponding to the IT asset data, the operating frequency of the IT asset data collection, the normalization processing and the change log aggregation is adjusted respectively.

[0105] The monitoring module is also used to monitor the connectivity of the interfaces corresponding to each of the data sources, the backlog of KAFKA messages, and the changes in data flow; when the connectivity of the interfaces corresponding to the data sources, the backlog of KAFKA messages, and the changes in data flow are abnormal, an alarm message is generated; when updating the target library, if the target node search of the target library fails, resulting in an abnormal relationship establishment, a relationship establishment abnormality log is generated, and failure retries are performed regularly.

[0106] Other embodiments or specific implementations of the asset data management system of the present invention can refer to the above-mentioned method embodiments and will not be described in detail here.

[0107] The present application provides an asset data management device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the asset data management method in the above-mentioned embodiment one.

[0108] Reference below Figure 7 , which shows a schematic diagram of the structure of an asset data management device suitable for implementing the embodiment of the present application. The asset data management device in the embodiment of the present application may include but is not limited to mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Descriptions), PMPs (Portable Media Players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Figure 7 The asset data management device shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0109] like Figure 7As shown, the asset data management device may include a processing device 1001 (e.g., a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM: Read Only Memory) 1002 or a program loaded from a storage device 1003 to a random access memory (RAM: Random Access Memory) 1004. In RAM1004, various programs and data required for the operation of the asset data management device are also stored. The processing device 1001, ROM1002, and RAM1004 are connected to each other through a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Generally, the following systems can be connected to the I / O interface 1006: an input device 1007 including, for example, a touch screen, a touch pad, a keyboard, a mouse, an image sensor, a microphone, an accelerometer, a gyroscope, etc.; an output device 1008 including, for example, a liquid crystal display (LCD: Liquid Crystal Display), a speaker, a vibrator, etc.; a storage device 1003 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 1009. The communication device 1009 can allow the asset data management device to communicate with other devices wirelessly or by wire to exchange data. Although the figure shows an asset data management device with various systems, it should be understood that it is not required to implement or have all the systems shown. More or fewer systems can be implemented or have alternatively.

[0110] In particular, according to the embodiments disclosed in the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the embodiments disclosed in the present application include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device, or installed from a storage device 1003, or installed from a ROM 1002. When the computer program is executed by the processing device 1001, the above-mentioned functions defined in the method of the embodiment disclosed in the present application are executed.

[0111] The asset data management device provided by the present application adopts the asset data management method in the above embodiment, which can solve the technical problem of low efficiency and security of IT asset management in the prior art. Compared with the prior art, the beneficial effects of the asset data management device provided by the present application are the same as the beneficial effects of the asset data management method provided by the above embodiment, and the other technical features in the asset data management device are the same as the features disclosed in the method of the previous embodiment, which will not be repeated here.

[0112] It should be understood that the various parts disclosed in this application can be implemented by hardware, software, firmware or a combination thereof. In the description of the above embodiments, specific features, structures, materials or characteristics can be combined in any one or more embodiments or examples in a suitable manner.

[0113] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

[0114] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or system including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or system. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or system including the element.

[0115] The serial numbers of the above embodiments of the present invention are only for description and do not represent the advantages or disadvantages of the embodiments.

[0116] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as a read-only memory / random access memory, a magnetic disk, or an optical disk), and includes a number of instructions for enabling a terminal device (which can be a mobile phone, a computer, a server, an air conditioner, or a network device, etc.) to execute the methods described in each embodiment of the present invention.

[0117] The above are only preferred embodiments of the present invention, and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A method for managing asset data, characterized in that: The method comprises: Collect IT asset data according to the configuration file corresponding to the data source type, and distribute the IT asset data to different interface tables based on preset classification rules; Normalize the change records of source data and interface table data, and write the change records of source data and interface table data into the model table by field according to the normalization processing results; The change logs of the model table are aggregated according to the primary keys corresponding to the change logs of the model table, and the aggregated change logs are used to update the target database according to the configuration files for establishing the relationship.

2. The asset data management method according to claim 1, characterized in that: After the step of aggregating the change log of the model table according to the primary key corresponding to the change log of the model table, and updating the target library with the aggregated change log according to the configuration file for establishing the relationship, the method further includes: Monitoring the operating frequency of the IT asset data collection, the normalization processing, and the change log aggregation to obtain monitoring results; Based on the monitoring results and the timeliness requirements corresponding to the IT asset data, the operating frequencies of collecting the IT asset data, the normalization processing, and aggregating the change log are adjusted respectively.

3. The asset data management method according to claim 1, characterized in that: The step of collecting IT asset data according to the configuration file corresponding to the data source type, and dividing the IT asset data into different interface tables based on preset classification rules includes: Determine a data source type, and determine parameters to be configured based on the data source type; Generate a data access configuration file according to the parameters to be configured, parse the configuration file through Connector, and extract a data list according to the parsing result; IT asset data is collected based on the data list, and the IT asset data is diverted to different interface tables based on preset classification rules, and the interface tables are designed for posting source.

4. The asset data management method according to claim 3, characterized in that: After the step of collecting IT asset data based on the data list and dividing the IT asset data into different interface tables based on preset classification rules, the method further includes: Determine the key fields to be compared, compare the source data with the interface table data based on the key fields to be compared, and obtain a comparison result; According to the comparison result, a change record of the source data is determined, wherein the change record includes an interface table name, a change attribute, an updated value, and a data source code.

5. The asset data management method according to claim 1, characterized in that: The step of normalizing the change records of the source data and the interface table data, and writing the change records of the source data and the interface table data into the model table by field according to the normalization processing result includes: Generate a change log of the source data based on the change records of the source data and the interface table data; Determine whether each field in the change log of the source data complies with standard data according to preset verification rules; If the field does not conform to the data specification, the non-conforming data is rejected and an alarm message is generated; If the field meets the standard data, the format of the corresponding content of the field is converted according to the preset conversion rules and then written into the model table.

6. The asset data management method according to claim 1, characterized in that: The step of aggregating the change log of the model table according to the primary key corresponding to the change log of the model table, and updating the target library with the aggregated change log according to the configuration file for establishing the relationship includes: When the record of the model table is changed, the change log of the model table is updated, and the change log of the model table is aggregated according to the primary key corresponding to the change log of the model table to obtain the aggregated change log; Analyze the mapping relationship between the model table and the target library, and determine the mapping rule and relationship strategy between the model table and the target library according to the relationship analysis result; Based on the mapping rule and the relationship strategy, the aggregated change log is used to update the target database according to the configuration file for establishing the relationship.

7. The asset data management method according to claim 2, characterized in that: After the step of aggregating the change log of the model table according to the primary key corresponding to the change log of the model table, and updating the target library with the aggregated change log according to the configuration file for establishing the relationship, the method further includes: Monitor the connectivity of the interfaces corresponding to each of the data sources, the backlog of KAFKA messages, and the changes in data traffic; When the connectivity of the interface corresponding to the data source, the backlog of KAFKA messages, and the change in data flow are abnormal, an alarm message is generated; When updating the target database, if the target node search in the target database fails, resulting in an abnormal relationship building, a relationship building abnormality log is generated and failure retries are performed regularly.

8. An asset data management system, characterized in that: The system comprises: A collection module, used to collect IT asset data according to the configuration file corresponding to the data source type, and to distribute the IT asset data to different interface tables based on preset classification rules; The summary module is used to normalize the change records of the source data and the interface table data, and write the change records of the source data and the interface table data into the model table according to the normalization processing results by field; The storage module is used to aggregate the change logs of the model table according to the primary key corresponding to the change logs of the model table, and update the target library with the aggregated change logs according to the configuration file for establishing the relationship.

9. An asset data management device, characterized in that: The device comprises: a memory, a processor, and an asset data management program stored in the memory and executable on the processor, wherein the asset data management program is configured to implement the steps of the asset data management method as described in any one of claims 1 to 7.

10. A storage medium, characterized in that: An asset data management program is stored on the storage medium, and when the asset data management program is executed by the processor, the steps of the asset data management method according to any one of claims 1 to 7 are implemented.

Images