Method, device, computer equipment and storage medium for entering gateway published data

Abstract

The present application relates to a method, device, computer equipment and storage medium for entering gateway published data. The method includes: obtaining a data product package of multiple gateway published data, and the current data to be published in the namespace of each cluster in the configuration center, and converting the data structure of each gateway published data in the data product package into array structure data, and then performing deduplication processing to obtain each target gateway published data of the data product package; identifying the target gateway published data already existing in the configuration center and backing it up; publishing each target gateway published data in the namespace of each cluster respectively; in the case of a target gateway published data failure, rolling back the failed namespace through the backed up target gateway published data, and republishing until all target gateway published data are successfully published; and then entering all target gateway published data. Thereby improving the efficiency of gateway published data entry.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence and data entry technology, and in particular to a method, device, computer equipment and storage medium for entering data published by a gateway. Background Art

[0002] When the gateway saves important data such as Application, Api, ApiOperation, and container, it needs to be manually entered. In the case of multiple clusters, the same data needs to be manually entered multiple times, and it cannot be automatically imported and published according to the configuration package provided by the other party.

[0003] When the other party provides a DSL product package containing Application, Api, and ApiOperation configurations, and needs to be published in multiple clusters, the gateway can only manually add its information to the database and publish it manually. Manual entry is not only inefficient, but also has a high error rate. In addition, if the same batch of product data is in multiple clusters, once the data in one cluster goes wrong, it cannot be rolled back in time, and the final consistency of the data cannot be ensured, resulting in low efficiency of gateway data entry. Summary of the invention

[0004] Based on this, it is necessary to provide a method, device, computer equipment, computer-readable storage medium and computer program product for entering gateway published data in response to the above technical problems.

[0005] In a first aspect, the present application provides a method for entering gateway published data. The method comprises:

[0006] Obtain a data product package of multiple gateway release data transmitted by a third-party platform, and the current data to be released in the namespace of each cluster in the configuration center, and if the integrity of the data product package is correct, convert the data structure of each gateway release data in the data product package to obtain each array structure data of the data product package;

[0007] Deduplication processing is performed on each array structure data of the data product package to obtain each target gateway publishing data of the data product package, and based on each target gateway publishing data and the current data to be published in each namespace, the target gateway publishing data existing in the configuration center is identified;

[0008] Backing up the existing target gateway publishing data, and publishing the target gateway publishing data in the namespace of each cluster respectively;

[0009] In the event that the target gateway fails to publish data, the target gateway publishes data through the backed-up target gateway, rolls back the namespace where the publishing failed, and republishes the target gateway publish data where the publishing failed, until all target gateway publish data are successfully published;

[0010] Store all successfully published target gateway publishing data into the database to complete the gateway publishing data entry task.

[0011] Optionally, when the integrity of the data product package is correct, before converting the data structure of each gateway publishing data in the data product package to obtain each array structure data of the data product package, the method further includes:

[0012] Extracting each gateway published data in the data product package, and respectively extracting the data content of each gateway published data and the data identification information of each gateway published data;

[0013] For each gateway published data content, determine whether the data content of the gateway published data is empty, and determine whether the data identification information of the gateway published data is correct;

[0014] When the data content of the gateway published data is empty or the data identification information of the gateway published data is incorrect, the gateway published data is marked as abnormal gateway published data, and each abnormal gateway published data is fed back to the staff client;

[0015] In the case that the data content of the data published by the gateway is empty and the data identification information of the data published by the gateway is incorrect, it is determined that the integrity of the data product package is correct.

[0016] Optionally, when the integrity of the data product package is correct, the data structure of each gateway publishing data in the data product package is converted to obtain each array structure data of the data product package, including:

[0017] Identify the data type of each gateway published data, and based on the data type of each gateway published data, identify the corresponding relationship between each gateway published data;

[0018] Based on the correspondence between the data published by each gateway, the data format of the data published by each gateway is converted into initial array structure data, and each initial array structure data is mapped and improved through a graph mapping strategy to obtain the array structure data corresponding to each gateway published data.

[0019] Optionally, the deduplication processing is performed on each array structure data of the data product package to obtain each target gateway publishing data of the data product package, including:

[0020] Respectively identify the same data content in each array structure data, and perform deduplication processing on each of the same data contents to obtain the initial target gateway release data;

[0021] In the target database, it is determined whether there is data content identical to that of each initial target gateway published data, and the initial target gateway published data with the same data content is marked to obtain the marked gateway published data;

[0022] The marked gateway publishing data and the non-marked initial target gateway publishing data are used as the target gateway publishing data of the data product package.

[0023] Optionally, the identifying the target gateway publishing data existing in the configuration center based on the target gateway publishing data and the current data to be published in each namespace includes:

[0024] Each target gateway publishing data is compared with the current data to be published in each namespace, the target current data to be published in each namespace that is the same as the target gateway publishing data is identified, and the target gateway publishing data corresponding to the target current data to be published is used as the target gateway publishing data already existing in the configuration center.

[0025] Optionally, in the case that the target gateway fails to publish data, publishing the data through the backed-up target gateway and rolling back the namespace where the publishing fails include:

[0026] In the case where there is a target gateway publishing data publishing failure, identifying the target cluster corresponding to each target gateway publishing data that failed to be published, and the target namespace of each target cluster;

[0027] Query the first data that has been successfully published in each target cluster and the second data that has been successfully published in each target namespace, and based on the backed-up target gateway publishing data of each target cluster, perform data rollback on the first data of each target cluster and the second data of the target namespace of each target cluster.

[0028] Optionally, storing all successfully published target gateway publication data into a database to complete the gateway publication data entry task includes:

[0029] Collecting the storage progress information of each successfully published target gateway publication data, and judging whether each successfully published target gateway publication data is successfully stored based on the storage progress information;

[0030] In the event that the successfully published target gateway publication data fails to be stored in the database, the successfully published target gateway publication data in the namespace of each cluster of the configuration center is rolled back through the backed up target gateway publication data, and the step of publishing each target gateway publication data in the namespace of each cluster is returned to execute, until all the successfully published target gateway publication data are successfully stored in the database, and the gateway publication data entry task is completed.

[0031] In a second aspect, the present application also provides a device for inputting gateway published data. The device comprises:

[0032] An acquisition module is used to acquire data product packages of multiple gateway release data transmitted by a third-party platform and current data to be released in the namespace of each cluster in the configuration center, and, if the integrity of the data product package is correct, convert the data structure of each gateway release data in the data product package to obtain each array structure data of the data product package;

[0033] An identification module, configured to perform deduplication processing on each array structure data of the data product package to obtain each target gateway publishing data of the data product package, and identify the target gateway publishing data already existing in the configuration center based on each target gateway publishing data and the current data to be published in each namespace;

[0034] A publishing module, used for backing up the existing target gateway publishing data, and publishing the target gateway publishing data in the namespace of each cluster respectively;

[0035] A rollback module is used to, in the event that the target gateway fails to publish data, publish the data through the backed-up target gateway, roll back the namespace where the publishing failed, and republish the target gateway data where the publishing failed, until all target gateway published data are successfully published;

[0036] The entry module is used to store all successfully published target gateway release data into the database to complete the gateway release data entry task.

[0037] Optionally, the device further comprises:

[0038] An extraction module, used to extract the data published by each gateway in the data product package, and respectively extract the data content of each gateway published data and the data identification information of each gateway published data;

[0039] A judgment module, used for judging, for each gateway published data content, whether the data content of the gateway published data is empty, and judging whether the data identification information of the gateway published data is correct;

[0040] A first judgment result module is used to mark the gateway-published data as abnormal gateway-published data when the data content of the gateway-published data is empty or the data identification information of the gateway-published data is incorrect, and feed back each abnormal gateway-published data to the staff client;

[0041] The second judgment result module is used to determine whether the integrity of the data product package is correct when the data content of the data published by the gateway is empty and the data identification information of the data published by the gateway is incorrect.

[0042] Optionally, the acquisition module is specifically used to:

[0043] Identify the data type of each gateway published data, and based on the data type of each gateway published data, identify the corresponding relationship between each gateway published data;

[0044] Based on the correspondence between the data published by each gateway, the data format of the data published by each gateway is converted into initial array structure data, and each initial array structure data is mapped and improved through a graph mapping strategy to obtain the array structure data corresponding to each gateway published data.

[0045] Optionally, the identification module is specifically used to:

[0046] Respectively identify the same data content in each array structure data, and perform deduplication processing on each of the same data contents to obtain the initial target gateway release data;

[0047] In the target database, it is determined whether there is data content identical to that of each initial target gateway published data, and the initial target gateway published data with the same data content is marked to obtain the marked gateway published data;

[0048] The marked gateway publishing data and the non-marked initial target gateway publishing data are used as the target gateway publishing data of the data product package.

[0049] Optionally, the identification module is specifically used to:

[0050] Each target gateway publishing data is compared with the current data to be published in each namespace, the target current data to be published in each namespace that is the same as the target gateway publishing data is identified, and the target gateway publishing data corresponding to the target current data to be published is used as the target gateway publishing data already existing in the configuration center.

[0051] Optionally, the rollback module is specifically used to:

[0052] In the case where there is a target gateway publishing data publishing failure, identifying the target cluster corresponding to each target gateway publishing data that failed to be published, and the target namespace of each target cluster;

[0053] Query the first data that has been successfully published in each target cluster and the second data that has been successfully published in each target namespace, and based on the backed-up target gateway publishing data of each target cluster, perform data rollback on the first data of each target cluster and the second data of the target namespace of each target cluster.

[0054] Optionally, the input module is specifically used for:

[0055] Collecting the storage progress information of each successfully published target gateway publication data, and judging whether each successfully published target gateway publication data is successfully stored based on the storage progress information;

[0056] In the event that the successfully published target gateway publication data fails to be stored in the database, the successfully published target gateway publication data in the namespace of each cluster of the configuration center is rolled back through the backed up target gateway publication data, and the step of publishing each target gateway publication data in the namespace of each cluster is returned to execute, until all the successfully published target gateway publication data are successfully stored in the database, and the gateway publication data entry task is completed.

[0057] In a third aspect, the present application provides a computer device, wherein the computer device comprises a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the steps of any one of the methods in the first aspect are implemented.

[0058] In a fourth aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the steps of any one of the methods in the first aspect are implemented.

[0059] In a fifth aspect, the present application provides a computer program product. The computer program product includes a computer program, and when the computer program is executed by a processor, the steps of any one of the methods in the first aspect are implemented.

[0060] The above-mentioned method, device, computer equipment and storage medium for inputting gateway published data obtain data product packages of multiple gateway published data transmitted by a third-party platform and the current data to be published in the namespace of each cluster in the configuration center, and when the integrity of the data product package is correct, the data structure of each gateway published data in the data product package is converted to obtain each array structure data of the data product package; the array structure data of the data product package is deduplicated to obtain each target gateway published data of the data product package, and based on each target gateway published data and each in the namespace The current data to be published, identify the target gateway publishing data that already exists in the configuration center; back up the existing target gateway publishing data, and publish the target gateway publishing data in the namespace of each cluster; in the case of a target gateway publishing data publishing failure, roll back the failed publishing namespace through the backed up target gateway publishing data, and republish the failed target gateway publishing data until all target gateway publishing data are successfully published; store all successfully published target gateway publishing data in the database to complete the gateway publishing data entry task. This solution converts the data product package into array structure data after performing occipital verification, so as to intuitively identify the same data content between each gateway publishing data, so as to perform data deduplication processing, and avoid the problem of low data entry efficiency caused by multiple queries on duplicate data and multiple entries of duplicate data. Then, based on the target gateway release data after deduplication, this solution queries the target gateway release data that already exists in the configuration center in the data of the namespaces of different clusters of the configuration center, and then backs it up. If there is a subsequent release failure or data entry failure, different clusters can perform data rollback operations to re-perform the gateway release and data entry process, avoiding the problem that the same batch of product data in multiple clusters cannot be rolled back in time once one of the cluster data goes wrong during manual operation, and the final consistency of the data cannot be ensured. Finally, this solution not only avoids inefficiency problems but also improves the accuracy of data entry by replacing manual operations with artificial intelligence. Then, by backing up the data that already exists in the configuration center in the gateway release data in advance, the data can be rolled back in time when the release fails or the entry fails, which improves the fault tolerance of data entry, thereby comprehensively improving the efficiency of gateway release data entry. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] Figure 1 A schematic diagram of a flow chart of a method for entering gateway published data in one embodiment;

[0062] Figure 2 A conversion diagram of data format conversion in one embodiment;

[0063] Figure 3 A schematic diagram of a flow chart of an example of entering data published by a gateway in one embodiment;

[0064] Figure 4 It is a structural block diagram of a device for inputting data published by a gateway in one embodiment;

[0065] Figure 5 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0066] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0067] The method for entering data published by a gateway provided in an embodiment of the present application can be applied to an application environment for publishing and entering data published by a gateway. It can be applied to a terminal, a server, or a system including a terminal and a server, and implemented through the interaction between the terminal and the server. Among them, the terminal may include but is not limited to various personal computers, laptops, tablet computers, etc. The terminal converts the data product package into array structure data after performing occipital verification, so that the same data content between the data published by each gateway can be intuitively identified to perform data deduplication processing, thereby avoiding the problem of low data entry efficiency caused by multiple queries on duplicate data and multiple entries of duplicate data. Then, based on the target gateway release data after deduplication, this solution queries the target gateway release data that already exists in the configuration center in the data of the namespaces of different clusters of the configuration center, and then backs it up. If there is a subsequent release failure or data entry failure, different clusters can perform data rollback operations to re-perform the gateway release and data entry process, avoiding the problem that the same batch of product data in multiple clusters cannot be rolled back in time once one of the cluster data goes wrong during manual operation, and the final consistency of the data cannot be ensured. Finally, this solution not only avoids inefficiency problems but also improves the accuracy of data entry by replacing manual operations with artificial intelligence. Then, by backing up the data that already exists in the configuration center in the gateway release data in advance, the data can be rolled back in time when the release fails or the entry fails, which improves the fault tolerance of data entry, thereby comprehensively improving the efficiency of gateway release data entry.

[0068] In one embodiment, Figure 1 As shown, a method for entering gateway published data is provided, which is described by taking the method applied to a terminal as an example, and includes the following steps:

[0069] Step S101, obtain the data product package of multiple gateway publishing data transmitted by the third-party platform, and the current data to be published in the namespace of each cluster in the configuration center, and when the integrity of the data product package is correct, convert the data structure of each gateway publishing data in the data product package to obtain the array structure data of the data product package.

[0070] In this embodiment, the terminal receives a data product package containing multiple gateway release data transmitted by a third-party platform, wherein the data product package is a DSL (Dynamic Script Language) product package containing application, API, and APIOperation configuration that needs to be released by the gateway. Before obtaining the data product package, the terminal verifies the serial number of the data product package to avoid multiple releases of the same batch of product packages. It mainly judges whether the current data product package to be released has been released based on the serial number. If it has been released, it directly feedbacks that the API product corresponding to the current serial number already exists. If it has not been released, it continues to execute. Then the terminal obtains the data in the namespace of each cluster of the configuration center to obtain the current data to be released, wherein the configuration center is the data configuration release center when the gateway is released under multiple clusters. The data configuration center includes multiple clusters, and the location where each cluster releases the data released by each gateway is the namespace (or namespace, i.e., namespace) of each cluster. The terminal then checks the integrity of the data product package and, if there is a problem with the integrity, it will inform the staff that the data product package needs to be resent. If the integrity of the data product package is correct, the terminal will convert the data structure of each gateway published data in the data product package to obtain the array structure data of the data product package. Figure 2 As shown, the data format of the data released by each gateway of the data product package is a tree structure format, and this solution needs to convert the data format of the data released by each gateway into an array structure format. The specific conversion process will be described in detail later.

[0071] Step S102, deduplicate the array structure data of the data product package to obtain the target gateway publishing data of the data product package, and identify the target gateway publishing data already existing in the configuration center based on the target gateway publishing data and the current data to be published in each namespace.

[0072] In this embodiment, the terminal performs deduplication processing on each array structure data of the data product package to obtain each target gateway publishing data of the data product package, and identifies the target gateway publishing data already existing in the configuration center based on each target gateway publishing data and the current data to be published in each namespace. Among them, the target gateway publishing data already existing in the configuration center is data that has not been published but has been stored in the namespace, and the data still needs to be published.

[0073] Step S103, backing up the existing target gateway publishing data, and publishing the target gateway publishing data in the namespace of each cluster respectively.

[0074] In this embodiment, the terminal backs up the existing target gateway publishing data, and publishes the target gateway publishing data in the namespace of each cluster. Specifically, based on the cluster information corresponding to the different target gateway publishing data, the terminal publishes the target gateway publishing data in the namespace corresponding to the cluster information (i.e., the namespace of each cluster). The correspondence between each target gateway publishing data and the cluster information is the preset correspondence in the data product package. Based on the preset correspondence, the terminal publishes the target gateway publishing data to the namespace corresponding to different clusters.

[0075] Step S104, when there is a target gateway publishing data publishing failure, the target gateway publishing data is backed up, the namespace where the publishing failed is rolled back, and the target gateway publishing data that failed is republished until all target gateway publishing data is successfully published.

[0076] In this embodiment, when the target gateway releases data that fails to be released, the terminal uses the backed-up target gateway release data to roll back the namespace that failed to be released, and re-releases the target gateway release data that failed to be released until all target gateway release data are successfully released. The data that needs to be rolled back includes the namespace data that has been released by the current cluster (i.e., the existing target gateway release data) and the cluster data that has been released in the current batch (i.e., the target gateway release data).

[0077] Step S105, storing all successfully published target gateway publication data into the database, completing the gateway publication data entry task.

[0078] In this embodiment, the terminal stores all successfully published target gateway publication data into the database, completing the gateway publication data entry task.

[0079] Based on the above scheme, by converting the data product package into array structure data after completing the occipital check, it is possible to intuitively identify the same data content between the data published by each gateway to perform data deduplication processing, avoiding multiple queries on duplicate data and multiple entries of duplicate data, which leads to low data entry efficiency. Then, based on the target gateway release data after deduplication, this solution queries the target gateway release data that already exists in the configuration center in the data of the namespaces of different clusters of the configuration center, and then backs it up. If there is a subsequent release failure or data entry failure, different clusters can perform data rollback operations to re-perform the gateway release and data entry process, avoiding the problem that the same batch of product data in multiple clusters cannot be rolled back in time once one of the cluster data goes wrong during manual operation, and the final consistency of the data cannot be ensured. Finally, this solution not only avoids inefficiency problems but also improves the accuracy of data entry by replacing manual operations with artificial intelligence. Then, by backing up the data that already exists in the configuration center in the gateway release data in advance, the data can be rolled back in time when the release fails or the entry fails, which improves the fault tolerance of data entry, thereby comprehensively improving the efficiency of gateway release data entry.

[0080] Optionally, when the integrity of the data product package is correct, the data structure of each gateway published data in the data product package is converted and processed to obtain each array structure data of the data product package, and it also includes: extracting each gateway published data in the data product package, and respectively extracting the data content of each gateway published data, and the data identification information of each gateway published data; for each gateway published data content, judging whether the data content of the gateway published data is empty, and judging whether the data identification information of the gateway published data is correct; when the data content of the gateway published data is empty, or the data identification information of the gateway published data is incorrect, marking the gateway published data as abnormal gateway published data, and feeding back each abnormal gateway published data to the staff client; when the data content of the gateway published data is empty, and the data identification information of the gateway published data is incorrect, determining that the integrity of the data product package is correct.

[0081] In this embodiment, the terminal extracts the data published by each gateway in the data product package, and extracts the data content of each gateway published data and the data identification information of each gateway published data respectively. Among them, the data content of each gateway published data, for example, since the API product package comes from a third-party platform, the gateway needs to check the data in the API product. First, the gateway backend will obtain and parse the API product package to obtain the data in the API product package. The API product package contains: application, API, operation, mapping and other information; each data identification information is the inspection information of each gateway published data, and the inspection information includes checking the application version number information, application paso (indicating the project group to which this application belongs) information, API version number information, API usage scope information, operation url information, operation method information, operation protocol type, etc.

[0082] For each gateway-published data content, the terminal determines whether the data content of the gateway-published data is empty, and determines whether the data identification information of the gateway-published data is correct. In the case where the data content of the gateway-published data is empty, or the data identification information of the gateway-published data is incorrect, the terminal marks the gateway-published data as abnormal gateway-published data, and feeds back each abnormal gateway-published data to the staff client. In the case where the data content of the gateway-published data is empty, and the data identification information of the gateway-published data is incorrect, the terminal determines that the integrity of the data product package is correct.

[0083] Based on the above scheme, this paper reduces the probability of publishing failure and data upload failure caused by abnormal gateway publishing data in the data product package by checking the integrity of the data product package, thereby improving the publishing efficiency, upload efficiency, publishing success rate and upload success rate of gateway publishing data.

[0084] Optionally, when the integrity of the data product package is correct, the data structure of the data published by each gateway in the data product package is converted and processed to obtain each array structure data of the data product package, including: identifying the data type of the data published by each gateway, and based on the data type of the data published by each gateway, identifying the correspondence between the data published by each gateway; based on the correspondence between the data published by each gateway, converting the data format of the data published by each gateway into initial array structure data, and mapping and improving each initial array structure data through a graph mapping strategy to obtain the array structure data corresponding to each gateway published data.

[0085] In this embodiment, the terminal identifies the data type of each gateway published data, and based on the data type of each gateway published data, identifies the corresponding relationship between each gateway published data, wherein the data type includes information types such as application type, API type, operation type, and mapping type.

[0086] Based on the correspondence between the data published by each gateway, the terminal converts the data format of the data published by each gateway into the initial array structure data, and through the graph mapping strategy, maps and completes each initial array structure data to obtain the array structure data corresponding to each gateway published data. Among them, the mapping relationship between each data includes the uuid of the API, the name of the API, and the version number of the API; an API contains multiple operations, and each API data contains a set of associated operations, which contains basic information: the uuid of the operation, the name of the operation, and the version number of the operation; the mapping data contains the operation information of one API mapped to the operation of another API. Among them, since the structure of the data is a tree structure, it is necessary to convert the tree structure data into an array structure data form according to the correspondence between application, API, and operation, and then complete its mapping relationship on the array data structure according to the mapping relationship, and finally convert its tree structure data into an array structure data form.

[0087] Based on the above scheme, by converting the format of the data published by each gateway, it is convenient to identify the same data content between the gateway published data of different data types, thereby reducing the number of queries in the database based on the data published by each gateway, and also reducing the duplication of data entry for the data published by each gateway.

[0088] Optionally, deduplication processing is performed on each array structure data of the data product package to obtain each target gateway publication data of the data product package, including: respectively identifying the same data content in each array structure data, and deduplication processing is performed on each same data content to obtain initial target gateway publication data; in the target database, determining whether there is data content that is the same as the data content of each initial target gateway publication data, and marking the initial target gateway publication data with the same data content to obtain marked gateway publication data; and using each marked gateway publication data and each unmarked initial target gateway publication data as each target gateway publication data of the data product package.

[0089] In this embodiment, the terminal identifies the same data content in each array structure data, and performs deduplication processing on each same data content to obtain the initial target gateway release data. In the target database, the terminal determines whether there is data content that is the same as the data content of each initial target gateway release data, and marks the initial target gateway release data with the same data content to obtain the marked gateway release data. Finally, the terminal uses each marked gateway release data and each unmarked initial target gateway release data as each target gateway release data of the data product package.

[0090] like Figure 2 As shown, data deduplication is mainly divided into two aspects. One is the deduplication of converted object data. From the converted data form in the above figure, it can be seen that the same data objects exist in the application column and the API column. Therefore, the gateway needs to deduplicate its duplicate data objects, mainly based on the unique identifier uuid of each object, and delete duplicate object data to reduce the number of database queries; the other is deduplication by comparison with the database. According to the uuid of each object, the database is queried. If the data already exists, it means that the current data to be published has been published and stored in the database, and the gateway does not need to publish it again. The existing data is set to null; when publishing the storage configuration later, if it is null, it will be skipped directly; if the data does not exist, the gateway needs to publish it into the database.

[0091] Based on the above solution, the number of queries in the database based on the data released by each gateway is reduced through data deduplication processing, and the duplication of data entry for the data released by each gateway is also reduced, thereby comprehensively improving the data publishing and entry efficiency of the gateway released data.

[0092] Optionally, based on the target gateway publication data and the current data to be published in each namespace, the target gateway publication data already existing in the configuration center is identified, including: performing data comparison processing on each target gateway publication data and the current data to be published in each namespace, identifying the target current data to be published in each namespace that is identical to the target gateway publication data, and using the target gateway publication data corresponding to the target current data to be published as the target gateway publication data already existing in the configuration center.

[0093] In this embodiment, the terminal compares each target gateway publishing data with the current data to be published in each namespace, identifies the target current data to be published in each namespace that is the same as the target gateway publishing data, and uses the target gateway publishing data corresponding to the target current data to be published as the target gateway publishing data already existing in the configuration center.

[0094] Based on the above solution, by identifying the target gateway publishing data that already exists in the configuration center, data can be backed up in advance, so that in the event of subsequent publishing failures and upload failures, data can be technically rolled back to avoid inconsistencies between the final stored data and the published data.

[0095] Optionally, in the event that a target gateway fails to publish data, the target gateway publishes the data through the backed-up target gateway, and the namespace where the publishing failed is rolled back, including: in the event that a target gateway fails to publish data, identifying the target clusters corresponding to each target gateway publish data that failed to publish, and the target namespaces of each target cluster; querying the first data that has been successfully published in each target cluster, and the second data that has been successfully published in each target namespace, and based on the backed-up target gateway publish data of each target cluster, rolling back the first data of each target cluster, and the second data of the target namespace of each target cluster.

[0096] In this embodiment, when there is a failure in publishing target gateway data, the terminal identifies the target cluster corresponding to each target gateway publishing data that failed to be published, and the target namespace of each target cluster. Then, the terminal queries the first data that has been successfully published in each target cluster, and the second data that has been successfully published in each target namespace, and based on the backed-up target gateway publishing data of each target cluster, the terminal performs data rollback processing on the first data of each target cluster and the second data of the target namespace of each target cluster. Among them, the first data is the namespace data that has been published by the current cluster (that is, the existing target gateway publishing data), and the second data is the cluster data that has been published in the current batch (that is, the target gateway publishing data).

[0097] Based on the above solution, by rolling back the data that failed to be published separately, it is ensured that the data published by each target gateway can be republished, thereby improving the fault tolerance rate of the network management data release. In addition, rolling back the data of a specific cluster and a specific namespace avoids interference with the release of data in all clusters, thereby improving the publishing and uploading efficiency of the gateway published data.

[0098] Optionally, all successfully published target gateway release data are stored in the database to complete the gateway release data entry task, including: collecting the storage progress information of each successfully published target gateway release data, and judging whether each successfully published target gateway release data is successfully stored based on the storage progress information; in the event that the storage of successfully published target gateway release data fails, the target gateway release data that has been successfully published in the namespace of each cluster of the configuration center is rolled back through the backed up target gateway release data, and the step of publishing each target gateway release data in the namespace of each cluster is returned to execute, until all successfully published target gateway release data are successfully stored, the gateway release data entry task is completed.

[0099] In this embodiment, the terminal collects the storage progress information of each successfully released target gateway release data, and based on the storage progress information, determines whether each successfully released target gateway release data is successfully stored. Then, in the case that the storage of the successfully released target gateway release data fails, the target gateway release data that has been successfully released in the namespace of each cluster of the configuration center is rolled back through the backed up target gateway release data, and returns to execute the step of publishing each target gateway release data in the namespace of each cluster, until all the successfully released target gateway release data are successfully stored, and the gateway release data entry task is completed.

[0100] Based on the above solution, through the rollback process, it is ensured that the stored data is consistent with the published data and both can be successfully stored, which improves the storage accuracy of the stored data.

[0101] This application also provides an example of entering gateway published data, such as Figure 3 As shown, the specific processing process includes the following steps:

[0102] Step S301, obtaining data product packages of multiple gateway release data transmitted by a third-party platform and current data to be released in the namespace of each cluster in the configuration center.

[0103] Step S302: extract the data published by each gateway in the data product package, and respectively extract the data content of each gateway published data and the data identification information of each gateway published data.

[0104] Step S303: for each gateway published data content, determine whether the data content of the gateway published data is empty, and determine whether the data identification information of the gateway published data is correct.

[0105] Step S304, when the data content of the gateway published data is empty or the data identification information of the gateway published data is incorrect, the gateway published data is marked as abnormal gateway published data, and each abnormal gateway published data is fed back to the staff client.

[0106] Step S305, when the data content of the data published by the gateway is empty and the data identification information of the data published by the gateway is incorrect, determining that the integrity of the data product package is correct.

[0107] Step S306, when the integrity of the data product package is correct, identify the data type of the data published by each gateway, and based on the data type of the data published by each gateway, identify the corresponding relationship between the data published by each gateway.

[0108] Step S307, based on the correspondence between the data published by each gateway, the data format of the data published by each gateway is converted into initial array structure data, and each initial array structure data is mapped and improved through a graph mapping strategy to obtain the array structure data corresponding to each gateway published data.

[0109] Step S308, respectively identifying the same data content in each array structure data, and performing deduplication processing on each same data content to obtain the initial target gateway publishing data.

[0110] Step S309: in the target database, determine whether there is data content identical to that of each initial target gateway published data, and mark the initial target gateway published data with identical data content to obtain marked gateway published data.

[0111] Step S310: using each marked gateway publishing data and each unmarked initial target gateway publishing data as each target gateway publishing data of the data product package.

[0112] Step S311, respectively compare and process the target gateway publishing data with the current data to be published in each namespace, identify the target current data to be published in each namespace that is the same as the target gateway publishing data, and use the target gateway publishing data corresponding to the target current data to be published as the target gateway publishing data already existing in the configuration center.

[0113] Step S312: back up the existing target gateway publishing data, and publish the target gateway publishing data in the namespace of each cluster respectively.

[0114] Step S313: in the case where there is a target gateway publishing data publishing failure, identifying the target cluster corresponding to each target gateway publishing data that failed to be published, and the target namespace of each target cluster.

[0115] Step S314, query the first data that has been successfully published in each target cluster and the second data that has been successfully published in each target namespace, and based on the backed up target gateway publishing data of each target cluster, perform data rollback processing on the first data of each target cluster and the second data of the target namespace of each target cluster.

[0116] Step S315 , collecting the storage progress information of each successfully published target gateway publication data, and judging whether each successfully published target gateway publication data is successfully stored based on the storage progress information.

[0117] Step S316, in the event that the successfully published target gateway publication data fails to be stored in the database, the target gateway publication data that has been successfully published in the namespace of each cluster of the configuration center is rolled back through the backed up target gateway publication data, and the step of publishing each target gateway publication data in the namespace of each cluster is returned to execute, until all the successfully published target gateway publication data are successfully stored in the database, and the gateway publication data entry task is completed.

[0118] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0119] Based on the same inventive concept, the embodiment of the present application also provides a device for entering gateway published data for implementing the method for entering gateway published data involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in one or more embodiments of the device for entering gateway published data provided below can refer to the limitations of the method for entering gateway published data above, and will not be repeated here.

[0120] In one embodiment, Figure 4 As shown, a device for entering data released by a gateway is provided, including: an acquisition module 410, an identification module 420, a release module 430, a rollback module 440 and an entry module 450, wherein:

[0121] The acquisition module 410 is used to acquire data product packages of multiple gateway release data transmitted by the third-party platform and the current data to be released in the namespace of each cluster in the configuration center, and when the integrity of the data product package is correct, convert the data structure of each gateway release data in the data product package to obtain each array structure data of the data product package;

[0122] The identification module 420 is used to perform deduplication processing on each array structure data of the data product package to obtain each target gateway publishing data of the data product package, and identify the target gateway publishing data already existing in the configuration center based on each target gateway publishing data and the current data to be published in each namespace;

[0123] The publishing module 430 is used to back up the existing target gateway publishing data and publish the target gateway publishing data in the namespace of each cluster respectively;

[0124] The rollback module 440 is used to, in the case where the target gateway release data fails to be released, roll back the namespace where the release failed through the backed up target gateway release data, and re-release the target gateway release data where the release failed, until all target gateway release data are successfully released;

[0125] The input module 450 is used to store all successfully published target gateway published data into the database, completing the gateway published data input task.

[0126] Optionally, the device further comprises:

[0127] An extraction module, used to extract the data published by each gateway in the data product package, and respectively extract the data content of each gateway published data and the data identification information of each gateway published data;

[0128] A judgment module, used for judging, for each gateway published data content, whether the data content of the gateway published data is empty, and judging whether the data identification information of the gateway published data is correct;

[0129] A first judgment result module is used to mark the gateway-published data as abnormal gateway-published data when the data content of the gateway-published data is empty or the data identification information of the gateway-published data is incorrect, and feed back each abnormal gateway-published data to the staff client;

[0130] The second judgment result module is used to determine whether the integrity of the data product package is correct when the data content of the data published by the gateway is empty and the data identification information of the data published by the gateway is incorrect.

[0131] Optionally, the acquisition module 410 is specifically configured to:

[0132] Identify the data type of each gateway published data, and based on the data type of each gateway published data, identify the corresponding relationship between each gateway published data;

[0133] Based on the correspondence between the data published by each gateway, the data format of the data published by each gateway is converted into initial array structure data, and each initial array structure data is mapped and improved through a graph mapping strategy to obtain the array structure data corresponding to each gateway published data.

[0134] Optionally, the identification module 420 is specifically configured to:

[0135] Respectively identify the same data content in each array structure data, and perform deduplication processing on each of the same data contents to obtain the initial target gateway release data;

[0136] In the target database, it is determined whether there is data content identical to that of each initial target gateway published data, and the initial target gateway published data with the same data content is marked to obtain the marked gateway published data;

[0137] The marked gateway publishing data and the non-marked initial target gateway publishing data are used as the target gateway publishing data of the data product package.

[0138] Optionally, the identification module 420 is specifically configured to:

[0139] Each target gateway publishing data is compared with the current data to be published in each namespace, the target current data to be published in each namespace that is the same as the target gateway publishing data is identified, and the target gateway publishing data corresponding to the target current data to be published is used as the target gateway publishing data already existing in the configuration center.

[0140] Optionally, the rollback module 440 is specifically configured to:

[0141] In the case where there is a target gateway publishing data publishing failure, identifying the target cluster corresponding to each target gateway publishing data that failed to be published, and the target namespace of each target cluster;

[0142] Query the first data that has been successfully published in each target cluster and the second data that has been successfully published in each target namespace, and based on the backed-up target gateway publishing data of each target cluster, perform data rollback on the first data of each target cluster and the second data of the target namespace of each target cluster.

[0143] Optionally, the input module 450 is specifically used for:

[0144] Collecting the storage progress information of each successfully published target gateway publication data, and judging whether each successfully published target gateway publication data is successfully stored based on the storage progress information;

[0145] In the event that the successfully published target gateway publication data fails to be stored in the database, the successfully published target gateway publication data in the namespace of each cluster of the configuration center is rolled back through the backed up target gateway publication data, and the step of publishing each target gateway publication data in the namespace of each cluster is returned to execute, until all the successfully published target gateway publication data are successfully stored in the database, and the gateway publication data entry task is completed.

[0146] Each module in the above-mentioned gateway data entry device can be implemented in whole or in part by software, hardware and a combination thereof. Each of the above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory in the computer device in the form of software, so that the processor can call and execute the operations corresponding to each of the above modules.

[0147] In one embodiment, a computer device is provided. The computer device may be a terminal, and its internal structure diagram may be as follows: Figure 5 As shown. The computer device includes a processor, a memory, a communication interface, a display screen and an input device connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be achieved through WIFI, a mobile cellular network, NFC (near field communication) or other technologies. When the computer program is executed by the processor, a method for entering data released by a gateway is implemented. The display screen of the computer device can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covered on the display screen, or a button, trackball or touchpad set on the computer device housing, or an external keyboard, touchpad or mouse, etc.

[0148] Those skilled in the art will understand that Figure 5The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0149] In one embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the computer program, the steps of any one of the methods in the first aspect are implemented.

[0150] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps of any one of the methods in the first aspect are implemented.

[0151] In one embodiment, a computer program product is provided, comprising a computer program, which, when executed by a processor, implements the steps of any one of the methods in the first aspect.

[0152] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties.

[0153] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but are not limited to this.

[0154] The technical features of the above embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0155] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A method for entering gateway published data, characterized in that: The method comprises: Obtain a data product package of multiple gateway release data transmitted by a third-party platform, and the current data to be released in the namespace of each cluster in the configuration center, and if the integrity of the data product package is correct, convert the data structure of each gateway release data in the data product package to obtain each array structure data of the data product package; Respectively identify the same data content in each array structure data, and perform deduplication processing on each of the same data contents to obtain the initial target gateway release data; In the target database, it is determined whether there is data content identical to that of each initial target gateway published data, and the initial target gateway published data with the same data content is marked to obtain the marked gateway published data; Using the marked gateway publishing data and the non-marked initial target gateway publishing data as the target gateway publishing data of the data product package; Based on the target gateway publishing data and the current data to be published in each namespace, identifying the target gateway publishing data already existing in the configuration center; Backing up the existing target gateway publishing data, and publishing the target gateway publishing data in the namespace of each cluster respectively; In the case where there is a target gateway publishing data publishing failure, identifying the target cluster corresponding to each target gateway publishing data that failed to be published, and the target namespace of each target cluster; Query the first data that has been successfully published in each target cluster and the second data that has been successfully published in each target namespace, and based on the backed-up target gateway publishing data of each target cluster, perform data rollback processing on the first data of each target cluster and the second data of the target namespace of each target cluster; Republishing the target gateway published data that failed to be published until all target gateway published data are successfully published; Store all successfully published target gateway publishing data into the database to complete the gateway publishing data entry task.

2. The method according to claim 1, characterized in that In the case where the integrity of the data product package is correct, before converting the data structure of each gateway publishing data in the data product package to obtain each array structure data of the data product package, the method further includes: Extracting each gateway published data in the data product package, and respectively extracting the data content of each gateway published data and the data identification information of each gateway published data; For each gateway published data content, determine whether the data content of the gateway published data is empty, and determine whether the data identification information of the gateway published data is correct; In the case where the data content of the gateway published data is empty or the data identification information of the gateway published data is incorrect, the gateway published data is marked as abnormal gateway published data, and each abnormal gateway published data is fed back to the staff client; In the case that the data content of the data published by the gateway is empty and the data identification information of the data published by the gateway is incorrect, it is determined that the integrity of the data product package is correct.

3. The method according to claim 1, characterized in that When the integrity of the data product package is correct, the data structure of each gateway publishing data in the data product package is converted to obtain each array structure data of the data product package, including: Identify the data type of each gateway published data, and based on the data type of each gateway published data, identify the corresponding relationship between each gateway published data; Based on the correspondence between the data published by each gateway, the data format of the data published by each gateway is converted into initial array structure data, and each initial array structure data is mapped and improved through a graph mapping strategy to obtain the array structure data corresponding to each gateway published data.

4. The method according to claim 1, characterized in that: The identifying the target gateway publishing data existing in the configuration center based on the target gateway publishing data and the current data to be published in each namespace includes: Each target gateway publishing data is compared with the current data to be published in each namespace, the target current data to be published in each namespace that is the same as the target gateway publishing data is identified, and the target gateway publishing data corresponding to the target current data to be published is used as the target gateway publishing data already existing in the configuration center.

5. The method according to claim 1, characterized in that: The step of storing all successfully published target gateway publication data into a database and completing the gateway publication data entry task includes: Collecting the storage progress information of each successfully published target gateway publication data, and judging whether each successfully published target gateway publication data is successfully stored based on the storage progress information; In the event that the successfully published target gateway publication data fails to be stored in the database, the successfully published target gateway publication data in the namespace of each cluster of the configuration center is rolled back through the backed up target gateway publication data, and the step of publishing each target gateway publication data in the namespace of each cluster is returned to execute, until all the successfully published target gateway publication data are successfully stored in the database, and the gateway publication data entry task is completed.

6. A device for inputting data released by a gateway, characterized in that: The device comprises: An acquisition module is used to acquire data product packages of multiple gateway release data transmitted by a third-party platform and current data to be released in the namespace of each cluster in the configuration center, and, if the integrity of the data product package is correct, convert the data structure of each gateway release data in the data product package to obtain each array structure data of the data product package; The identification module is used to respectively identify the same data content in each array structure data, and perform deduplication processing on each of the same data content to obtain the initial target gateway release data; in the target database, determine whether there is data content that is the same as the data content of each initial target gateway release data, and mark the initial target gateway release data with the same data content to obtain the marked gateway release data; use each of the marked gateway release data and each non-marked initial target gateway release data as each target gateway release data of the data product package; based on each of the target gateway release data and the current data to be released in each of the namespaces, identify the target gateway release data that already exists in the configuration center; A publishing module, used for backing up the existing target gateway publishing data, and publishing the target gateway publishing data in the namespace of each cluster respectively; A rollback module is used to, in the case where there is a target gateway publishing data publishing failure, identify the target cluster corresponding to each target gateway publishing data that failed to be published, and the target namespace of each target cluster; query the first data that has been successfully published in each target cluster, and the second data that has been successfully published in each target namespace, and based on the backed up target gateway publishing data of each target cluster, perform data rollback processing on the first data of each target cluster and the second data of the target namespace of each target cluster; republish the target gateway publishing data that failed to be published until all target gateway publishing data are successfully published; The entry module is used to store all successfully published target gateway release data into the database to complete the gateway release data entry task.

7. The device according to claim 6, characterized in that The device also includes: An extraction module, used to extract the data published by each gateway in the data product package, and respectively extract the data content of each gateway published data and the data identification information of each gateway published data; A judgment module, used for judging, for each gateway published data content, whether the data content of the gateway published data is empty, and judging whether the data identification information of the gateway published data is correct; A first judgment result module is used to mark the gateway-published data as abnormal gateway-published data when the data content of the gateway-published data is empty or the data identification information of the gateway-published data is incorrect, and feed back each abnormal gateway-published data to the staff client; The second judgment result module is used to determine whether the integrity of the data product package is correct when the data content of the data published by the gateway is empty and the data identification information of the data published by the gateway is incorrect.

8. The device according to claim 6, characterized in that The acquisition module is specifically used for: Identify the data type of each gateway published data, and based on the data type of each gateway published data, identify the corresponding relationship between each gateway published data; Based on the correspondence between the data published by each gateway, the data format of the data published by each gateway is converted into initial array structure data, and each initial array structure data is mapped and improved through a graph mapping strategy to obtain the array structure data corresponding to each gateway published data.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 5 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 5 are implemented.

Images