Method and device for determining fault in data, electronic equipment and storage medium

By querying matching data parameter information in the data platform and using component identification information, the abnormal components are quickly positioned, and the problem of difficulty and time-consuming troubleshooting in the data platform is solved, and the troubleshooting efficiency is improved.

CN119003216APending Publication Date: 2024-11-22CHINA TELECOM CORP LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202410985314.5
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-22
Publication Date
2024-11-22

AI Technical Summary

Technical Problem

The troubleshooting of data middle platform is difficult and time-consuming, and it is difficult for the existing technology to quickly locate abnormal components.

Method used

By obtaining query requests in the data, using the stored correspondence relationship to query the matching data parameter information, determine the abnormal component based on the identification of the data, and narrow the scope of fault location.

Benefits of technology

Reduces the difficulty and time-consuming of fault location and improves the efficiency of troubleshooting.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119003216A_ABST
    Figure CN119003216A_ABST
Patent Text Reader

Abstract

The embodiment of the invention provides a method and device for determining a fault in data, electronic equipment and a storage medium, and the method comprises the steps: obtaining a query request of a first service, the query request comprising a query keyword; in the data medium table, first data corresponding to the data parameter information matched with the query keyword is queried according to a stored first corresponding relation, and the first corresponding relation comprises the corresponding relation among the target data, the data parameter information of the target data and the identifier of the target data; the identifier of the target data comprises identifier information of a component for processing the target data in the data-in-data table; and under the condition that the first data is abnormal, according to the identifier of the first data, determining an abnormal component in the components for processing the first data in the data-in-table. Therefore, according to the embodiment of the invention, the problems of relatively high difficulty and relatively long time consumption in troubleshooting of the data medium table in the prior art can be solved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and in particular to a method, device, electronic device and storage medium for determining a data center fault. Background Art

[0002] The data middle platform is a platform that integrates internal and external data resources of the enterprise and realizes unified management, efficient utilization and value mining of data through technical means such as data governance, data integration, and data services.

[0003] Among them, the data center needs to do four aspects of work, namely "collection", "storage", "connection" and "use". "Collection" is to collect useful data such as business data, log data, user behavior data of various business lines; "storage" is to store data in a more scientific way, generally using a three-layer modeling method to form data assets from the collected data; "connection" is to connect user behavior data (such as e-commerce users' browsing and clicking behaviors) and user business data (such as users' payment business data); "use" is to empower business personnel to make decisions based on the connected data, so that data can feed back to the business.

[0004] It can be seen that the data center is a data management platform located between the front desk and the back desk. It provides data support for the front desk and data services for the back desk. If the operation of the data center fails, it will affect the operation of the front desk and the back desk. Therefore, the operation of the data center is very important.

[0005] Among them, currently when a fault occurs in the data middle platform, it is necessary to manually troubleshoot the fault from each component (i.e., module) included in the data middle platform in turn; however, the data middle platform includes a large number of components, which makes troubleshooting difficult and time-consuming. Summary of the invention

[0006] The embodiments of the present application provide a data middle station fault determination method, device, electronic device and storage medium to solve the problem that troubleshooting of the data middle station in the prior art is difficult and time-consuming.

[0007] In a first aspect, an embodiment of the present application provides a method for determining a data center fault, the method comprising:

[0008] Obtaining a query request for a first service, wherein the query request includes a query keyword;

[0009] In the data center, according to the stored first correspondence, query the first data corresponding to the data parameter information matching the query keyword, wherein the first correspondence includes the correspondence between the target data, the data parameter information of the target data, and the identifier of the target data, and the identifier of the target data includes the identifier information of the component in the data center that processes the target data;

[0010] In the case that the first data is abnormal, the abnormal component among the components processing the first data in the data center is determined according to the identifier of the first data.

[0011] Optionally, the query request also includes a service type of the first service;

[0012] In the data center, searching for first data corresponding to the data parameter information matching the query keyword according to the stored first corresponding relationship includes:

[0013] In a case where the service type of the first service belongs to a first predetermined type, in the data center, first data corresponding to the data parameter information matching the query keyword is searched according to the first corresponding relationship.

[0014] Optionally, the method further comprises:

[0015] When the business type of the first business belongs to the second predetermined type, data corresponding to the data parameter information matching the query keyword is queried in the data bottom layer according to a stored second correspondence relationship, wherein the second correspondence relationship includes a correspondence between source data and data parameter information of the source data.

[0016] Optionally, the method further comprises:

[0017] When there is no abnormality in the first data, obtaining a first value of a target indicator of the first business according to the first data;

[0018] Compare the first value with the second value, wherein the second value is the value of the target indicator determined by the data of the first business queried last time according to the query keyword;

[0019] When the absolute value of the difference between the first value and the second value exceeds a target threshold, obtaining source data generating the first data in the data bottom layer;

[0020] Processing the source data through a component identified by first identification information to obtain second data, wherein the first identification information includes identification information of the component included in the identification of the first data;

[0021] Obtaining a third value of the target indicator according to the second data;

[0022] Comparing the third value with the second value;

[0023] When the absolute value of the difference between the third value and the second value still exceeds the target threshold, a first prompt operation is performed, wherein the first prompt operation is used to indicate that an abnormal component exists in the components identified by the first identification information.

[0024] Optionally, the method further comprises:

[0025] In the case that there is no abnormality in the first data, a new identifier of the first data is generated based on the query keyword, wherein the new identifier of the first data includes the query keyword.

[0026] In a second aspect, an embodiment of the present application further provides a data center fault determination device, the device comprising:

[0027] A first acquisition module, configured to acquire a query request for a first service, wherein the query request includes a query keyword;

[0028] A first query module is used to query, in the data center, first data corresponding to the data parameter information matching the query keyword according to a stored first correspondence, wherein the first correspondence includes a correspondence between target data, data parameter information of the target data, and an identifier of the target data, and the identifier of the target data includes identifier information of a component in the data center that processes the target data;

[0029] The first processing module is used to determine, when an abnormality exists in the first data, an abnormal component among the components in the data processing station according to an identifier of the first data.

[0030] Optionally, the query request further includes a service type of the first service; and the first query module is specifically configured to:

[0031] In the data center, searching for first data corresponding to the data parameter information matching the query keyword according to the stored first corresponding relationship includes:

[0032] In a case where the service type of the first service belongs to a first predetermined type, in the data center, first data corresponding to the data parameter information matching the query keyword is searched according to the first corresponding relationship.

[0033] Optionally, the device further comprises:

[0034] A second query module is used to query data corresponding to data parameter information matching the query keyword based on a stored second correspondence in the data bottom layer when the business type of the first business belongs to a second predetermined type, wherein the second correspondence includes a correspondence between source data and data parameter information of the source data.

[0035] Optionally, the device further includes: a second processing module, configured to perform the following process:

[0036] When there is no abnormality in the first data, obtaining a first value of a target indicator of the first business according to the first data;

[0037] Compare the first value with the second value, wherein the second value is the value of the target indicator determined by the data of the first business queried last time according to the query keyword;

[0038] When the absolute value of the difference between the first value and the second value exceeds a target threshold, obtaining source data generating the first data in the data bottom layer;

[0039] Processing the source data through a component identified by first identification information to obtain second data, wherein the first identification information includes identification information of the component included in the identification of the first data;

[0040] Obtaining a third value of the target indicator according to the second data;

[0041] Comparing the third value with the second value;

[0042] When the absolute value of the difference between the third value and the second value still exceeds the target threshold, a first prompt operation is performed, wherein the first prompt operation is used to indicate that an abnormal component exists in the components identified by the first identification information.

[0043] Optionally, the device further comprises:

[0044] The identification updating module is used to generate a new identification of the first data based on the query keyword when there is no abnormality in the first data, wherein the new identification of the first data includes the query keyword.

[0045] In a third aspect, an embodiment of the present application provides an electronic device, including a memory, a transceiver, and a processor:

[0046] A memory for storing computer programs; a transceiver for sending and receiving data under the control of the processor; and a processor for reading the computer program in the memory and executing the method for determining a data mid-stage failure as described in the first aspect above.

[0047] In a fourth aspect, an embodiment of the present application provides a readable storage medium, on which a program or instruction is stored. When the program or instruction is executed by a processor, the method for determining a data middle station failure described in the first aspect is implemented.

[0048] In an embodiment of the present application, when a query request for a first business is obtained, the first data corresponding to the data parameter information matching the query keyword included in the query request can be queried in the data center according to a stored first correspondence relationship, so that when an abnormality exists in the first data, the abnormal component in the component that processes the first data in the data center can be determined according to an identifier of the first data; wherein the first correspondence relationship includes a correspondence between the target data, the data parameter information of the target data, and the identifier of the target data, and the identifier of the target data includes the identifier information of the component that processes the target data in the data center.

[0049] It can be seen that in the embodiment of the present application, the target data processed by the data center can obtain the identification of the target data according to the identification information of the component that processes the target data in the data center, and establish a first correspondence between the target data, the data parameter information of the target data, and the identification of the target data; in this way, the relevant data of the business can be queried based on the first correspondence, and if the queried data is abnormal, the faulty component in the data center can be located according to the identification of the data (that is, the identification information of the component that processes the data in the data center). It can be seen that in the embodiment of the present application, the data query process in the data center can be utilized, and combined with the above-mentioned identification of the data, the scope of fault location can be narrowed, thereby reducing the difficulty of fault location and reducing time consumption. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the description of the embodiments of the present application will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0051] Figure 1 A flowchart of a method for determining a data center failure provided in an embodiment of the present application;

[0052] Figure 2 This is a schematic diagram of the architecture of the data center in the embodiment of the present application;

[0053] Figure 3 A structural block diagram of a data center fault determination device provided in an embodiment of the present application;

[0054] Figure 4A structural block diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0055] In the embodiments of the present application, the term "and / or" describes the association relationship of the associated objects, indicating that three relationships may exist. For example, A and / or B may represent: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the associated objects before and after are in an "or" relationship.

[0056] In the embodiments of the present application, the term "plurality" refers to two or more than two, and other quantifiers are similar.

[0057] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0058] The embodiments of the present application provide a data middle station fault determination method, device, electronic device and storage medium to solve the problem in the prior art that troubleshooting of the data middle station is difficult and time-consuming.

[0059] Among them, the method and the device are based on the same application concept. Since the method and the device solve the problem in a similar principle, the implementation of the device and the method can refer to each other, and the repeated parts will not be repeated.

[0060] Figure 1 A schematic diagram of a data center fault determination method provided in an embodiment of the present application is shown. Figure 1 As shown, the method may include the following steps 101 to 103:

[0061] Step 101: Obtain a query request for a first service.

[0062] The query request includes a query keyword.

[0063] In addition, the user can enter the query keyword of the first service in the client corresponding to the data middle station, thereby sending the query request of the first service to the data middle station through the client.

[0064] Step 102: In the data center, according to the stored first correspondence, query the first data corresponding to the data parameter information matching the query keyword.

[0065] Among them, the first corresponding relationship includes the corresponding relationship between the target data, the data parameter information of the target data, and the identifier of the target data, and the identifier of the target data includes the identifier information of the component that processes the target data in the data center.

[0066] From this, it can be seen that in the process of data processing by the data middle platform through the components it includes, the identification information of the components used in the process can be recorded, so that after the data processing is completed, the first correspondence between the data, the data parameter information of the data, and the identification of the data can be stored.

[0067] In addition, in the first corresponding relationship, the corresponding relationship between the target data and the data parameter information of the target data is used for querying data, that is, the target data corresponding to the data parameter information matching the query keyword is the data of the first business to be searched (that is, the first data);

[0068] In the first correspondence, the correspondence between the target data and the identifier of the target data is used for fault location. That is, if there is an abnormality in the target data, it means that there is an abnormality in the component that processes the target data in the data center, so that the fault can be located in these components, which can reduce the difficulty of fault location and shorten the time of fault location.

[0069] It should be noted that the components in the data center can also be called modules.

[0070] In addition, if Figure 2 As shown, the data center includes a data operation system module, a data security management module, a data service system module, a data asset management module, a data system (i.e., data warehouse) module, a data aggregation module, and a data development module;

[0071] The data service system modules include: service creation module, application programming interface (API) gateway module, service authorization module, call management module, etc.

[0072] The data asset management module includes: data asset catalog module, metadata module, data quality module, data lineage module, data life cycle module, etc. Among them, metadata is data that describes data, also known as intermediary data or relay data, which mainly provides relevant information about the content, attributes, source, quality, etc. of the data, and supports the storage, retrieval, management and use of data;

[0073] The data system (i.e. data warehouse) module includes: original data layer, detailed data layer, data summary layer, and data application layer;

[0074] The data aggregation module includes: offline synchronization module, real-time access module, heterogeneous data source module, visual configuration module, etc.

[0075] The data development modules include: offline development module, real-time development module, algorithm development module, intelligent scheduling module, intelligent operation and maintenance module, monitoring and alarm module, etc.

[0076] It should be noted that Figure 2 MySQL in the table represents a relational database management system, Oracle represents a database for network computing environments, Hadoop represents an open source software framework that provides distributed storage and computing, Spark represents a fast and general computing engine designed for large-scale data processing, Flink represents an open source stream processing framework, and Kafka represents an open source stream processing platform.

[0077] It is understandable that the module for processing the target data may include one or more modules in the data center listed above, such as the detailed data layer, the data summary layer, etc. In this way, the identification of the target data includes the identification information of the modules in the data center involved in processing the target data.

[0078] For example, based on Figure 2 In the data center shown, the format of the target data identifier can be:

[0079] Flag = [data service system ID, data asset management ID, data system ID, data aggregation ID, data development ID, data security management ID, data operation system ID], where, in this format, if a module participates in processing the target data, the value of the corresponding field of the module can be 1; if a module does not participate in processing the target data, the value of the corresponding field of the module can be 0; for example, Flag = [0,1,1,1,0,0], the processing modules of the target data to which this Flag belongs include: data asset management module, data system module, and data aggregation module.

[0080] It is understandable that the identification information included in the Flag can also be further refined to the functional modules included in a certain module, for example, based on Figure 2In the data middle platform shown, the format of the target data identification can also be: Flag = [service creation ID, API gateway ID, service authorization ID, call management ID, data asset catalog ID, metadata ID, data quality ID, data lineage ID, data life cycle ID, original data layer ID, detailed data layer ID, data summary layer ID, data application layer ID, offline synchronization ID, real-time access ID, heterogeneous data source ID, visualization configuration ID, offline development ID, real-time development ID, algorithm development ID, intelligent scheduling ID, intelligent operation and maintenance ID, monitoring alarm ID, data security management ID, data operation system ID]; similarly, in this format, if a module participates in processing the target data, the value of the corresponding field of the module can be 1, and if a module does not participate in processing the target data, the value of the corresponding field of the module can be 0.

[0081] It should also be noted that the data processed by the data middle platform can be stored in the above-mentioned data system (i.e., data warehouse) module, and the above-mentioned first correspondence can be stored in the data system (i.e., data warehouse) module; in this case, in the data system (i.e., data warehouse) module in the data middle platform, according to the first correspondence, the data parameter information matching the query keyword is queried.

[0082] Step 103: When an abnormality exists in the first data, determine the abnormal component in the component processing the first data in the data center according to the identifier of the first data.

[0083] Among them, if the first data is garbled or has a null value (i.e., it does not include specific data content), it means that there is an abnormality in the first data; in this case, based on the identifier of the first data, it is possible to determine which components (i.e., modules) in the data center have processed the source data of the first data, so that the fault can be located in these modules.

[0084] In addition, according to the identifier of the first data, the abnormal component in the component processing the first data in the data processing station is determined, that is: the abnormal component is determined in the component identified by the identification information included in the identifier of the first data.

[0085] In addition, in step 103, for the component in the data center that processes the first data, the status and performance can be monitored through the monitoring system of the data center, so as to provide alarm information based on the status and performance, point out the specific location and cause of the problem, and help the administrator quickly locate the abnormal module; or, according to the operation log of the component in the data center that processes the first data, an alarm can be issued for the module with the log abnormality, so as to help the administrator quickly locate the abnormal module. It can be seen that in the embodiment of the present application, the scope of fault location is narrowed by the above steps 101 and 102, so that the speed of fault location can be improved.

[0086] It can be seen from the above steps 101 to 103 that in an embodiment of the present application, when a query request for the first business is obtained, the first data corresponding to the data parameter information matching the query keyword included in the query request can be queried in the data center according to the stored first correspondence relationship, so that when there is an abnormality in the first data, the abnormal component in the component that processes the first data in the data center is determined according to the identifier of the first data; wherein the first correspondence relationship includes the correspondence between the target data, the data parameter information of the target data, and the identifier of the target data, and the identifier of the target data includes the identifier information of the component that processes the target data in the data center.

[0087] It can be seen that in the embodiment of the present application, the target data processed by the data center can obtain the identification of the target data according to the identification information of the component that processes the target data in the data center, and establish a first correspondence between the target data, the data parameter information of the target data, and the identification of the target data; in this way, the relevant data of the business can be queried based on the first correspondence, and if the queried data is abnormal, the faulty component in the data center can be located according to the identification of the data (that is, the identification information of the component that processes the data in the data center). It can be seen that in the embodiment of the present application, the data query process in the data center can be utilized, and combined with the above-mentioned identification of the data, the scope of fault location can be narrowed, thereby reducing the difficulty of fault location and reducing time consumption.

[0088] Optionally, the query request also includes a business type of the first business; in the above step 102, the querying of the first data corresponding to the data parameter information matching the query keyword in the data center according to the stored first correspondence includes the following step A-1:

[0089] Step A-1: ​​When the service type of the first service belongs to the first predetermined type, in the data center, first data corresponding to the data parameter information matching the query keyword is searched according to the first corresponding relationship.

[0090] Among them, the first predetermined type can be a business with weaker timeliness, such as a business in which the delay in querying data is allowed to be greater than a first threshold; that is, in an embodiment of the present application, for a business with weaker timeliness, the relevant data of the business can be queried from the data center without having to query the data bottom layer, thereby alleviating the pressure on the data bottom layer.

[0091] It can be seen from this that in the embodiments of the present application, some businesses that need to query data from the data bottom layer can be separated according to the type of business; that is, the business data of some types of businesses (for example, businesses with weak timeliness) can be saved in a copy in the data center. In this way, these types of businesses can query data in the data center without having to query the data bottom layer, thereby alleviating the pressure on the data bottom layer.

[0092] Optionally, the method further comprises the following step B-1:

[0093] Step B-1: When the business type of the first business belongs to the second predetermined type, query the data corresponding to the data parameter information matching the query keyword in the data bottom layer according to the stored second correspondence relationship, wherein the second correspondence relationship includes the correspondence between the source data and the data parameter information of the source data.

[0094] Among them, the second predetermined type can be a business with strong timeliness, such as a business in which the delay requirement for querying data is less than or equal to the first threshold; that is, in an embodiment of the present application, for a business with strong timeliness, it is necessary to query the relevant data of the business from the data bottom layer, thereby reducing data delay and meeting the timeliness requirement of the business.

[0095] It can be seen from this that in an embodiment of the present application, some services that need to query data from the data bottom layer can be retained according to the type of business (for example, those with strong timeliness), so that these services can query relevant data from the data bottom layer, thereby meeting the needs of this type of business.

[0096] From the above, it can be seen that in the embodiment of the present application, it is possible to determine from which place in the data middle platform and the data bottom layer to query relevant data based on the business type of the first business.

[0097] It is understandable that if the business type of the first business belongs to the first predetermined type (i.e., the validity is weak), but the data of the first business is updated periodically, and when the query request of the first business is currently received, the update time has not yet arrived, in this case, the data most recently updated before the current time can be directly queried in the data center according to the query keyword;

[0098] Alternatively, the source data of the current first business can be queried in the data bottom layer according to the query keyword, and then processed by the relevant modules of the data middle station to obtain updated data; wherein, the module that processes the source data here can be the module indicated by the identification information included in the identification corresponding to the data parameter information matching the query keyword found in the data middle station.

[0099] In addition, the data stored in the data warehouse of the data middle platform includes publicly available data, and the data stored in the data bottom layer includes private data; if the private data of a certain business is allowed to be exposed, the data of the business can be queried in the data bottom layer; and if the private data of a certain business is not allowed to be exposed, the data of the business can be queried in the data middle platform.

[0100] For example, if the privacy data of region A (such as user information) is not allowed to be exposed, the operational sales data of region A can be queried in the data center; if the privacy data of region B is allowed to be exposed, the source data of region B can be queried in the data bottom layer, and the sales operation data of region B can be further obtained based on the source data.

[0101] In addition, Trino is an analysis engine for efficient distributed query of large amounts of data. It mainly has the following advantages: providing a unified query interface; based on memory computing, it can complete federated queries across different data sources; data governance, through Trino to extract, integrate and analyze heterogeneous data, break down data silos and improve data governance capabilities. Therefore, in an embodiment of the present application, a connector based on Trino can be used to query data from the data middle platform and the data bottom layer. That is, the connector provides Trino with an interface for accessing any data source between modules, greatly improving the transmission efficiency of data flow between different modules.

[0102] It should also be noted that the data in the data warehouse of the data middle platform can come from different databases. These data are stored in a unified format in the data warehouse, so that when data is retrieved from the data warehouse, the speed is faster; and the data warehouse of the data middle platform can include the original data that needs to be calculated and the data with calculated results. The original data and result data can be retrieved separately, which will be faster.

[0103] Optionally, the method further comprises the following steps C-1 to C-7:

[0104] Step C-1: when there is no abnormality in the first data, obtaining a first value of a target indicator of the first service according to the first data;

[0105] Step C-2: Compare the first value with the second value, wherein the second value is the value of the target indicator determined by the data of the first business queried last time according to the query keyword;

[0106] Step C-3: when the absolute value of the difference between the first value and the second value exceeds the target threshold, obtaining source data for generating the first data in the data bottom layer;

[0107] Step C-4: Processing the source data by the component identified by the first identification information to obtain second data, wherein the first identification information includes identification information of the component included in the identification of the first data;

[0108] Step C-5: Obtaining a third value of the target indicator according to the second data;

[0109] Step C-6: Compare the third value with the second value;

[0110] Step C-7: When the absolute value of the difference between the third value and the second value still exceeds the target threshold, perform a first prompt operation, wherein the first prompt operation is used to indicate that there are abnormal components in the components identified by the first identification information.

[0111] Among them, if there is no abnormality in the first data obtained from the data center, but the first data queried this time is significantly different from the data queried last time based on the same query keyword, it means that there is an abnormality in the generation process of the first data queried this time.

[0112] In addition, the first value of the target indicator can be calculated based on the first data of this query, and the second value of the target indicator can be calculated based on the data of the last query based on the same query keyword, so as to compare the first value with the second value to obtain the difference between the first data and the data of the last query based on the same query keyword; wherein, if the absolute value of the difference between the first value and the second value exceeds the target threshold, it means that the difference between the two is large.

[0113] In an embodiment of the present application, if there is a large gap between the first data and the data last queried based on the same query keyword, the source data for generating the first data can be obtained from the data bottom layer, and the same module involved in the process of generating the first data is used to process the source data to obtain regenerated second data; and then the second data is compared with the data last queried based on the same query keyword by the same method. If the gap between the two is still large, it means that there is a problem with the algorithm of the module involved in the process of generating the first data. In this case, the above-mentioned first prompt operation can be executed so that relevant personnel can check the relevant modules.

[0114] It can be understood that the performing of the first prompt operation may include at least one of the following:

[0115] Display prompt information in the client of the data center;

[0116] Send a notification message to the terminals of relevant personnel.

[0117] In order to facilitate the understanding of the above steps C-1 to C-7, the following example is given:

[0118] As a first example, 100 operation records of the first business were queried in the data center this time, of which 80 were of the first category, so the first category of operation records accounted for 80% this time; but last time, 100 operation records of the first business were queried in the data center, of which 5 were of the first category, so the first category of operation records accounted for 0.5% last time; it can be seen that the proportion of the first category of operation records this time is quite different from that of the first category of operation records last time;

[0119] In this case, it is necessary to re-acquire the source data of the first business and process it through the module that generates operation records to obtain 100 new operation records. If the new operation records include 81 first-category operation records, the proportion is 81%, which is also quite different from 0.5%. In this case, relevant personnel need to be prompted to troubleshoot the module that generates operation records.

[0120] Optionally, the method further comprises the following step D-1:

[0121] Step D-1: when there is no abnormality in the first data, generate a new identifier of the first data based on the query keyword, wherein the new identifier of the first data includes the query keyword.

[0122] It can be seen that when the first data is found according to the query keyword and there is no abnormality in the first data, a new identifier can be generated according to the current identifier of the first data, that is, the query keyword is added on the basis of the current identifier.

[0123] For example, if the current identifier of the first data is Flag = [data service system ID, data asset management ID, data system ID, data aggregation ID, data development ID, data security management ID, data operation system ID], then the new identifier is: Flag = [data service system ID, data asset management ID, data system ID, data aggregation ID, data development ID, data security management ID, data operation system ID, query keyword].

[0124] It is understandable that a new identifier may be generated for the first data after querying the first data once.

[0125] The above introduces the data middle platform fault determination method provided in the embodiment of the present application. The following will introduce the data middle platform fault determination device provided in the embodiment of the present application in conjunction with the accompanying drawings.

[0126] See also Figure 3 The embodiment of the present application also provides a data center fault determination device, the device comprising:

[0127] A first acquisition module 301 is used to acquire a query request for a first service, wherein the query request includes a query keyword;

[0128] A first query module 302 is used to query the first data corresponding to the data parameter information matching the query keyword in the data center according to the stored first correspondence, wherein the first correspondence includes the correspondence between the target data, the data parameter information of the target data, and the identifier of the target data, and the identifier of the target data includes the identifier information of the component in the data center that processes the target data;

[0129] The first processing module 303 is used to determine, when an abnormality exists in the first data, an abnormal component among the components in the data processing station that process the first data according to an identifier of the first data.

[0130] From the above, it can be seen that in an embodiment of the present application, when the first acquisition module 301 obtains a query request for the first business, the first query module 302 can be triggered to query the first data corresponding to the data parameter information matching the query keyword included in the query request in the data center according to the stored first correspondence relationship, so that the first processing module 303, when there is an abnormality in the first data, determines the abnormal component in the component that processes the first data in the data center according to the identifier of the first data; wherein the first correspondence relationship includes the correspondence between the target data, the data parameter information of the target data, and the identifier of the target data, and the identifier of the target data includes the identifier information of the component that processes the target data in the data center.

[0131] It can be seen that in the embodiment of the present application, the target data processed by the data center can obtain the identification of the target data according to the identification information of the component that processes the target data in the data center, and establish a first correspondence between the target data, the data parameter information of the target data, and the identification of the target data; in this way, the relevant data of the business can be queried based on the first correspondence, and if the queried data is abnormal, the faulty component in the data center can be located according to the identification of the data (that is, the identification information of the component that processes the data in the data center). It can be seen that in the embodiment of the present application, the data query process in the data center can be utilized, and combined with the above-mentioned identification of the data, the scope of fault location can be narrowed, thereby reducing the difficulty of fault location and reducing time consumption.

[0132] Optionally, the query request further includes a service type of the first service; and the first query module 302 is specifically configured to:

[0133] In the data center, searching for first data corresponding to the data parameter information matching the query keyword according to the stored first corresponding relationship includes:

[0134] In a case where the service type of the first service belongs to a first predetermined type, in the data center, first data corresponding to the data parameter information matching the query keyword is searched according to the first corresponding relationship.

[0135] Among them, the first predetermined type can be a business with weaker timeliness, such as a business in which the delay in querying data is allowed to be greater than a first threshold; that is, in an embodiment of the present application, for a business with weaker timeliness, the relevant data of the business can be queried from the data center without having to query the data bottom layer, thereby alleviating the pressure on the data bottom layer.

[0136] It can be seen from this that in the embodiments of the present application, some businesses that need to query data from the data bottom layer can be separated according to the type of business; that is, the business data of some types of businesses (for example, businesses with weak timeliness) can be saved in a copy in the data center. In this way, these types of businesses can query data in the data center without having to query the data bottom layer, thereby alleviating the pressure on the data bottom layer.

[0137] Optionally, the device further comprises:

[0138] A second query module is used to query data corresponding to data parameter information matching the query keyword based on a stored second correspondence in the data bottom layer when the business type of the first business belongs to a second predetermined type, wherein the second correspondence includes a correspondence between source data and data parameter information of the source data.

[0139] Among them, the second predetermined type can be a business with strong timeliness, such as a business in which the delay requirement for querying data is less than or equal to the first threshold; that is, in an embodiment of the present application, for a business with strong timeliness, it is necessary to query the relevant data of the business from the data bottom layer, thereby reducing data delay and meeting the timeliness requirement of the business.

[0140] It can be seen from this that in an embodiment of the present application, some services that need to query data from the data bottom layer can be retained according to the type of business (for example, those with strong timeliness), so that these services can query relevant data from the data bottom layer, thereby meeting the needs of this type of business.

[0141] Optionally, the device further includes: a second processing module, configured to perform the following process:

[0142] When there is no abnormality in the first data, obtaining a first value of a target indicator of the first business according to the first data;

[0143] Compare the first value with the second value, wherein the second value is the value of the target indicator determined by the data of the first business queried last time according to the query keyword;

[0144] When the absolute value of the difference between the first value and the second value exceeds a target threshold, obtaining source data for generating the first data in the data bottom layer;

[0145] Processing the source data through a component identified by first identification information to obtain second data, wherein the first identification information includes identification information of the component included in the identification of the first data;

[0146] Obtaining a third value of the target indicator according to the second data;

[0147] Comparing the third value with the second value;

[0148] When the absolute value of the difference between the third value and the second value still exceeds the target threshold, a first prompt operation is performed, wherein the first prompt operation is used to indicate that an abnormal component exists in the components identified by the first identification information.

[0149] Among them, if there is no abnormality in the first data obtained from the data center, but the first data queried this time is significantly different from the data queried last time based on the same query keyword, it means that there is an abnormality in the generation process of the first data queried this time.

[0150] In addition, the first value of the target indicator can be calculated based on the first data of this query, and the second value of the target indicator can be calculated based on the data of the last query based on the same query keyword, so as to compare the first value with the second value to obtain the difference between the first data and the data of the last query based on the same query keyword; wherein, if the absolute value of the difference between the first value and the second value exceeds the target threshold, it means that the difference between the two is large.

[0151] In an embodiment of the present application, if there is a large gap between the first data and the data last queried based on the same query keyword, the source data for generating the first data can be obtained from the data bottom layer, and the same module involved in the process of generating the first data is used to process the source data to obtain regenerated second data; and then the second data is compared with the data last queried based on the same query keyword by the same method. If the gap between the two is still large, it means that there is a problem with the algorithm of the module involved in the process of generating the first data. In this case, the above-mentioned first prompt operation can be executed so that relevant personnel can check the relevant modules.

[0152] It can be understood that the performing of the first prompt operation may include at least one of the following:

[0153] Display prompt information in the client of the data center;

[0154] Send a notification message to the terminals of relevant personnel.

[0155] Optionally, the device further comprises:

[0156] The identification updating module is used to generate a new identification of the first data based on the query keyword when there is no abnormality in the first data, wherein the new identification of the first data includes the query keyword.

[0157] It can be seen that when the first data is found according to the query keyword and there is no abnormality in the first data, a new identifier can be generated according to the current identifier of the first data, that is, the query keyword is added on the basis of the current identifier.

[0158] It should be noted that the division of units in the embodiments of the present application is schematic and is only a logical function division. There may be other division methods in actual implementation. In addition, each functional unit in each embodiment of the present application may be integrated into a processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0159] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a processor-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) or a processor (processor) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, ROM), random access memory (Random Access Memory, RAM), disk or optical disk and other media that can store program codes.

[0160] It should be noted here that the above-mentioned device provided in the embodiment of the present application can implement all the method steps implemented in the above-mentioned method embodiment, and can achieve the same technical effect. The parts and beneficial effects of this embodiment that are the same as those in the method embodiment will not be described in detail here.

[0161] The embodiment of the present application also provides an electronic device, such as Figure 4 As shown, the electronic device includes a memory 420, a transceiver 410, and a processor 400;

[0162] Memory 420, used for storing computer programs;

[0163] The transceiver 410 is used to receive and send data under the control of the processor 400;

[0164] The processor 400 is used to read the computer program in the memory 420 and execute the aforementioned data center failure determination method.

[0165] Among them, Figure 4 In the embodiment, the bus architecture may include any number of interconnected buses and bridges, specifically linking together various circuits of one or more processors represented by processor 400 and memory represented by memory 420. The bus architecture may also link together various other circuits such as peripherals, voltage regulators, and power management circuits, which are well known in the art and are therefore not further described herein. The bus interface provides an interface. The transceiver 410 may be a plurality of components, namely, a transmitter and a receiver, providing a unit for communicating with various other devices on a transmission medium, which transmission medium may include a wireless channel, a wired channel, an optical cable, and the like. The processor 400 is responsible for managing the bus architecture and general processing, and the memory 420 may store data used by the processor 400 when performing operations.

[0166] The processor 400 may be a central processing unit (CPU), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or a complex programmable logic device (CPLD). The processor 400 may also adopt a multi-core architecture.

[0167] It should be noted here that the above-mentioned device provided in the embodiment of the present application can implement all the method steps implemented in the above-mentioned method embodiment, and can achieve the same technical effect. The parts and beneficial effects of this embodiment that are the same as those in the method embodiment will not be described in detail here.

[0168] An embodiment of the present application also provides a readable storage medium, on which a program or instruction is stored. When the program or instruction is executed by a processor, the above-mentioned data middle station fault determination method is implemented.

[0169] The computer-readable storage medium can be any available medium or data storage device that can be accessed by the processor, including but not limited to magnetic storage (such as floppy disks, hard disks, magnetic tapes, magneto-optical disks (Magnet-Optica, MO)), etc.), optical storage (such as Compact Disk (CD), Digital Versatile Disc (DVD), Blu-ray Disc (BD), Holographic Versatile Disc (HVD), etc.), and semiconductor memory (such as Read-Only Memory (ROM), Erasable Programmable Read-Only Memory (EPROM), Electrically Erasable Programmable Read Only Memory (EEPROM), non-volatile memory (NAND FLASH), solid-state drive (Solid State Disk or Solid State Drive, SSD)), etc.

[0170] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage and optical storage, etc.) that contain computer-usable program code.

[0171] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer executable instructions. These computer executable instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0172] These processor executable instructions may also be stored in a processor readable memory that can direct a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the processor readable memory produce an article of manufacture including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0173] These processor-executable instructions may also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for executing on the computer or other programmable device to implement the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0174] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.

Claims

1. A method for determining a data center fault, characterized in that: The method comprises: Obtaining a query request for a first service, wherein the query request includes a query keyword; In the data center, according to the stored first correspondence, query the first data corresponding to the data parameter information matching the query keyword, wherein the first correspondence includes the correspondence between the target data, the data parameter information of the target data, and the identifier of the target data, and the identifier of the target data includes the identifier information of the component in the data center that processes the target data; In the case that the first data is abnormal, the abnormal component among the components processing the first data in the data center is determined according to the identifier of the first data.

2. The method according to claim 1, characterized in that The query request also includes the service type of the first service; In the data center, searching for first data corresponding to the data parameter information matching the query keyword according to the stored first corresponding relationship includes: In a case where the service type of the first service belongs to a first predetermined type, in the data center, first data corresponding to the data parameter information matching the query keyword is searched according to the first corresponding relationship.

3. The method according to claim 2, characterized in that The method further comprises: When the business type of the first business belongs to the second predetermined type, data corresponding to the data parameter information matching the query keyword is queried in the data bottom layer according to the stored second correspondence relationship, wherein the second correspondence relationship includes the correspondence between the source data and the data parameter information of the source data.

4. The method according to claim 1 or 2, characterized in that: The method further comprises: When there is no abnormality in the first data, obtaining a first value of a target indicator of the first business according to the first data; Compare the first value with the second value, wherein the second value is the value of the target indicator determined by the data of the first business queried last time according to the query keyword; When the absolute value of the difference between the first value and the second value exceeds a target threshold, obtaining source data generating the first data in the data bottom layer; Processing the source data through a component identified by first identification information to obtain second data, wherein the first identification information includes identification information of the component included in the identification of the first data; Obtaining a third value of the target indicator according to the second data; Comparing the third value with the second value; When the absolute value of the difference between the third value and the second value still exceeds the target threshold, a first prompt operation is performed, wherein the first prompt operation is used to indicate that an abnormal component exists in the components identified by the first identification information.

5. The method according to claim 1, characterized in that The method further comprises: In the case that there is no abnormality in the first data, a new identifier of the first data is generated based on the query keyword, wherein the new identifier of the first data includes the query keyword.

6. A data center fault determination device, characterized in that: The device comprises: A first acquisition module, configured to acquire a query request for a first service, wherein the query request includes a query keyword; A first query module is used to query, in the data center, first data corresponding to the data parameter information matching the query keyword according to a stored first correspondence, wherein the first correspondence includes a correspondence between target data, data parameter information of the target data, and an identifier of the target data, and the identifier of the target data includes identifier information of a component in the data center that processes the target data; The first processing module is used to determine, when an abnormality exists in the first data, an abnormal component among the components in the data processing station according to an identifier of the first data.

7. The device according to claim 6, characterized in that The device further includes: a second processing module, configured to perform the following process: When there is no abnormality in the first data, obtaining a first value of a target indicator of the first business according to the first data; Compare the first value with the second value, wherein the second value is the value of the target indicator determined by the data of the first business queried last time according to the query keyword; When the absolute value of the difference between the first value and the second value exceeds a target threshold, obtaining source data for generating the first data in the data bottom layer; Processing the source data through a component identified by first identification information to obtain second data, wherein the first identification information includes identification information of the component included in the identification of the first data; Obtaining a third value of the target indicator according to the second data; Comparing the third value with the second value; When the absolute value of the difference between the third value and the second value still exceeds the target threshold, a first prompt operation is performed, wherein the first prompt operation is used to indicate that an abnormal component exists in the components identified by the first identification information.

8. The device according to claim 6, characterized in that The device also includes: The identification updating module is used to generate a new identification of the first data based on the query keyword when there is no abnormality in the first data, wherein the new identification of the first data includes the query keyword.

9. An electronic device, characterized in that: Including memory, transceiver, processor: Memory for storing computer programs; a transceiver, for transmitting and receiving data under the control of the processor; A processor, configured to read the computer program in the memory and execute the data center failure determination method according to any one of claims 1 to 5.

10. A readable storage medium, characterized in that: The readable storage medium stores a program or instruction, and when the program or instruction is executed by the processor, the data center failure determination method according to any one of claims 1 to 5 is implemented.