Data processing method, device, system and electronic equipment for multiple data centers

Abstract

The present disclosure provides a data processing method and device, system, electronic device, and storage medium for multiple data centers, which relates to the field of data processing technology and can be applied to scenarios of multiple data centers. The data processing method includes: displaying a query task configuration interface; generating query information in response to configuration operations in the query task configuration interface; sending query information to at least one local data center; receiving statistical analysis result information of buried data obtained based on the query information returned by at least one local data center; and outputting statistical analysis result information of buried data. The technical solution of the embodiments of the present disclosure can realize statistical analysis of buried data in multiple data centers and reduce the traffic of data transmission.

Description

Background Art

[0002] Data centers are infrastructure that can calculate, store, and transmit data, and they concentrate various data resources and business systems. For some special fields, such as medical care, education, and finance, a large amount of data needs to be processed every day, and as the business continues to grow, the amount of data carried by data centers is also growing rapidly. In this way, data collection and statistical analysis in the same data center will consume a lot of data center resources and data transmission traffic.

[0003] It should be noted that the information disclosed in the above background technology section is only used to enhance the understanding of the background of the present disclosure, and therefore may include information that does not constitute the prior art known to ordinary technicians in the field. Summary of the invention

[0004] The purpose of the embodiments of the present disclosure is to provide a multi-data center data processing method, a multi-data center data processing device, a data processing system, an electronic device and a computer-readable storage medium, thereby at least to a certain extent overcoming the problem in the related art that data collection and statistical analysis on the same data center consumes a large amount of data center resources and data transmission traffic.

[0005] Other features and advantages of the present disclosure will become apparent from the following detailed description, or may be learned in part by the practice of the present disclosure.

[0006] According to a first aspect of an embodiment of the present disclosure, a multi-data center data processing method is provided, which is applied to a headquarters data center in a data processing system, wherein the data system further includes at least one local data center in communication connection with the headquarters data center, and the data processing method includes:

[0007] Display the query task configuration interface;

[0008] In response to a configuration operation in the query task configuration interface, generating query information;

[0009] Sending the query information to the at least one local data center;

[0010] Receiving statistical analysis result information of the buried point data obtained based on the query information returned by the at least one local data center;

[0011] Output the statistical analysis result information of the buried point data.

[0012] In some example embodiments of the present disclosure, based on the above solution, the query task configuration interface includes a task information list, and the query information is generated in response to the configuration operation in the query task configuration interface, including:

[0013] In response to a selection operation of selecting a query task from the task information list, the selected query task is generated as query information.

[0014] In some example embodiments of the present disclosure, based on the above solution, the data processing method further includes:

[0015] In response to triggering the task management function control, displaying the task management interface;

[0016] In response to a management operation in the task management interface, the task information list is processed accordingly, and the management operation includes at least one of deleting a task, adding a task, and modifying a task.

[0017] In some example embodiments of the present disclosure, based on the above solution, the query information includes a query object and a query path identifier, and the receiving of the statistical analysis result information of the buried data obtained based on the query information returned by the at least one local data center includes:

[0018] Receive the buried data statistical analysis result information corresponding to the query path identifier and the query object obtained from the local database and returned by the at least one local data center, and obtain the buried data statistical analysis result information corresponding to the query information, wherein the query path identifier includes a local database identifier or a distributed file system identifier.

[0019] In some example embodiments of the present disclosure, based on the above solution, the data processing method further includes:

[0020] The query information and the statistical analysis result information of the buried data are saved.

[0021] According to a second aspect of an embodiment of the present disclosure, a multi-data center data processing method is provided, which is applied to a local data center in a data processing system, wherein the data processing system further includes a headquarters data center, and the headquarters data center is communicatively connected to at least one of the local data centers. The data processing method includes:

[0022] Collect buried point data;

[0023] Receiving query information sent by the headquarters data center, the query information is generated based on a configuration operation in a query task configuration interface of the headquarters data center;

[0024] Performing statistical analysis on the buried point data based on the query information to obtain statistical analysis result information of the buried point data;

[0025] The statistical analysis result information of the buried data is sent to the headquarters data center.

[0026] In some example embodiments of the present disclosure, based on the above solution, the query information includes a query object and a query path identifier, and the statistical analysis of the buried data based on the query information includes:

[0027] Read the buried point data from the data storage system corresponding to the query path identifier;

[0028] The read buried point data is statistically analyzed based on the query object.

[0029] According to a third aspect of an embodiment of the present disclosure, a multi-data center data processing device is provided, which is applied to a headquarters data center in a data processing system, wherein the data processing system further includes at least one local data center communicatively connected to the headquarters data center, and the data processing device includes:

[0030] Display module, used to display the query task configuration interface;

[0031] A generating module, used for generating query information in response to a configuration operation in the query task configuration interface;

[0032] A first sending module, configured to send the query information to the at least one local data center;

[0033] A first receiving module is used to receive the statistical analysis result information of the buried point data obtained based on the query information returned by the at least one local data center;

[0034] The output module is used to output the statistical analysis result information of the buried point data.

[0035] According to a fourth aspect of an embodiment of the present disclosure, a multi-data center data processing device is provided, which is applied to a local data center in a data processing system, wherein the data processing system further includes a headquarters data center, and the headquarters data center is communicatively connected to at least one of the local data centers, and the data processing device includes:

[0036] Collection module, used to collect buried point data;

[0037] A second receiving module, configured to receive query information sent by the headquarters data center, wherein the query information is generated based on a configuration operation in a query task configuration interface of the headquarters data center;

[0038] An analysis module, used to perform statistical analysis on the buried point data based on the query information to obtain statistical analysis result information of the buried point data;

[0039] The second sending module is used to send the statistical analysis result information of the buried data to the headquarters data center.

[0040] According to a fifth aspect of an embodiment of the present disclosure, there is provided an electronic device, comprising: a processor; and a memory, wherein the memory stores computer-readable instructions, and when the computer-readable instructions are executed by the processor, the multi-data center data processing method described in any one of the above is implemented.

[0041] According to a fourth aspect of an embodiment of the present disclosure, there is provided a computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the multi-data center data processing method according to any one of the above items is implemented.

[0042] The data processing method, data processing device, electronic device and computer-readable storage medium of multiple data centers provided by the embodiments of the present disclosure can configure query tasks in the headquarters data center, generate query information, and send the query information to at least one local data center. The at least one local data center obtains the corresponding statistical analysis result information of the buried data according to the query information, and returns the statistical analysis result information of the buried data to the headquarters data center for output, thereby realizing the statistical analysis of the buried data under multiple data centers. On the one hand, multiple local data centers can be used to perform statistical analysis on their respective buried data, and the huge amount of data can be distributed to multiple data centers for separate processing, thereby avoiding the problem that data collection and statistical analysis on the same data center will consume a large amount of data center resources; on the other hand, the statistical analysis of the buried data can be completed in each local data center, and the headquarters data center only needs to send query tasks to the local data center required for statistics, and the local data center can feedback the corresponding statistical analysis result information of the buried data to the headquarters data center. Only the statistical analysis result information of the buried data is transmitted during data transmission, and there is no need to aggregate all the buried data to the headquarters data center for statistical analysis, thereby reducing the traffic of data transmission.

[0043] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] The drawings herein are incorporated into the specification and constitute a part of the specification, showing embodiments consistent with the present disclosure, and together with the specification, are used to explain the principles of the present disclosure. Obviously, the drawings described below are only some embodiments of the present disclosure, and for ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work. In the drawings:

[0045] Figure 1 A schematic diagram of an application scenario deployment of multiple data centers according to some embodiments of the present disclosure is schematically shown;

[0046] Figure 2A system architecture diagram of a data processing system according to some embodiments of the present disclosure is schematically shown;

[0047] Figure 3 A schematic diagram schematically shows a data processing method flow of multiple data centers according to some embodiments of the present disclosure;

[0048] Figure 4 A schematic diagram schematically shows another multi-data center data processing method flow according to some embodiments of the present disclosure;

[0049] Figure 5 A schematic diagram schematically illustrates the working principle of a data processing system according to some embodiments of the present disclosure;

[0050] Figure 6 A schematic diagram of the structure of a data processing device in multiple data centers according to some embodiments of the present disclosure is shown;

[0051] Figure 7 A schematic diagram schematically shows the structure of another multi-data center data processing device according to some embodiments of the present disclosure;

[0052] Figure 8 The structure diagram of the electronic device according to some embodiments of the present disclosure is schematically shown;

[0053] Fig. 9 A schematic diagram of a computer-readable storage medium according to some embodiments of the present disclosure is schematically shown.

[0054] In the drawings, the same or corresponding reference numerals represent the same or corresponding parts. DETAILED DESCRIPTION

[0055] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in a variety of forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that this disclosure will be more comprehensive and complete and will fully convey the concept of the example embodiments to those skilled in the art.

[0056] In addition, the described features, structures or characteristics may be combined in one or more embodiments in any suitable manner. In the following description, many specific details are provided to provide a full understanding of the embodiments of the present disclosure. However, those skilled in the art will appreciate that the technical solutions of the present disclosure may be practiced without one or more of the specific details, or other methods, components, devices, steps, etc. may be adopted. In other cases, known methods, devices, implementations or operations are not shown or described in detail to avoid blurring the various aspects of the present disclosure.

[0057] Furthermore, the drawings are only schematic illustrations and are not necessarily drawn to scale. The block diagrams shown in the drawings are only functional entities and do not necessarily correspond to physically separate entities. That is, these functional entities may be implemented in software form, or in one or more hardware modules or integrated circuits, or in different networks and / or processor devices and / or microcontroller devices.

[0058] In this article, it is important to understand that the terms involved are explained as follows:

[0059] Tracking statistics is the statistical calculation of tracking data, which can include PV (Page View) and UV (Unique Visitor), but not limited to these.

[0060] PV, or page views, is the total number of web pages actually viewed by visitors. It can be the main indicator for measuring an online news channel or website or even an online news item.

[0061] UV, also known as unique IP (Internet Protocol) visitors, can be counted using two numerical standards, "number of visits" and "number of unique visitors", that is, the number of visits and the number of unique visitors are two different concepts;

[0062] A data center is a set of interconnected computers in a fixed geographical area that provide services.

[0063] Multi-data center refers to data centers in different geographical areas;

[0064] Flow rate refers to the amount of fluid flowing through the effective cross-section of a closed pipe or open channel per unit time, also known as instantaneous flow rate.

[0065] Data tracking is the recording of user behavior processes and results in order to meet the needs of fast, efficient and rich data applications. Data tracking is a method of data collection, and tracking is the source of data. The data collected through tracking can be used to analyze the usage of websites or APPs (Applications), user behavior habits, etc., and is the basis for establishing data products such as user portraits and user behavior paths.

[0066] With the rapid development of technologies such as cloud computing, big data, the Internet, and artificial intelligence, the amount of data generated by users accessing websites, apps, etc. through various terminal devices is increasing dramatically. Moreover, as the business of enterprises continues to grow, the amount of data and different forms of business needs will also prompt enterprises to have multiple data centers. There will be a statistical demand for the use of services in different data centers. At this time, it is necessary to collect statistics on the buried data in different data centers and aggregate them to the headquarters for query. For the scenario of multiple data centers, it is necessary to aggregate all the buried data of each data center for statistical analysis, which will consume a large amount of data traffic.

[0067] For example, in a cloud computing network environment, the headquarters data center can be deployed on a public cloud, and the local data centers can be deployed on a private cloud, with limited network traffic between clouds. In a multi-data center scenario, if all the data points of the local data centers are aggregated to the headquarters data center for statistical calculation, it will consume a large amount of network traffic, put great pressure on the cloud computing network, and affect network performance.

[0068] Based on this, the scheme of the present disclosure is proposed. In the embodiment of the present disclosure, the headquarters data center can be deployed on a public cloud, and the local data center can be deployed on a private cloud; the query task configuration interface can be displayed in the headquarters data center, and the headquarters data center generates query information in response to the configuration operation in the query task configuration interface, and then sends the query information to at least one local data center required for statistics; the local data center can obtain the corresponding point data statistical analysis result information based on the query information and feed it back to the headquarters data center for output.

[0069] Figure 1 The following schematic diagram shows a deployment diagram of a multi-data center application scenario according to some embodiments of the present disclosure. Figure 1 As shown, taking the medical big data scenario as an example, a medical service enterprise can provide a medical record management system service for the hospital, and can deploy a medical record management platform in the local data center on each hospital's local private cloud, such as the local data center 110 deployed by hospital A and the local data center 130 deployed by hospital B. The buried point data analysis platform can be deployed in the headquarters data center 120 of the local public cloud of the medical service enterprise. Hospitals can use the medical record management platform to manage patient medical records, such as creating, modifying, and viewing patient medical record details. Medical service companies can use the buried point data analysis platform to statistically analyze the use of the medical record management platform, such as the number of times users browse the medical record details page, the user's behavioral habits in using the medical record management platform, etc. Through the statistical analysis of buried point data, it can be known whether there are problems with the products provided by the medical service company in order to further improve the products, and the user's usage habits can be analyzed to optimize the products, etc.

[0070] according to Figure 1As shown in the application scenario deployment diagram, the managers of the medical service enterprise can issue the buried data statistics task to the local data center 110 of Hospital A and the local data center 130 of Hospital B through the buried data analysis platform of the headquarters data center 120. Taking the local data center 110 as an example, after receiving the buried data statistics task, the local data center 110 can obtain the corresponding buried data according to the buried data statistics task for statistical analysis, and then feed back the statistical analysis results to the headquarters data center 120. Alternatively, the local data center 110 can also perform a comprehensive statistical analysis on the buried data collected according to the set period, and save the statistical analysis results locally. After receiving the buried data statistics task issued by the headquarters data center 120, the corresponding results can be read from the locally saved statistical analysis results according to the buried data statistics task and fed back to the headquarters data center 120. Alternatively, after receiving the buried data statistics task issued by the headquarters data center 120, the local data center 110 performs statistical analysis on the buried data. The headquarters data center 120 periodically summarizes the statistical analysis results from the local data center 110 and saves them in the local database. The medical service enterprise managers can query the required statistical analysis results from the local database through the buried data analysis platform.

[0071] Based on this, cloud computing networks only need to transmit the point data statistics tasks issued by the headquarters data center 120 and the statistical analysis results fed back by the local data center 110, without the need to transmit a large amount of point data, which effectively reduces the data transmission traffic between cloud computing networks and ensures the network performance of the cloud computing network.

[0072] A multi-data center data processing method is provided in an example embodiment of the present disclosure. The data processing method can be applied to a headquarters data center in a data processing system. The headquarters data center can be communicatively connected to at least one local data center, and can issue instructions to the local data centers, transmit data, and receive data transmitted by the local data centers.

[0073] In this example embodiment, reference Figure 2In the system architecture shown, the data processing system may include a headquarters data center 210 deployed in a public cloud, at least one local data center 220 deployed in a private cloud, and a gateway device 230. The private cloud may be deployed in a hospital, a business center, or other places, for example. The headquarters data center 210 may be used to display a query task configuration interface, generate query information in response to configuration operations in the query task configuration interface, send query information to at least one local data center 220 through a gateway device 230, and receive the statistical analysis result information of the buried point data obtained based on the query information returned by at least one local data center 220 through the gateway device 230, and output the statistical analysis result information of the buried point data. The local data center 220 may be used to collect buried point data, receive the query information sent by the headquarters data center 210 through the gateway device 230, perform statistical analysis on the buried point data based on the query information, obtain the statistical analysis result information of the buried point data, and send the statistical analysis result information of the buried point data to the headquarters data center 210. The gateway device 230 may be used to send query information to the local data center 220, and feedback the statistical analysis result information of the buried point data to the headquarters data center 210.

[0074] The gateway device 230 may be an access gateway for managing and transmitting signaling and data between the headquarters data center and the local data centers, as well as performing authority verification, etc.

[0075] The data processing system may also provide a medium for communication links between data centers via network 240. Network 240 may include various connection types, such as wired communication links, wireless communication links, and the like.

[0076] It should be understood that Figure 2 The number of networks and local data centers in the figure is only for illustration, and any number of networks and local data centers may be provided according to implementation requirements.

[0077] In an exemplary embodiment, the network status in the data processing system can also be monitored by the gateway device 230. When the network status is abnormal, such as disconnection, a network abnormality prompt message can be sent to the headquarters data center 210 to prompt the network abnormality and make abnormal processing in time. The gateway device 230 can also send the network abnormality prompt message to the local data center 220 at the same time.

[0078] Figure 3 The following is a schematic diagram of a data processing method for multiple data centers according to some embodiments of the present disclosure. Figure 3 As shown, the data processing method may include the following steps S310 to S350:

[0079] In step S310, a query task configuration interface is displayed.

[0080] The query task configuration interface can be displayed on the human-computer interaction interface of the headquarters data center. For example, a function control for configuring a query task can be provided. When the headquarters data center needs to collect data from buried points to learn about the usage of each business module, APP, etc., the function control for configuring the query task can be triggered. The headquarters data center responds to the trigger operation and displays the query task configuration interface on the human-computer interaction interface.

[0081] In step S320, query information is generated in response to the configuration operation in the query task configuration interface.

[0082] Exemplarily, the query task configuration interface may include a task information list, and the headquarters data center generates the selected query task as query information in response to a selection operation of selecting a query task from the task information list. The query information may indicate the statistical analysis result information of the buried point data to be obtained.

[0083] For example, optional query tasks can be displayed in the task information list, and the correspondence between the task number and the task name of each query task can be displayed. When a query task is clicked, the query task can be generated as corresponding query information. Exemplarily, a selection control can also be provided for each optional query task, and the query task is selected when the selection control is triggered. The task name can be named according to the specific task content, such as the number of APP visits, the user's operating behavior preferences, the access time of the business module, etc.

[0084] Exemplarily, the query information may include one query task or multiple query tasks.

[0085] In an exemplary embodiment, the headquarters data center can also manage the task information list. Specifically, the headquarters data center can provide a task management function control. In response to triggering the task management function control, a task management interface is displayed. In response to a management operation in the task management interface, the task information list is processed accordingly. The management operation may include at least one of deleting a task, adding a task, and modifying a task. By deleting, adding, modifying, and other operations on the query task, actual needs can be met.

[0086] In step S330, query information is sent to at least one local data center.

[0087] The headquarters data center can communicate with at least one local data center and can send the generated query information to at least one local data center communicating with it. Figure 2The gateway device 230 shown interacts with at least one local data center for data exchange, and the headquarters data center sends the generated query information to at least one local data center communicating with it through the gateway device 230 .

[0088] Exemplarily, the headquarters data center may directly send the generated query information to all local data centers communicating with it, or may specify a local data center according to the query information and send the query information to the specified local data center.

[0089] In step S340, statistical analysis result information of the buried point data obtained based on the query information is received and returned by at least one local data center.

[0090] After the headquarters data center sends the query information to the local data center, the local data center can obtain the corresponding point-of-sale data statistical analysis result information locally based on the query information, and then return the point-of-sale data statistical analysis result information to the headquarters data center.

[0091] Exemplarily, the query information may include a query object and a query path identifier. The query object may indicate the content to be queried, and the query path identifier may indicate the source of the required query content. The query path identifier may include, for example, a local database identifier or a distributed file system identifier.

[0092] Accordingly, receiving the buried point data statistical analysis result information obtained based on the query information returned by at least one local data center may include: receiving the buried point data statistical analysis result information corresponding to the query path identifier and the query object obtained from the local database returned by at least one local data center, and obtaining the corresponding buried point data statistical analysis result information. Figure 2 The gateway device 230 shown sends the statistical analysis result information of the buried data to the headquarters data center.

[0093] Exemplarily, the query information may also include a task identifier, which may be, for example, a task number, and the acquired statistical analysis result information of the buried data may be matched with the corresponding query task through the task identifier.

[0094] Exemplarily, the local data center can store the collected point-of-sale data in its local database and distributed file system respectively. In an exemplary embodiment, after receiving the query information, the local data center can obtain the collected point-of-sale data from its local database or distributed file system according to the query path identifier in the query information, and perform statistical analysis on the point-of-sale data, obtain the statistical analysis result information of the point-of-sale data corresponding to the query object in the query information from the statistical analysis results, and feed it back to the headquarters data center in real time. In another exemplary embodiment, after the headquarters data center sends the query information to the local data center, the local data center can save the query information, and at a set time point, start to obtain the collected point-of-sale data from its local database or distributed file system according to the query path identifier in the query information and perform statistical analysis on the point-of-sale data, and then return the statistical analysis results to the headquarters data center. The headquarters data center can regularly summarize these statistical analysis results into the headquarters database, and then query the corresponding point-of-sale data statistical analysis result information from the headquarters database according to the query object in the query information.

[0095] In step S350, the statistical analysis result information of the buried point data is output.

[0096] In an example embodiment, the headquarters data center may also store query information and information on statistical analysis results of buried data, for example, in the form of a data table.

[0097] In an example embodiment, the headquarters data center can view, delete, modify, and other operations on the saved query information and buried data statistical analysis result information.

[0098] The data processing method provided by the example embodiment of the present disclosure can, on the one hand, utilize multiple local data centers to perform statistical analysis on their respective point-of-sale data, and distribute the huge amount of data to multiple data centers for separate processing, thereby avoiding the problem that data collection and statistical analysis in the same data center will consume a large amount of data center resources; on the other hand, the statistical analysis of the point-of-sale data can be completed in each local data center, and the headquarters data center only needs to issue query tasks to the local data centers that need statistics, and the local data centers can then feed back the corresponding statistical analysis result information of the point-of-sale data to the headquarters data center. Only the statistical analysis result information of the point-of-sale data is transmitted during data transmission, and there is no need to aggregate all the point-of-sale data to the headquarters data center for statistical analysis, thereby reducing the data transmission traffic.

[0099] The exemplary embodiment of the present disclosure also provides another multi-data center data processing method, which can be applied to a local data center in a data processing system, and the data processing system can also include a headquarters data center, which can be connected to at least one local data center. The architecture of the data processing system can be referred to as Figure 2 shown. Figure 4 The flowchart of the data processing method of the multi-data center is schematically shown. The method may include the following steps S410 to S440:

[0100] In step S410, the buried point data is collected.

[0101] The local data center can monitor the applications, business systems, networks, etc. under its jurisdiction and collect tracking data. The tracking data may include at least one of user behavior, client information, and network information, and may also include collection time and collection object information. The collection object information may include business system information, business module information in the business system, etc. The local data center can track data according to actual data collection needs.

[0102] In step S420, query information sent by the headquarters data center is received, and the query information is generated based on the configuration operation in the query task configuration interface of the headquarters data center.

[0103] The headquarters data center may provide a query task configuration interface, which may include a task information list. In response to a selection operation of selecting a query task from the task information list, the headquarters data center may generate the selected query task as query information. After the query information is generated, the query information may be sent to the local data center, for example, by Figure 2 The gateway device 230 shown sends the query information to the local data center.

[0104] Exemplarily, the query information may include a query object and a query path identifier. The query object may indicate the content to be queried, such as the number of visits to the APP, the user's operating behavior preferences, the access time of the business module, etc. The query path identifier may indicate the data source of the query object, such as from a local database or a distributed file system.

[0105] In step S430, statistical analysis is performed on the buried point data based on the query information to obtain statistical analysis result information of the buried point data.

[0106] Exemplarily, the local data center can store the collected burial point data in different data storage systems, such as in a local local database and a distributed file system.

[0107] Exemplarily, the query information may include a query object and a query path identifier. After the local data center receives the query information sent by the headquarters data center, it can read the buried data from the data storage system corresponding to the query path identifier, perform statistical analysis on the read buried data based on the query object, and obtain the buried data statistical analysis result information. For example, when the query path identifier is a local database identifier, the collected buried data can be read from the local database, and then the read buried data can be statistically analyzed according to the query object. For example, if the query object is the total number of visits to application A, the access data of application A in the buried data can be counted and its total number of visits can be calculated. For example, when the query path identifier is a distributed file system identifier, the collected buried data can be obtained from the distributed file system, and then the read buried data can be statistically analyzed according to the query object.

[0108] In step S440, the statistical analysis result information of the buried point data is sent to the headquarters data center.

[0109] For example, local data centers can Figure 2 The gateway device 230 shown sends the statistical analysis result information of the buried data to the headquarters data center.

[0110] For example, the local data center can provide real-time feedback of the statistical analysis results of the buried data to the headquarters data center.

[0111] Exemplarily, the local data center may also feed back the statistical analysis results of the buried data to the headquarters data center at a set period, such as summarizing the statistical analysis results every day. Specifically, after the headquarters data center sends the query information to the local data center, the local data center may save the query information. When the set time point is reached, the local data center obtains the buried data collected within the set time period (such as the day) from the local database or distributed file system according to the query path identifier in the query information and performs statistical analysis on the buried data, and then feeds back the statistical analysis results to the headquarters data center. The headquarters data center can summarize these statistical analysis results in the headquarters database, and then can query the corresponding buried data statistical analysis result information from the headquarters database according to the query information.

[0112] The data processing method provided by the disclosed embodiment can, on the one hand, complete the statistical analysis of the buried data in each local data center. The headquarters data center only needs to issue query tasks to the local data centers that need statistics, and the local data centers can feed back the corresponding statistical analysis results of the buried data to the headquarters data center. There is no need to aggregate all the buried data to the headquarters data center for statistical analysis, which effectively reduces the data transmission traffic. On the other hand, the collected buried data can be stored in different data storage systems respectively, so that the buried data can be backed up, and the data storage advantages and data query advantages of different data storage systems can be brought into play, so that data query is more flexible and has higher scalability.

[0113] Based on the above Figure 3 and Figure 4 The method of the corresponding embodiment is further illustrated below with examples of the scheme of the present disclosure.

[0114] Figure 5 The working principle diagram of the data processing system according to some embodiments of the present disclosure is schematically shown. Figure 5 As shown, the data processing system may include a headquarters data center 51 and at least one local data center that is communicatively connected to the headquarters data center 51. Figure 5 Two local data centers (local data center 52 and local data center 53) are used as examples for illustration, but they are not used to limit the number of local data centers disclosed in the present invention. The headquarters data center and the local data centers can exchange data through gateway devices, for example. It should be noted that Figure 5 Only the working principle block diagram of the local data center 52 is shown for ease of description. The working principle block diagram of the local data center 53 can refer to the local data center 52.

[0115] like Figure 5 As shown, the local data center 52 can set up buried points for APP1, APP2 and APP3 respectively. The monitoring system can collect the buried point data of APP1, APP2 and APP3 through these buried points. The buried point data may include, for example, the collection time, product identification, business function module information, user behavior data, client information and network information. The collected buried point data is collected by the buried point collection interface and copied into two copies for transmission separately. One is written into the local database, and the other is cached in the data queue through the collection program. For the buried point data cached in the data queue, the buried point data of the day can be uploaded to the distributed file system every day in units of days. The distributed file system can provide a variety of query types and methods, and the data can also be shared with other systems for use, which has high scalability.

[0116] The headquarters data center 51 may provide a query task configuration interface, which may include a task information list, and the user may select the required query task based on the task information list, such as querying the most frequently used APPs by the user, querying the user's access to each functional module of APP1, querying the user's stay time on chart G in APP2, etc. Exemplarily, the query task configuration interface may be displayed in the form of a web page, or in a developed application program interface.

[0117] After the user selects the required query task, the headquarters data center 51 can generate query information for these query tasks and then send it to the local data center 52 and the local data center 53, wherein the query information may include the query object and the query path identifier. Exemplarily, the headquarters data center 51 can also determine the local data center to be sent to based on the query task selected by the user. For example, if it is necessary to query the user's access to each functional module of APP1, and only the local data center 52 collects the buried point data of APP1, the query information can be sent only to the local data center 52.

[0118] For example, SQL (Structured Query Language) can be used to obtain relevant data of the selected query task from the headquarters database, and these data can be used to generate SQL query information and sent to the local data center. For example, data transmission between the headquarters data center and the local data center can be carried out through Web API (Web Application Programming Interface).

[0119] After receiving the query information sent by the headquarters data center 51, the local data center 52 can save the query information. When the specified time point is reached, such as 24:00 every day, the local data center 52 starts to statistically analyze the buried data of the day, such as performing statistical analysis processing such as counting, summing, sorting, and averaging the number of visits.

[0120] In an exemplary embodiment, when the query path identifier in the query information is a local database identifier pointing to a local database, when the set time point is reached, the local data center 52 reads the buried data of the day from the local local database for statistical analysis. Exemplarily, the local data center 52 can perform statistical analysis on the read buried data according to the query object in the query information to obtain the buried data statistical analysis result information corresponding to the query object, and then return the buried data statistical analysis result information to the headquarters data center 51 for output. Exemplarily, the local data center 52 can also directly perform a comprehensive statistical analysis on the read buried data, save the statistical analysis results in the local database, and then read the buried data statistical analysis result information corresponding to the query object from the statistical analysis results based on the query object in the query information, and then return the buried data statistical analysis result information to the headquarters data center 51 for output.

[0121] In an example embodiment, when the query path identifier in the query information is a distributed file system identifier pointing to a distributed file system, the local data center 52 can read the query information to be calculated every day from the distributed file system through the query client, and can read the relevant buried data from the distributed file system through the data query engine at a specified time point (such as 24:00 every day) according to the query object in the query information for statistical analysis, and then return the obtained buried data statistical analysis result information to the headquarters data center 51 for output.

[0122] In an exemplary embodiment, the local data center 53 may also save the information of the statistical analysis results of the buried data in a local database. The headquarters data center 51 may periodically summarize the information of the statistical analysis results of the buried data from the local database, and then save it in the headquarters database, such as summarizing the information of the statistical analysis results of the buried data every day. The headquarters data center 51 may use SQL to send query information to the headquarters database at any time to query the required statistical analysis results of the buried data. Exemplarily, the headquarters data center 51 may also use SQL to manage the information in the headquarters database, such as deleting data, modifying, adding or deleting the query information of the query task, etc.

[0123] Exemplarily, the headquarters data center 51 may also specify the query time in the query information sent to the local data center 52 and the local data center 55 to instruct statistical analysis of the buried data within the query time period. For example, if the query time is 48 hours, statistical analysis of the buried data within two days is performed; or the buried data statistical analysis result information of the buried data within the query time period is read from the local database.

[0124] It should be noted that, although the steps of the method in the present disclosure are described in a specific order in the drawings, this does not require or imply that the steps must be performed in this specific order, or that all the steps shown must be performed to achieve the desired results. Additionally or alternatively, some steps may be omitted, multiple steps may be combined into one step, and / or one step may be decomposed into multiple steps, etc.

[0125] In addition, in this exemplary embodiment, a multi-data center data processing device is also provided, which can be applied to a headquarters data center in a data processing system, and the multi-data center data processing system can also include at least one local data center that is communicatively connected to the headquarters data center. Figure 6 As shown, the data processing device 600 may include a display module 610, a generation module 620, a first sending module 630, a first receiving module 640 and an output module 650. Among them:

[0126] The display module 610 may be used to display a query task configuration interface;

[0127] The generating module 620 may be used to generate query information in response to a configuration operation in the query task configuration interface;

[0128] The first sending module 630 may be used to send query information to at least one local data center;

[0129] The first receiving module 640 may be used to receive statistical analysis result information of the buried point data obtained based on the query information returned by at least one local data center;

[0130] The output module 650 can be used to output the statistical analysis result information of the buried point data.

[0131] In an exemplary embodiment of the present disclosure, based on the aforementioned scheme, the query task configuration interface may include a task information list, and the generation module 620 may be specifically used to generate the selected query task as query information in response to a selection operation of selecting a query task from the task information list.

[0132] In an exemplary embodiment of the present disclosure, based on the above solution, the data processing device 600 may further include a management module; wherein:

[0133] The display module 610 may also be used to display a task management interface in response to triggering a task management function control;

[0134] The management module may be used to process the task information list accordingly in response to management operations in the task management interface, wherein the management operations may include at least one of deleting a task, adding a task, and modifying a task.

[0135] In an exemplary embodiment of the present disclosure, based on the aforementioned scheme, the query information may include a query object and a query path identifier, and the first receiving module 640 may be specifically used to receive the buried data statistical analysis result information corresponding to the query path identifier and the query object obtained from the local database and returned by at least one local data center, to obtain the buried data statistical analysis result information corresponding to the query information, wherein the query path identifier may include a local database identifier or a distributed file system identifier.

[0136] In addition, in this exemplary embodiment, another multi-data center data processing device is also provided, which can be applied to a local data center in a data processing system, and the data processing system can also include a headquarters data center, which can be connected to at least one local data center in communication. Figure 7 As shown, the data processing device 700 may include a collection module 710, a second receiving module 720, an analysis module 730 and a second sending module 740. Among them:

[0137] The collection module 710 can be used to collect buried point data;

[0138] The second receiving module 720 may be used to receive query information sent by the headquarters data center, where the query information is generated based on a configuration operation in the query task configuration interface of the headquarters data center;

[0139] The analysis module 730 may be used to perform statistical analysis on the buried point data based on the query information to obtain statistical analysis result information of the buried point data;

[0140] The second sending module 740 can be used to send the statistical analysis result information of the buried data to the headquarters data center.

[0141] In an exemplary embodiment of the present disclosure, based on the above solution, the query information may include a query object and a query path identifier, and the analysis module 730 may include a reading unit and a statistical analysis unit.

[0142] The reading unit may be used to read the buried point data from the data storage system corresponding to the query path identifier;

[0143] The statistical analysis unit can be used to perform statistical analysis on the read buried point data based on the query object.

[0144] The specific details of each module of the data processing device in the above-mentioned exemplary embodiments have been described in detail in the corresponding data processing method, so they will not be repeated here.

[0145] It should be noted that, although several modules or units of the data processing device are mentioned in the above detailed description, this division is not mandatory. In fact, according to the embodiments of the present disclosure, the features and functions of two or more modules or units described above can be embodied in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided into multiple modules or units to be embodied.

[0146] In addition, in an exemplary embodiment of the present disclosure, an electronic device capable of implementing the above data processing method is also provided.

[0147] Those skilled in the art will appreciate that various aspects of the present disclosure may be implemented as systems, methods or program products. Therefore, various aspects of the present disclosure may be specifically implemented in the following forms, namely: complete hardware embodiments, complete software embodiments (including firmware, microcode, etc.), or embodiments combining hardware and software aspects, which may be collectively referred to herein as "circuits", "modules" or "systems".

[0148] Refer to the following Figure 8 800 according to this embodiment of the present disclosure is described. Figure 8 The electronic device 800 shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0149] like Figure 8 As shown, the electronic device 800 is in the form of a general computing device. The components of the electronic device 800 may include, but are not limited to: the at least one processing unit 810, the at least one storage unit 820, a bus 830 connecting different system components (including the storage unit 820 and the processing unit 810), and a display unit 840.

[0150] The storage unit 820 stores program codes, which can be executed by the processing unit 810, so that the processing unit 810 can perform the steps according to various exemplary embodiments of the present disclosure described in the above embodiments of this specification. For example, the processing unit 810 can perform the following steps: Figure 3 The steps shown in: Step S310, displaying the query task configuration interface; Step S320, generating query information in response to the configuration operation in the query task configuration interface; Step S330, sending the query information to at least one local data center; Step S340, receiving the statistical analysis result information of the buried point data obtained based on the query information returned by at least one local data center; Step S350, outputting the statistical analysis result information of the buried point data.

[0151] The storage unit 820 may include a readable medium in the form of a volatile storage unit, such as a random access memory unit (RAM) 821 and / or a cache memory unit 822 , and may further include a read-only memory unit (ROM) 823 .

[0152] The storage unit 820 may also include a program / utility 824 having a set (at least one) of program modules 825, such program modules 825 including but not limited to: an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment.

[0153] Bus 830 may represent one or more of several types of bus structures, including a memory unit bus or memory unit controller, a peripheral bus, an accelerated graphics port, a processing unit, or a local bus using any of a variety of bus architectures.

[0154] The electronic device 800 may also communicate with one or more external devices 870 (e.g., keyboards, pointing devices, Bluetooth devices, etc.), may also communicate with one or more devices that enable a user to interact with the electronic device 800, and / or communicate with any device that enables the electronic device 800 to communicate with one or more other computing devices (e.g., routers, modems, etc.). Such communication may be performed via an input / output (I / O) interface 850. Furthermore, the electronic device 800 may also communicate with one or more networks (e.g., local area networks (LANs), wide area networks (WANs), and / or public networks, such as the Internet) via a network adapter 860. As shown, the network adapter 860 communicates with other modules of the electronic device 800 via a bus 830. It should be understood that, although not shown in the figure, other hardware and / or software modules may be used in conjunction with the electronic device 800, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems, etc.

[0155] Through the description of the above embodiments, it is easy for those skilled in the art to understand that the example embodiments described here can be implemented by software, or by software combined with necessary hardware. Therefore, the technical solution according to the embodiment of the present disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, and includes several instructions to enable a computing device (which can be a personal computer, a server, a terminal device, or a network device, etc.) to execute the method according to the embodiment of the present disclosure.

[0156] In an exemplary embodiment of the present disclosure, a computer-readable storage medium is also provided, on which a program product capable of implementing the above method of the present specification is stored. In some possible embodiments, various aspects of the present disclosure may also be implemented in the form of a program product, which includes a program code, and when the program product is run on a terminal device, the program code is used to enable the terminal device to perform the steps according to various exemplary embodiments of the present disclosure described in the above "Exemplary Method" section of the present specification.

[0157] refer to Fig. 9 As shown, a program product 900 for implementing the above-mentioned multi-data center data processing method according to an embodiment of the present disclosure is described, which can adopt a portable compact disk read-only memory (CD-ROM) and include program code, and can be run on a terminal device, such as a personal computer. However, the program product of the present disclosure is not limited thereto. In this document, a readable storage medium can be any tangible medium containing or storing a program, which can be used by or in combination with an instruction execution system, an apparatus, or a device.

[0158] The program product may use any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. The readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples (non-exhaustive list) of readable storage media include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0159] Computer readable signal media may include data signals propagated in baseband or as part of a carrier wave, in which readable program code is carried. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Readable signal media may also be any readable medium other than a readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0160] The program code embodied on the readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wired, optical cable, radio frequency (RF), etc., or any suitable combination of the foregoing.

[0161] Program code for performing the operations of the present disclosure may be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java, C++, etc., and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as a separate software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server. In cases involving a remote computing device, the remote computing device may be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computing device (e.g., through the Internet using an Internet service provider).

[0162] In addition, the above-mentioned figures are only schematic illustrations of the processes included in the method according to the exemplary embodiments of the present disclosure, and are not intended to be limiting. It is easy to understand that the processes shown in the above-mentioned figures do not indicate or limit the time sequence of these processes. In addition, it is also easy to understand that these processes can be performed synchronously or asynchronously, for example, in multiple modules.

[0163] Through the description of the above embodiments, it is easy for those skilled in the art to understand that the example embodiments described here can be implemented by software, or by software combined with necessary hardware. Therefore, the technical solution according to the embodiment of the present disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, and includes several instructions to enable a computing device (which can be a personal computer, a server, a touch terminal, or a network device, etc.) to execute the method according to the embodiment of the present disclosure.

[0164] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary techniques in the art that are not disclosed in the present disclosure. The specification and examples are to be considered exemplary only, and the true scope and spirit of the present disclosure are indicated by the claims.

[0165] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. A data processing method for multiple data centers, characterized in that: A headquarters data center is applied to a data processing system, wherein the data processing system further comprises at least one local data center in communication connection with the headquarters data center, and the data processing method comprises: Displaying a query task configuration interface, wherein the query task configuration interface is used to display a task information list, wherein the task information list displays query tasks to be selected; In response to a configuration operation in the query task configuration interface, generating query information; Sending the query information to the at least one local data center; Receiving statistical analysis result information of the buried point data obtained based on the query information returned by the at least one local data center; Outputting the statistical analysis result information of the buried point data; The tracking data is collected by the local data center by monitoring one or more of the applications, business systems, and networks under the jurisdiction of the local data center. The tracking data includes one or more of user behavior, client information, and network information. The local data center stores the collected tracking data in the local database or distributed file system of the local data center.

2. The data processing method according to claim 1, characterized in that: The query task configuration interface includes a task information list, and the query information is generated in response to the configuration operation in the query task configuration interface, including: In response to a selection operation of selecting a query task from the task information list, the selected query task is generated as query information.

3. The data processing method according to claim 2, characterized in that: Also includes: In response to triggering the task management function control, displaying the task management interface; In response to a management operation in the task management interface, the task information list is processed accordingly, and the management operation includes at least one of deleting a task, adding a task, and modifying a task.

4. The data processing method according to claim 1, characterized in that: The query information includes a query object and a query path identifier, and the receiving of the statistical analysis result information of the buried point data obtained based on the query information returned by the at least one local data center includes: Receive the buried data statistical analysis result information corresponding to the query path identifier and the query object obtained from the local database and returned by the at least one local data center, and obtain the buried data statistical analysis result information corresponding to the query information, wherein the query path identifier includes a local database identifier or a distributed file system identifier.

5. A data processing method for multiple data centers, characterized in that: A local data center applied to a data processing system, wherein the data processing system further comprises a headquarters data center, wherein the headquarters data center is communicatively connected to at least one of the local data centers, and the data processing method comprises: Collect buried point data; Receiving query information sent by the headquarters data center, the query information is generated based on a configuration operation in a query task configuration interface of the headquarters data center, the headquarters data center displays a query task configuration interface, the query task configuration interface is used to display a task information list, and the task information list displays query tasks to be selected; Performing statistical analysis on the buried point data based on the query information to obtain statistical analysis result information of the buried point data; Sending the statistical analysis result information of the buried data to the headquarters data center; The tracking data is collected by the local data center by monitoring one or more of the applications, business systems, and networks under the jurisdiction of the local data center. The tracking data includes one or more of user behavior, client information, and network information. The local data center stores the collected tracking data in the local database or distributed file system of the local data center.

6. The data processing method according to claim 5, characterized in that: The query information includes a query object and a query path identifier, and the statistical analysis of the buried data based on the query information includes: Read the buried point data from the data storage system corresponding to the query path identifier; The read buried point data is statistically analyzed based on the query object.

7. A data processing device for multiple data centers, characterized in that: A headquarters data center applied to a data processing system, wherein the data processing system further comprises at least one local data center in communication connection with the headquarters data center, and the data processing device comprises: A display module, used to display a query task configuration interface, wherein the query task configuration interface is used to display a task information list, wherein the task information list displays query tasks to be selected; A generating module, used for generating query information in response to a configuration operation in the query task configuration interface; A first sending module, configured to send the query information to the at least one local data center; A first receiving module is used to receive the statistical analysis result information of the buried point data obtained based on the query information returned by the at least one local data center; An output module is used to output the statistical analysis result information of the buried point data; The tracking data is collected by the local data center by monitoring one or more of the applications, business systems, and networks under the jurisdiction of the local data center. The tracking data includes one or more of user behavior, client information, and network information. The local data center stores the collected tracking data in the local database or distributed file system of the local data center.

8. A data processing device for multiple data centers, characterized in that: A local data center applied to a data processing system, wherein the data processing system further comprises a headquarters data center, wherein the headquarters data center is communicatively connected to at least one of the local data centers, and the data processing device comprises: Collection module, used to collect buried point data; A second receiving module is used to receive query information sent by the headquarters data center, the query information is generated based on a configuration operation in a query task configuration interface of the headquarters data center, the headquarters data center displays a query task configuration interface, the query task configuration interface is used to display a task information list, and the task information list displays query tasks to be selected; An analysis module, used to perform statistical analysis on the buried point data based on the query information to obtain statistical analysis result information of the buried point data; A second sending module is used to send the statistical analysis result information of the buried data to the headquarters data center; The tracking data is collected by the local data center by monitoring one or more of the applications, business systems, and networks under the jurisdiction of the local data center. The tracking data includes one or more of user behavior, client information, and network information. The local data center stores the collected tracking data in the local database or distributed file system of the local data center.

9. A data processing system, characterized in that: It includes a headquarters data center deployed in a public cloud, at least one local data center and gateway equipment deployed in a private cloud; The headquarters data center is used to display a query task configuration interface, generate query information in response to configuration operations in the query task configuration interface, send the query information to the at least one local data center through the gateway device, and receive the buried point data statistical analysis result information obtained based on the query information returned by the at least one local data center through the gateway device, and output the buried point data statistical analysis result information; The local data center is used to collect the buried data, receive the query information sent by the headquarters data center through the gateway device, perform statistical analysis on the buried data based on the query information, obtain the buried data statistical analysis result information, and send the buried data statistical analysis result information to the headquarters data center; The gateway device is used to send the query information to the local data center, and to feed back the statistical analysis result information of the buried data to the headquarters data center; The query task configuration interface is used to display a task information list, and the task information list displays the query tasks to be selected; the tracking data is collected by the local data center by monitoring one or more of the applications, business systems, and networks under the jurisdiction of the local data center, and the tracking data includes one or more of user behavior, client information, and network information. The local data center stores the collected tracking data in the local database or distributed file system of the local data center.

10. An electronic device comprising: processor; as well as A memory having computer-readable instructions stored thereon, wherein the computer-readable instructions, when executed by the processor, implement the multi-data center data processing method according to any one of claims 1 to 6.

11. A computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the multi-data center data processing method according to any one of claims 1 to 6 is implemented.

Images