Data processing method, electronic device, storage medium and program product

Abstract

The present disclosure provides a data processing method, an electronic device, a readable storage medium and a computer program product. The method first receives operation data collected from a business system, wherein the business system is deployed on a third-party platform, and then determines a data processing channel corresponding to the operation data through a group identifier carried in the operation data, and then sends the operation data to a designated storage location for storage through the determined corresponding data processing channel.

Description

Technical Field

[0001] The present disclosure relates to technical fields such as data processing, and in particular to a data processing method, an electronic device, a readable storage medium, and a computer program product. Background Art

[0002] When users deploy their own business systems, they can choose to deploy the business systems on a platform provided by a third-party service provider (referred to as the third-party platform), which will be managed by the third-party platform.

[0003] During the operation of the business system, users need to obtain observation data of each business system. The observation data can be used to perceive the status of the business system, evaluate the health of the business system, and measure the level of business development.

[0004] Since the amount of observation data transmitted is large, if a failure occurs during the data transmission process, the user will not be able to receive any observation data normally until the failure is eliminated, which will result in a large amount of observation data missing on the user side. Summary of the invention

[0005] The present disclosure provides a data processing method, an electronic device, a readable storage medium, and a computer program product.

[0006] The first aspect of the present disclosure proposes a data processing method, including: receiving operation data collected from a business system, wherein the business system is deployed on a third-party platform; determining a data processing channel corresponding to the operation data through a group identifier carried in the operation data, wherein a mapping relationship exists between the group identifier and the data processing channel; and sending the operation data to a designated storage location for storage through the determined corresponding data processing channel.

[0007] According to some embodiments of the present disclosure, the method further includes: sending collection configuration information to a third-party platform, wherein the collection configuration information is used to instruct the third-party platform to add corresponding group identifiers to the operation data collected from the business system according to the specified business system grouping.

[0008] According to some embodiments of the present disclosure, the designated business system grouping is obtained by dividing the business system according to one or more of the business relevance, whether it belongs to the same account of the third-party platform, the flow of the operating data, and the resources required to process the operating data.

[0009] According to some embodiments of the present disclosure, there is a one-to-one mapping relationship between the group identifier and the data processing channel.

[0010] According to some embodiments of the present disclosure, the operating data includes one or more of indicator data, log data, and link data.

[0011] According to some embodiments of the present disclosure, receiving the operating data collected from the business system specifically includes at least one of the following methods: method one, passively receiving the operating data collected from the business system sent by the third-party platform; method two, actively sending a data request to the third-party platform, and receiving the operating data collected from the business system sent by the third-party platform in response to the data request.

[0012] According to some embodiments of the present disclosure, there are multiple designated storage locations, the operating data also carries storage indication information, and the transmission node of the data processing channel includes a data processing module, and the operating data is sent to the designated storage location for storage through the determined corresponding data processing channel, including: sending the operating data to the determined corresponding data processing module; determining the designated storage location corresponding to the operating data according to the storage indication information through the data processing module; and sending the operating data to the corresponding designated storage location for storage.

[0013] According to some embodiments of the present disclosure, the designated storage location corresponding to the operating data is determined by the data processing module according to the storage indication information, including: determining the target routing condition satisfied by the operating data according to the specified routing condition and the parameter information contained in the storage indication information; and determining the designated storage location corresponding to the target routing condition.

[0014] According to some embodiments of the present disclosure, the method further includes: configuring the data processing channel to perform data division and data transmission according to specified storage indication information.

[0015] A second aspect of the present disclosure proposes an electronic device, comprising: a memory storing execution instructions; and a processor executing the execution instructions stored in the memory, so that the processor executes the method described in any one of the above embodiments.

[0016] A third aspect of the present disclosure provides a readable storage medium, wherein the readable storage medium stores a computer program, and when the computer program is executed by a processor, it is used to implement the method described in any of the above embodiments.

[0017] A fourth aspect of the present disclosure provides a computer program product, which includes a computer program. When the computer program is executed by a processor, it is used to implement the method described in any of the above embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The accompanying drawings illustrate exemplary embodiments of the present disclosure and together with the description serve to explain the principles of the present disclosure. These drawings are included to provide a further understanding of the present disclosure and are incorporated in and constitute a part of this specification.

[0019] Figure 1 A schematic diagram of an application scenario of a data processing method according to some embodiments of the present disclosure is shown.

[0020] Figure 2 A schematic diagram of the overall flow of a data processing method M100 according to some embodiments of the present disclosure is shown.

[0021] Figure 3 Some embodiments of the present disclosure are shown. Figure 1 The data processing method M100 shown is a structural block diagram for performing data processing.

[0022] Figure 4 The overall flow chart of the data processing method M100 according to some other embodiments of the present disclosure is shown.

[0023] Figure 5 Some embodiments of the present disclosure are shown. Figure 4 The data processing method M100 shown is a structural block diagram for performing data processing.

[0024] Figure 6 The overall flow chart of the data processing method M100 according to some other embodiments of the present disclosure is shown.

[0025] Figure 7 Some embodiments of the present disclosure are shown. Figure 6 The data processing method M100 shown is a structural block diagram for performing data processing.

[0026] Figure 8 It is a schematic block diagram of the structure of a data processing device according to one embodiment of the present disclosure.

[0027] Fig. 9 1 is a schematic block diagram of the structure of an electronic device 1000 according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0028] The present disclosure is further described in detail below in conjunction with the accompanying drawings and implementations. It is understood that the specific implementations described herein are only used to explain the relevant content, rather than to limit the present disclosure. It should also be noted that, for ease of description, only the parts related to the present disclosure are shown in the accompanying drawings.

[0029] It should be noted that, in the absence of conflict, the embodiments and features of the embodiments in the present disclosure can be combined with each other. The technical solution of the present disclosure will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0030] Unless otherwise specified, the exemplary embodiments / embodiments shown will be understood as providing exemplary features of various details of some ways in which the technical concept of the present disclosure can be implemented in practice. Therefore, unless otherwise specified, the features of the various embodiments / embodiments can be combined, separated, interchanged and / or rearranged without departing from the technical concept of the present disclosure.

[0031] The terms used herein are for the purpose of describing specific embodiments, rather than being restrictive. As used herein, unless the context clearly indicates otherwise, the singular forms "one (kind, person)" and "said (the)" are also intended to include plural forms. In addition, when the terms "comprise" and / or "include" and their variations are used in this specification, it is explained that there are stated features, integral bodies, steps, operations, parts, assemblies and / or their groups, but it is not excluded that there are or add one or more other features, integral bodies, steps, operations, parts, assemblies and / or their groups. It should also be noted that, as used herein, the terms "substantially", "approximately" and other similar terms are used as approximate terms and not as degree terms, so that they are used to explain the inherent deviations of the measured values, calculated values ​​and / or provided values ​​that will be recognized by those of ordinary skill in the art.

[0032] For the convenience of description and to make the technical solutions of the specific implementation methods of the present disclosure easier to understand, before describing the data processing method implemented in the present disclosure, the technical terms involved in the specific implementation methods of the present disclosure are first introduced.

[0033] Collector Agent: A service provided by the cloud platform that can monitor the operating status of business systems deployed in the cloud platform and collect observation data.

[0034] Tenant: refers to an organization or individual that uses cloud services on a cloud platform. A tenant corresponds to an account on a cloud platform.

[0035] Enterprises usually deploy their business systems in cloud environments, such as public or private clouds, and manage and maintain business systems through cloud platforms. In order to monitor the operating status of business systems on the cloud in real time, the cloud platform can collect observation data through the deployed Collector Agent. As a tenant of the cloud platform, users can obtain and store the observation data collected by the Collector Agent, and then monitor the operation of the business system based on the observation data.

[0036] The user can configure a corresponding transmission channel to receive the observation data sent by the collection agent, and send the observation data to the designated storage location for storage. Since there is only one transmission channel configured, which is responsible for acquiring and transferring the observation data of all business systems, the large amount of observation data or other reasons may cause the node in the transmission channel to fail, resulting in the failure of observation data transmission. Observation data is continuously acquired and transferred, so this will affect the transmission of subsequent observation data. Before the fault is eliminated, the observation data cannot be acquired and stored normally, and even the observation data formed during the fault period will be lost, and the user will not be able to monitor the operation of the business system during the fault period.

[0037] To this end, the present disclosure proposes a data processing method.

[0038] Figure 1 The following is a schematic diagram of an application scenario of a data processing method according to some embodiments of the present disclosure. In this application scenario, a user terminal 10 and a cloud platform 20 may be included. For example, the following application scenario diagram is shown.

[0039] The client 10 can communicate with the cloud platform 20 to send and receive data or instructions. In the present disclosure, the client 10 includes at least one processor and at least one memory.

[0040] For example, the user terminal 10 may be a system or device running an enterprise-owned platform, and the cloud platform 20 can send the observed data of the business system deployed by the user to the user terminal 10 through the collection agent, so that the user terminal 10 can store the observed data and use the observed data to analyze the operation of the business system. Figure 1 The shapes and structures of the user terminal 10 and the cloud platform 20 shown in the figure should not be understood as limiting the scope of protection of the present disclosure.

[0041] Figure 2 FIG. 1 is a schematic diagram showing the overall flow of a data processing method M100 in some embodiments of the present disclosure. Figure 2 The method shown includes step S110, step S120 and step S130. The method can be executed by an electronic device such as a tablet computer or a desktop computer as an execution subject.

[0042] S110, receiving operation data collected from the business system, wherein the business system is deployed on a third-party platform.

[0043] Business systems can be used to manage and support business processes, for example, they can be used to provide customers with query, calculation or management services. The operation data of the business system (i.e. the above-mentioned observation data) is used to characterize the operation status of the business system. By analyzing the operation data, it can be determined whether there are any abnormalities in the business system, such as insufficient remaining storage space, excessive hardware resource utilization, etc.; and fault identification and system performance optimization can be carried out through analysis results, such as optimizing code logic and adjusting resource configuration by analyzing indicator data such as request response time and database query efficiency.

[0044] The same tenant can deploy multiple business systems on the same third-party platform. The same enterprise can register multiple accounts on the same third-party platform, that is, multiple tenants on the same third-party platform can only correspond to the same enterprise. In this case, the operating data recipients of the business systems of these different tenants are the same, for example, the operating data of the business systems of each tenant are received by the same data processing system.

[0045] The third-party platform can be a cloud platform, such as a public cloud or private cloud platform, or an external platform such as other hosting service providers or outsourcing service providers. The operation data is usually collected by the third-party platform where the business system is located. The third-party platform can collect the operation data of the tenant's (user's) business system through the configured collection agent service. The collection method can be based on the collection cycle set by the tenant or the collection cycle set by the platform itself by default.

[0046] For the same third-party platform, after completing the collection, the collection agent service can periodically send the operating data of the tenant's business system to the corresponding tenant. For the same enterprise, the received operating data can come from different third-party platforms and can be multiple business systems corresponding to different accounts (tenants).

[0047] S120, determining a data processing channel corresponding to the operating data through a group identifier carried in the operating data, wherein a mapping relationship exists between the group identifier and the data processing channel.

[0048] S130, sending the operating data to a designated storage location through the determined corresponding data processing channel for storage.

[0049] The collected operation data may include different types of data, and each type of data in the operation data may include multiple data (multiple data). The operation data includes a grouping identifier. The grouping identifier is added to the operation data before the business system sends the operation data outward.

[0050] The enterprise is configured with multiple different data processing channels locally or at other designated locations, and the data processing channels can replace the above-mentioned transmission channels for data transmission. Different data processing channels correspond to different transmission paths or transmission links or hardware devices used for data transmission. The group identifier is used to determine which data processing channel the operating data should be processed through. The data processing channel is used to receive the operating data of the corresponding group identifier and send the data to a designated storage location for storage. A designated storage location can correspond to a storage system. The storage system can be set in the enterprise's local, external platform or external server. Each data processing channel can send the operating data to the same storage system, and use the same storage system to store the operating data of all business systems of the tenant.

[0051] The grouping identifier can be divided according to the operation characteristics, relevance and other attributes of the business system. The operation data of business systems with certain relevance can use the same grouping identifier, or it can be set according to the number of business systems, that is, each business system uses a unique grouping identifier. In other words, the operation data of the same business system usually corresponds to the same grouping identifier.

[0052] After receiving the operation data sent by the third-party platform, the enterprise can use the first item of the operation data as the current data to be processed, extract the group identifier of the current data to be processed, determine the data processing channel of the current data to be processed, and then send the current data to the data processing channel. Then, the next item of data is used as the current data to be processed, and so on, until the group processing of all the operation data is completed.

[0053] It can be understood that the grouping identifier can be added in units of single data, that is, each data carries a grouping identifier; the grouping identifier can also be added in units of a group of data or a batch of data. For example, after the collection agent collects a batch of data, a data file is formed, which carries a grouping identifier.

[0054] Figure 3 Some embodiments of the present disclosure are shown. Figure 1 The data processing method M100 shown in FIG. Figure 3, P1, P2 and P3 are three different third-party platforms. P1 is a public cloud, and A1 is the collection agent of the public cloud. P2 is a private cloud, and A2 is the collection agent of the private cloud. P3 is a hosting platform. T can represent the enterprise (user), or a department of the enterprise or a working group of the enterprise. B11 and B12 are two different business systems deployed by user T in the public cloud P1, and B21 and B22 are two different business systems deployed by user T in the private cloud P2. D is a processing-side agent and is also the executor of the data processing method M100. R1~Rn are n different data processing channels configured locally by the tenant, and n>1. S is the storage system deployed by the tenant on the hosting platform P3.

[0055] After the collection agent A1 collects the operation data from the business systems B11 and B12, it will send it to the processing agent D. After the collection agent A2 collects the operation data from the business systems B21 and B22, it will also send it to the processing agent D. The processing agent D can be set on the gateway side of the enterprise to identify the group identifier on the gateway side. Whenever the processing agent D receives the operation data sent by any collection agent, it will extract the group identifier in the operation data, and then send each item of data or each group of data or each batch of data in the operation data to the corresponding data processing channel one by one according to the group identifier, and the data processing channel will transmit the data to the storage system S.

[0056] Assume that the operation data of B11 and B12 both carry the group identifier ID1, and the operation data of B21 and B22 both carry the group identifier ID2. The operation data carrying the group identifier ID1 is configured by the processing side agent D to send the data to the storage system S through the data processing channel R1, and the operation data carrying the group identifier ID2 is configured by the processing side agent D to send the data to the storage system S through the data processing channel R2. When the traffic of the business system B11 is large, resulting in a large amount of operation data, the data processing channel R1 fails to transmit, but does not affect the data processing channel R2 to transmit the operation data of B21 and B22.

[0057] According to the data processing method proposed in the embodiment of the present invention, a packet-based data transmission solution is proposed to address the problems of transmission failures and a wide range of impact from the collection side to the processing side. By adding a processing side agent, the business systems that need to be collected are grouped, and different groups correspond to their own data processing channels, thereby realizing group transmission of operating data. If a channel fails to transmit or fails due to excessive operating data traffic or the channel itself, it will only affect the business system operating data transmission through this channel, and the business system operating data transmission of other channels can proceed normally, avoiding the situation where all operating data cannot be transmitted due to failures.

[0058] Figure 4 FIG. 1 is a schematic diagram showing the overall flow of a data processing method M100 in some other embodiments of the present disclosure. Figure 4 The data processing method M100 may further perform the following step S101. Step S101 may be performed before step S110, or may be performed during the execution of steps S110 to S130.

[0059] S101, sending collection configuration information to a third-party platform, wherein the collection configuration information is used to instruct the third-party platform to add corresponding group identifiers to the operation data collected from the business system according to the specified business system group.

[0060] Business system groups are groups obtained by users dividing deployed business systems according to their own needs. A business system belongs to only one group. Each group has a unique group identifier, which is used to represent the business system group to which the operating data belongs.

[0061] Figure 5 Some embodiments of the present disclosure are shown. Figure 4 The data processing method M100 shown in FIG. Figure 5 , F1 represents the file storing the collection configuration information. After the business systems B11, B12, B21, and B22 are deployed and before they start running, users can log in to the third-party platforms P1 and P2 and perform the following operations through scripts: read the collection configuration information in the file F1, and perform collection configuration on the collection agents A1 and A2 of P1 and P2 according to the collection configuration information.

[0062] If business systems B11 and B12 belong to the same business system group, after pressing F1 for configuration, collection agent A1 will add the same group identifier to the operation data when collecting the operation data of business systems B11 and B12. If business systems B21 and B22 belong to different business system groups, after pressing F1 for configuration, collection agent A2 will add different group identifiers to the collected operation data of business system B21 and the operation data of business system B22.

[0063] It is understandable that the collection configuration information may include information such as the location of the operating data to be collected in the business system, a custom data classification label, and the name of the required data processing channel.

[0064] Exemplarily, the designated business system grouping may be obtained by dividing the business systems according to one or more of the following: business relevance, whether they belong to the same account of a third-party platform, flow of operating data, and resources required to process operating data.

[0065] Business relevance refers to the degree of relevance between business processes managed and supported by different business systems. For example, if an APP (application) launched by an enterprise is operated and maintained through business systems B11, B12 and B21, the three business systems B11, B12 and B21 can be divided into one business system group.

[0066] Business system groups can also be divided by tenants, for example Figure 3 and Figure 5 In the example, user T has an account in platform P1 and another account in platform P2. Then, two business system groups can be divided. The first group includes all business systems under the account of platform P1 (including B11 and B12), and the second group includes all business systems under the account of platform P2 (including B21 and B22). If there are multiple departments in the organizational structure of user T, and different departments are responsible for different business systems, when opening an account and deploying a business system, the business system of the same department can be deployed on the same third-party platform. At this time, it is equivalent to grouping business systems by department. One group corresponds to one department, and the business systems of the same department are in the same group.

[0067] The flow rate of running data can reflect the business volume of the business system. For business systems whose average business volume is significantly higher than that of other business systems, each of these business systems with significantly higher average business volume can be divided into a group separately, so that the data processing volume and throughput of each data processing channel can be more balanced.

[0068] In addition to data transmission, the data processing channel may also need to perform some logical processing on the operating data before transmission. For some business systems, the hardware resources required for processing their operating data are significantly higher than the operating data of other business systems. In this case, each business system in the business system corresponding to the operating data requiring higher hardware resources can be divided into a separate group. This can also make the data processing volume and throughput of each data processing channel more balanced.

[0069] If each business system has already started running, the user can dynamically modify the configuration information in file F1, and can log in to the platform at any time to start the script to modify the configuration. After the modification, the modified third-party platform will run the collection agent according to the modified configuration. In other words, the number of business system groups and the groups to which the business system belongs can be flexibly adjusted by modifying the collection configuration information, and the processing side agent also needs to be adjusted accordingly to adapt to the modified group identifier and group, which can further reduce the scope of impact when a failure occurs.

[0070] There may be a one-to-one mapping relationship between the group identifier and the data processing channel, that is, each data processing channel may only process and transmit the operation data of the same group identifier.

[0071] Operation data may include one or more of indicator data, log data, and link data. Among them, indicator data can be used to measure the specific values ​​of the performance and status of the business system, for example, it may include application indicators such as request response time, number of concurrent users, business throughput, and business indicators such as sales, order volume, and user activity. Log data can be used to record various information during the operation of the business system, including normal operations and abnormal events. Log data may include request logs such as request time, request method, and request path, as well as fault logs such as fault type, fault time, and fault stack when the system fails, and may also include operation logs, database logs, and other content. Link data can be used to track the flow path of a request or thing in the business system, for example, it may include information such as the call or operation ID, timestamp, label, time, etc.

[0072] Continue reading Figure 4 In step S110, the method of receiving the operation data collected from the business system may include at least one of step S111 and step S112.

[0073] S111, passively receiving the operation data collected from the business system and sent by the third-party platform.

[0074] S112, actively sending a data request to the third-party platform, and receiving the operation data collected from the business system and sent by the third-party platform in response to the data request.

[0075] Among them, step S111 is applicable to the sending of log data. The collection agent can actively send the collected log data to the processing agent. Step S112 can be applicable to the sending of all operation data. The processing agent can regularly send requests to the third-party platform, and the third-party platform will respond to the request and send the operation data collected during the period. Step S111 and step S112 can be selected and executed at the same time.

[0076] In actual operation, users may set up multiple storage systems to store observation data, and use different storage systems to store different types of observation data. In other words, there can be multiple designated storage locations. The enterprise will take out the operation data from the storage system for analysis, for example, to generate an indicator dashboard. However, the platform currently sends operation data (observation data) to the user in batches, and the same batch of data contains multiple different types of operation data, which is not conducive to the business side of the enterprise to directly analyze the data from its own storage system.

[0077] Figure 6 FIG. 1 is a schematic diagram showing the overall flow of a data processing method M100 in some other embodiments of the present disclosure. Figure 6 In step S130, the method of sending the operating data to the designated storage location for storage through the determined corresponding data processing channel may include the following steps S131, S132 and S133.

[0078] S131, sending the operating data to the determined corresponding data processing module.

[0079] S132, determining, by the data processing module, a designated storage location corresponding to the operating data according to the storage indication information.

[0080] S133, sending the operating data to a corresponding designated storage location for storage.

[0081] The operation data may also carry storage indication information. Each type of storage indication information corresponds to a storage system. The storage indication information is pre-configured and can be used to distinguish different types of operation data. The storage indication information may indicate the storage type in the form of a label. The collection configuration information may also be used to instruct the third-party platform to add corresponding storage indication information to the operation data collected from the business system according to the specified storage location grouping. The above-mentioned specified storage location grouping can be obtained by dividing according to the data type.

[0082] The transmission node of the data processing channel includes a data processing module. During the data processing process, the processing side agent sends the operating data to each data processing module, and the data processing module determines to which storage system the received data is sent for storage. The transmission link of "processing side agent-data processing module-storage system" forms the above-mentioned data processing channel. The data processing channel determined in step S120 only determines the data processing module. In step S131, the grouped operating data is sent to the corresponding data processing module. In step S132, the data processing module determines the designated storage location.

[0083] Assuming that a data processing module has determined multiple storage systems, the first item of data in the operating data can be used as the current data to be processed, the storage indication information of the current data to be processed can be extracted, the corresponding storage system of the current data to be processed can be determined, and then the current data to be processed can be sent to the storage system. After that, the next item of data can be used as the current data to be processed, and so on, until the reorganization of all operating data is completed. Alternatively, the received operating data of this batch can be divided according to the storage indication information to obtain multiple reorganized data sets, each of which corresponds to a storage system, and then these data sets are sent to the corresponding storage systems.

[0084] Through steps S131, S132 and S133, in order to solve the problem that batched operation data cannot be classified and transmitted in a finer granularity, the batched operation data is reorganized and classified, so that the data meeting the same label can be routed to the storage system to which it belongs after reorganization. Specifically, multiple pieces of data within the batched operation data can be reorganized according to the storage indication information (label), and after reorganization, subsequent batch transmission can be performed, thereby achieving the purpose of unified observation, storage and security isolation of the operation data.

[0085] In the above step S131, the method of determining the designated storage location corresponding to the running data according to the storage indication information through the data processing module may include the following steps: determining the target routing conditions satisfied by the running data according to the parameter information contained in the storage indication information and the designated routing conditions; and determining the designated storage location corresponding to the target routing conditions.

[0086] Figure 7 Some embodiments of the present disclosure are shown. Figure 6 The data processing method M100 shown in FIG. Figure 7 , there are m storage systems deployed on platform P3, namely S1~Sm, m>1. Parameter information can represent the parameters or attributes of operating data and business systems. For example, parameter information can include APP tags, namespaces or other parameters. The APP tag can represent the APP (application) that generates the operating data, and the namespace is the namespace corresponding to the operating data. The routing conditions can include the following two conditions: routing condition 1, if the APP tag is app1, then the specified storage location is determined to be storage system S1; routing condition 2, if the namespace is namespace2, then the specified storage location is determined to be storage system S2.

[0087] When the storage system is determined according to the above parameter information and the above routing conditions, for the operation data d1 in the data processing module R1, the parameter information of d1 is app1 and namespace1, and the determined target routing condition is routing condition 1, and the data d1 is stored in the storage system s1; for the operation data d2 in the data processing module R1, the parameter information of d1 is app2 and namespace2, and the determined target routing condition is routing condition 2, and the data d2 is stored in the storage system s2. Other situations can be deduced in the same way.

[0088] Exemplarily, the data processing method M100 may further perform the following step S102. Step S102 may be performed before step S110, or may be performed during the execution of steps S110 to S130. The execution order between step S101 and step S102 may be arbitrary.

[0089] S102, configuring the data processing channel to perform data division and data transmission according to the specified storage indication information.

[0090] Continue reading Figure 6 and Figure 7 , F2 represents a file storing routing configuration information, which includes routing conditions. After the deployment of business systems B11, B12, B21, and B22 is completed, before starting to run, users can perform the following operations through scripts: read the routing configuration information in file F2, perform routing configuration for data processing modules R1 to Rn according to the routing configuration information, so that the data processing modules can reorganize the operation data sent by the processing side agent D by type, so that the operation data with the same characteristics and meeting the same routing conditions are stored in the same storage system, so that the reorganized data meets the user's purpose of accurate data routing, and achieves more fine-grained storage, which is convenient for users to directly read the operation data from the storage system and generate indicator dashboards.

[0091] Based on any of the above implementations, the present disclosure also provides a data processing device. Figure 8 FIG. 1 is a schematic block diagram of the structure of a data processing device according to an embodiment of the present disclosure. Figure 8 As shown, the data processing device includes: a data receiving module 1002 , a channel determining module 1004 , and a data transmission module 1006 .

[0092] The data receiving module 1002 is used to receive the operation data collected from the business system, wherein the business system is deployed on a third-party platform.

[0093] The channel determination module 1004 is used to determine the data processing channel corresponding to the operation data according to the group identifier carried in the operation data.

[0094] The data transmission module 1006 is used to send the operation data to a designated storage location for storage through the determined corresponding data processing channel.

[0095] Exemplarily, the operation data may also carry storage indication information. The data transmission module 1006 may include the above-mentioned data processing module. The data transmission module 1006 sends the operation data to the determined corresponding data processing module, and the data processing module determines the designated storage location corresponding to the operation data according to the storage indication information, and sends the operation data to the corresponding designated storage location for storage.

[0096] The data processing device can be in the form of computer software, and each module of the data processing device can be implemented by a computer software module. The implementation process of the functions and effects of each module in the device is specifically described in the implementation process of the corresponding steps in the method, which will not be repeated here.

[0097] The execution subject of the data processing method in the specific implementation manner of the present disclosure may be an electronic device such as a tablet computer, a desktop computer, a server, etc.

[0098] Therefore, based on any one of the above embodiments, the present disclosure further provides an electronic device, which can execute the data processing method of any one of the embodiments described above in the present disclosure.

[0099] Fig. 9 FIG. 1 is a schematic block diagram of the structure of an electronic device 1000 according to an embodiment of the present disclosure. Fig. 9 , the hardware structure of the electronic device 1000 can be implemented using a bus architecture. The bus architecture can include any number of interconnecting buses and bridges, depending on the specific application and overall design constraints of the hardware. The bus 1100 connects various circuits including one or more processors 1200, memory 1300 and / or hardware modules together. The bus 1100 can also connect various other circuits 1400 such as peripherals, voltage regulators, power management circuits, external antennas, etc.

[0100] The bus 1100 may be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Component (EISA) bus, etc. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, the figure only uses one connecting line, but does not mean that there is only one bus or one type of bus.

[0101] The processor 1200 may be a central processing unit (CPU). The processor 1200 may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, or a combination of the above chips.

[0102] The memory 1300 can be used as a non-transitory computer-readable storage medium, and can be used to store non-transitory software programs, non-transitory computer executable programs and modules, such as program instructions of the computer program in the embodiment of the present disclosure. The processor 1200 implements the data processing method by running the non-transitory software programs, instructions and modules stored in the memory 1300.

[0103] The memory 1300 may include a program storage area and a data storage area, wherein the program storage area may store an operating system and an application required by at least one function; the data storage area may store data created by the processor 1200, such as structural data, model data, sample data, stiffness evaluation results, etc. of the bridge. In addition, the memory 1300 may include a high-speed random access memory, and may also include a non-transitory memory, such as at least one disk storage device, a flash memory device, or other non-transitory solid-state storage device. In some embodiments, the memory 1300 may optionally include a memory remotely arranged relative to the processor 1200, and these remote memories may be connected to the processor 1200 via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0104] The present disclosure also provides a readable storage medium, in which a computer program is stored, and the computer program is used to implement the above method when executed by a processor. "Readable storage medium" can be any device that can contain, store, communicate, propagate or transmit a program for use in an instruction execution system, device or equipment or in combination with these instruction execution systems, devices or equipment. More specific examples of readable storage media include the following: an electrical connection portion (electronic device) with one or more wirings, a portable computer disk box (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and editable read-only memory (EPROM or flash memory), an optical fiber device, and a portable read-only memory (CDROM), etc.

[0105] The present disclosure also provides a computer program product, and the method of the present disclosure can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer programs or instructions. When the computer program or instruction is loaded and executed, the process or function of the present disclosure is executed in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, a network device, a user device, a core network device, an OAM or other programmable device.

[0106] The computer program or instructions may be stored in a readable storage medium or transmitted from one readable storage medium to another readable storage medium, for example, the computer program or instructions may be transmitted from one website, computer, server or data center to another website, computer, server or data center by wired or wireless means. The readable storage medium may be any available medium that can be accessed or a data storage device such as a server, data center, etc. that integrates one or more available media. The available medium may be a magnetic medium, such as a floppy disk, a hard disk, or a magnetic tape; it may also be an optical medium, such as a digital video disk; it may also be a semiconductor medium, such as a solid state drive. The computer readable storage medium may be a volatile or non-volatile storage medium, or may include both volatile and non-volatile types of storage media.

[0107] Those skilled in the art will appreciate that the embodiments of the present disclosure may be provided as methods, systems, or computer program products. Therefore, the present disclosure may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present disclosure may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0108] The present disclosure is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to the present disclosure. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0109] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0110] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0111] In the description of this specification, the description with reference to the terms "one embodiment / method", "some embodiments / methods", "example", "specific example", or "some examples" etc. means that the specific features, structures, or characteristics described in conjunction with the embodiment / method or example are included in at least one embodiment / method or example of the present disclosure. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment / method or example. Moreover, the specific features, structures, or characteristics described may be combined in any one or more embodiments / methods or examples in a suitable manner. In addition, those skilled in the art may combine and combine different embodiments / methods or examples described in this specification and features of different embodiments / methods or examples, unless they are contradictory.

[0112] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of the features. In the description of the present disclosure, "plurality" means at least two, such as two, three, etc., unless otherwise clearly and specifically defined.

[0113] Those skilled in the art should understand that the above embodiments are only for the purpose of clearly illustrating the present disclosure, and are not intended to limit the scope of the present disclosure. For those skilled in the art, other changes or modifications may be made based on the above disclosure, and these changes or modifications are still within the scope of the present disclosure.

Claims

1. A data processing method, characterized in that: include: Receiving operation data collected from a business system, where the business system is deployed on a third-party platform; Determining a data processing channel corresponding to the operation data by means of a group identifier carried in the operation data, wherein a mapping relationship exists between the group identifier and the data processing channel; as well as The operation data is sent to a designated storage location through the determined corresponding data processing channel for storage.

2. The data processing method according to claim 1, characterized in that: The method further comprises: The collection configuration information is sent to the third-party platform, where the collection configuration information is used to instruct the third-party platform to add corresponding group identifiers to the operation data collected from the business system according to the specified business system group.

3. The data processing method according to claim 2, characterized in that: The designated business system grouping is obtained by dividing the business systems according to one or more of the business relevance, whether they belong to the same account of the third-party platform, the flow of the operation data, and the resources required to process the operation data.

4. The data processing method according to any one of claims 1 to 3, characterized in that: There is a one-to-one mapping relationship between the group identifier and the data processing channel.

5. The data processing method according to claim 1, characterized in that: The operation data includes one or more of indicator data, log data, and link data.

6. The data processing method according to claim 5, characterized in that: Receiving the operation data collected from the business system, specifically including at least one of the following methods: Method 1: passively receiving the operation data collected from the business system and sent by the third-party platform; Method 2 is to actively send a data request to the third-party platform, and receive the operation data collected from the business system and sent by the third-party platform in response to the data request.

7. The data processing method according to claim 1 or 2, characterized in that: There are multiple designated storage locations, the operation data also carries storage indication information, the transmission node of the data processing channel includes a data processing module, and the operation data is sent to the designated storage location through the determined corresponding data processing channel for storage, including: Sending the operating data to the determined corresponding data processing module; Determining, by the data processing module, a designated storage location corresponding to the operating data according to the storage indication information; and Sending the operating data to the corresponding designated storage location for storage; Optionally, determining, by the data processing module, a designated storage location corresponding to the operating data according to the storage indication information includes: Determining a target routing condition satisfied by the operation data according to the specified routing condition and the parameter information included in the storage indication information; and Determining a designated storage location corresponding to the target routing condition; Optionally, the method further comprises: The data processing channel is configured to perform data division and data transmission according to the specified storage indication information.

8. An electronic device, characterized in that: include: A memory storing execution instructions; as well as A processor, wherein the processor executes the execution instructions stored in the memory, so that the processor executes the method according to any one of claims 1 to 7.

9. A readable storage medium, characterized in that: The readable storage medium stores a computer program, and when the computer program is executed by a processor, it is used to implement the method according to any one of claims 1 to 7.

10. A computer program product, characterized in that The computer program product comprises a computer program, and when the computer program is executed by a processor, the computer program is used to implement the method according to any one of claims 1 to 7.

Images