Data processing method and management platform
By obtaining and utilizing the control level of data sets on the management platform and controlling change requests, the problems of large labor costs and low management efficiency in data set management and approval in the existing technology are solved, and more efficient data change management is achieved.
Patent Information
- Application Number
- CN202411828438.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-12-11
- Publication Date
- 2025-04-11
AI Technical Summary
In the prior art, when managing data sets in production environments, administrator approval requires a large labor cost and low management efficiency, especially in the management of critical data sets and non-critical data sets.
By implementing data processing methods on the management platform, the control level of the downstream data set of the target data set is obtained, and the control of change requests is carried out based on this level, such as strict approval of critical services, and loosening control of non-critical services.
While ensuring the security and stability of key business, this method saves manpower and improves the efficiency of data change.
Smart Images
Figure CN119807273A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of computer technology, and in particular to a data processing method and management platform. Background Art
[0002] With the development of information technology (IT), more and more data needs to be managed, such as the management of big data (BD). Data management includes operations such as data set storage and change. Among them, data sets can be divided into data sets in the development environment (development) and data sets in the production environment (production). For data sets in the development environment (development), users can change them freely. Data sets in the production environment involve external services, and changes require administrator approval to achieve change control.
[0003] The data sets in the production environment include key data sets and non-key data sets. In the related art, both key data sets and non-key data sets are approved by administrators, which leads to high manpower costs and low management efficiency. Summary of the invention
[0004] This application provides a data processing method and management platform, which can save manpower expenses in data change management and improve management efficiency.
[0005] In a first aspect, a data processing method is provided, which is applied to a management platform, and the management platform is used to manage multiple data sets, wherein each of the multiple data sets records at least one data, and the data set has a control level; the method includes: obtaining a change request, the change request is used to request to perform a change operation on a target data set among the multiple data sets; obtaining the control level of each data set in at least one of the multiple data sets; wherein all or part of the data recorded in each of the at least one data set comes from the target data set; based on the control level of each of the at least one data set, the change request is controlled. The at least one data set is a downstream data set of the target data set. The control level of a data set reflects the criticality or importance of the business involved in the data set.
[0006] In this method, when the target data set is about to be changed, the control level of the downstream data set of the target data set is obtained, and then the change to the target data set is controlled by using the control level of the downstream data set. In this way, when the business involved in the downstream data set is more critical, the change request can be more strictly controlled (for example, administrator approval is required), and when the business involved in the downstream data set is not critical, the change request is not controlled or is loosely controlled (for example, administrator approval is not required), thereby ensuring the security and stability of key businesses while saving manpower and improving the efficiency of data changes.
[0007] In one possible implementation, based on the control level of each data set in at least one data set, controlling change requests includes: identifying a first data set in at least one data set; wherein the control level of the first data set is greater than or equal to a threshold; issuing a change control request to a control user of the first data set; the change control request is used to request the control user to make a change indication in response to the change request; receiving the change indication issued by the control user; and allowing or rejecting the change request based on the change indication.
[0008] In this implementation, when the business involved in the downstream data set is critical, the management and control user of the data set can be requested to approve the change request, thereby further ensuring the security and stability of the critical business.
[0009] In a possible implementation, the change control request includes the impact of the change operation on the data in the first data set.
[0010] In this implementation, the management platform can evaluate the impact of the change operation requested by the change request on the data in the first data set, and carry the impact into the change control request. In this way, the control user does not need to evaluate the impact of the change operation requested by the change request on the data in the first data set, which reduces the work of the control user and improves the user experience.
[0011] In a possible implementation, based on the control level of each data set in at least one data set, the change request is controlled, including: when the control level of each data set in at least one data set is less than a threshold, allowing the change request.
[0012] In this implementation, the control level of each data set in the at least one data set is less than the threshold, indicating that the business that can be affected by the change request is not a critical business. In this case, the management platform can directly allow the change request, thereby improving the change efficiency.
[0013] In a possible implementation, the method further includes: if the change request is allowed, saving a backup of the target data set and at least one data set; performing a change operation on the target data set to obtain a change result; and when the change result does not meet the requirements, restoring the target data and at least one data set using the backup.
[0014] In this implementation, a full backup of the data set affected by the change request can be performed. When the change result does not meet expectations (for example, it affects the security or stability of the business), it can be restored in time to ensure the security and stability of the business.
[0015] In a second aspect, a management platform is provided, wherein the management platform is used to manage multiple data sets, wherein each of the multiple data sets records at least one data, and the data sets have a management level; the management platform includes: a first acquisition module, used to obtain a change request, the change request is used to request a change operation to be performed on a target data set among the multiple data sets; a second acquisition module, used to obtain the management level of each data set in at least one of the multiple data sets; wherein all or part of the data recorded in each data set in the at least one data set comes from the target data set; a management module, used to manage the change request based on the management level of each data set in the at least one data set.
[0016] In one possible implementation, the management and control module is used to: identify a first data set in at least one data set; wherein the management level of the first data set is greater than or equal to a threshold; issue a change management request to a management user of the first data set; the change management request is used to request the management user to make a change indication in response to the change request; receive the change indication issued by the management user; and allow or reject the change request based on the change indication.
[0017] In a possible implementation, the change control request includes the impact of the change operation on the data in the first data set.
[0018] In a possible implementation, the control module is configured to: allow a change request when a control level of each data set in at least one data set is less than a threshold.
[0019] In a possible implementation, the management and control module is also used to: save a backup of the target data set and at least one data set when the change request is allowed; perform a change operation on the target data set to obtain a change result; and when the change result does not meet the requirements, restore the target data and at least one data set using the backup.
[0020] In a third aspect, a computing device cluster is provided, comprising at least one computing device, each computing device comprising a processor and a memory; the processor of at least one computing device is used to execute instructions stored in the memory of at least one computing device, so that the computing device cluster executes the method provided in the first aspect.
[0021] In a fourth aspect, a computer-readable storage medium is provided, comprising computer program instructions. When the computer program instructions are executed by a computing device cluster, the computing device cluster executes the method provided in the first aspect.
[0022] In a fifth aspect, a computer program product comprising instructions is provided. When the instructions are executed by a computer device cluster, the computer device cluster executes the method provided in the first aspect.
[0023] The beneficial effects of the second to fifth aspects can be referred to the above introduction to the beneficial effects of the first aspect, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS
[0024] Figure 1 It is a schematic diagram of a system architecture provided by an embodiment of the present application;
[0025] Figure 2 is a schematic diagram of a management platform provided in an embodiment of the present application;
[0026] Figure 3 is a schematic diagram of a relationship between data sets provided in an embodiment of the present application;
[0027] Figure 4 It is a schematic diagram of a control table provided in an embodiment of the present application;
[0028] Figure 5 is a schematic diagram of a data set change provided in an embodiment of the present application;
[0029] Figure 6 is a schematic diagram of a data set change provided in an embodiment of the present application;
[0030] Figure 7 is a schematic diagram of a data set recovery provided by an embodiment of the present application;
[0031] Figure 8 is a schematic diagram of a data processing method provided in an embodiment of the present application;
[0032] Fig. 9 is a flow chart of a data processing method provided in an embodiment of the present application;
[0033] Fig.10 A schematic diagram of the structure of a management platform provided in an embodiment of the present application;
[0034] Fig.11 A schematic diagram of the structure of a computing device provided in an embodiment of the present application;
[0035] Fig.12 A schematic diagram of the structure of a computing device cluster provided in an embodiment of the present application;
[0036] Fig.13 A schematic diagram of the structure of a computing device cluster provided in an embodiment of the present application. DETAILED DESCRIPTION
[0037] The scheme provided by the embodiment of the present application will be described below in conjunction with the accompanying drawings. In the embodiment of the present application, "multiple" means two or more, and "multiple" means two or more. "First", "second", etc. are only used to distinguish similar objects and are not necessarily used to describe a specific order or number of objects.
[0038] To facilitate understanding of the solutions provided by the embodiments of the present application, the technical terms that may be involved in the embodiments of the present application are first introduced.
[0039] Management platform: A platform provided by resource providers for interaction with users. Users can register accounts on the management platform and use their accounts to rent resources in the infrastructure managed by the management platform. Users can store data in the infrastructure through the management platform. In cloud technology, the management platform can be called a cloud management platform.
[0040] Infrastructure: It is a facility that provides computing resources, storage resources and / or network resources for services. An infrastructure may include at least one data center, and each data center includes multiple servers. The service may be provided by an application (APP) deployed in the infrastructure, and the server provides resources for the operation of the application deployed in the infrastructure. The application can be deployed directly in the server or in a computing instance in the server. In cloud technology, infrastructure is also called cloud infrastructure, which is used to provide resources for cloud computing services.
[0041] Dataset: Also called production resources or business resources, it refers to a collection of multiple data related to production or business. Common data sets include tables and jobs.
[0042] Table: It can also be called a data table, which is used to record data. Tables can be divided into horizontal tables and vertical tables.
[0043] Job: Also known as an occupation task, it is the basic unit of work submitted by a user to the system. It is the work and task that a user requires a computer to do during a transaction or calculation. A job can consist of multiple job stages, which are also called job nodes.
[0044] Development environment: refers to the environment used by developers during the requirements development phase.
[0045] Production environment: refers to the environment where external services are formally provided.
[0046] Control level: also known as business level, refers to the criticality of a data set. A data set corresponds to a business, and the execution of the business requires the data recorded in the data set. The importance of the business corresponding to the data set is positively correlated with the control level of the data set. For example, the control level can be divided into high level and low level.
[0047] Data lineage: also known as data lineage, data origin, and data genealogy, refers to the natural relationship between data during its entire life cycle, from generation, processing, processing, fusion, and circulation to final extinction. It records the link relationship of data generation. Data lineage analysis is one of the important applications of metadata management. It sorts out the relationship between systems, tables, views, stored procedures, ETL (extract-transform-load), program codes, fields, etc., and uses graph databases for visualization. Simply put, it is to show how the data came from, what processes, stages, and calculation logic it has gone through through visualization.
[0048] Downstream dataset: refers to a dataset that is downstream of a dataset with data lineage. Suppose the downstream dataset of dataset A1 includes dataset A2, and part or all of the data in dataset A2 comes from dataset A1, or in other words, dataset A2 includes one or more data in dataset A1.
[0049] In related technologies, when a dataset in a production environment needs to be changed, it is submitted for approval, and the approver determines whether the change to the dataset is allowed. In this solution, each change to a dataset requires approval, which makes the change inefficiency low. In addition, when making relevant judgments, the approver needs to consider the impact of the change to the dataset on downstream businesses, which requires the approver to have rich knowledge and experience. In addition, it is difficult to quickly roll back changes to the dataset. If the change results affect downstream businesses, it will take a lot of time to eliminate the impact.
[0050] The embodiment of the present application provides a data processing method. When a target data set is to be changed, the control level of the downstream data set of the target data set is obtained, and then the change to the target data set is controlled by using the control level of the downstream data set. For example, when the control level of the downstream data set is high, the change to the target data set is approved; when the control level of the downstream data set is low, the change to the target data set is not approved. Thus, the efficiency of data change is improved.
[0051] Next, the data processing method provided in the embodiment of the present application is described.
[0052] Figure 1 A system architecture provided by an embodiment of the present application and which can be used to implement the method is shown. The system architecture includes a management platform 110 , an infrastructure 120 , and a user 130 .
[0053] The user 130 may be a data development engineer or other types of users.
[0054] The infrastructure 120 stores multiple data sets, each of which may include one or more data. For example, a data set may be a data table, such as a horizontal table, and a row of data recorded in the data table may be referred to as one data. For example, a data set may be a job, and a job node in the job may be referred to as one data.
[0055] The management platform 110 may manage multiple data sets stored in the infrastructure 120 . For example, in response to a change request issued by a user 130 , a change operation may be performed on a related data set among the multiple data sets.
[0056] The management platform 110 may be any module, component, device, equipment, cluster or platform with data processing capabilities.
[0057] In some embodiments, Figure 2 As shown, the management platform 110 may include a data set identification module 111 , a change diagnosis module 112 , and a change control module 113 .
[0058] Among them, the data set identification module 111 can provide a control level marking capability, and the user 130 can mark the control level of the data set through the data set identification module 111. Exemplarily, the user 130 can set and mark the control level of the data set based on the importance of the business corresponding to the data set. Exemplarily, the user 130 can associate the data set with one or more users through the data set identification module 111, wherein the one or more users are the control users of the data set. For example, the data recorded in the data set is the employee information of a department, and the control user of the data set can be the manager or human resources (HR) of the department.
[0059] The data set identification module 111 can associate a data set with a downstream data set of the data set. Exemplarily, the data set can be associated with the control level of the downstream data set of the data set. In one example, when there are multiple downstream data sets of a data set, the data set with the highest control level among the multiple downstream data sets can be identified, and the control level of the downstream data set with the highest control level can be associated with the data set. In one example, when there are multiple downstream data sets of a data set, the data set with the highest control level among the multiple downstream data sets can be identified, and the control user of the downstream data set with the highest control level can be associated with the data set.
[0060] In one example, the association between a data set and its downstream data set can be recorded in a control table. In one example, the data set can be set to table T1, the name of table T1 is T_abc, and T1 is the identity (ID) of the table. Figure 3 , the downstream data sets of table T1 include table T2, table T3, table T4, etc. It can also be set that among tables T2, T3, T4, etc., table T2 has the highest control level, and the control level of table T2 is high, and the control user of table T2 is user 21. Then the relevant information of table T1 in the control table can be as follows Figure 4 As shown. The control table is a horizontal table, which can record the name of table T1, the ID of the downstream data set, the control level of the downstream data set, the control user of the downstream data set, etc. in the same row to record the association between table T1 and its downstream data sets. Figure 3 and Figure 4 X represents the ID of other datasets, XXX represents the ID of downstream datasets or upstream datasets, and XX represents other types. Table T1 can also be called dataset T1, table T2 can also be called dataset T2, table T3 can also be called dataset T3, and table T4 can also be called dataset T4.
[0061] In an example, the association between a data set and its upstream data set may be recorded in a management table, for example, the association between a data set and its upstream data set ID and a management user may be recorded in a management table.
[0062] The change diagnosis module 112 can manage and control change requests. The user 130 can issue a change request for one or some data sets in the infrastructure 120. For the convenience of description, the data set targeted by the change request is referred to as the target data set. Among them, the change request is used to request a change operation to be performed on the target data set. Exemplarily, the change operation can be an operation such as modification, deletion, or writing. Exemplarily, the change request can include information such as change scripts, tables, and fields. The change-targeting module 112 can receive a change request, and then use the control level of the downstream data set of the target data set to manage the change request. For example, the change-targeting module 112 can diagnose the change requested by the change request, the degree of impact on all related production businesses of the downstream data sets of the target data set, and give detailed impact points of the production business, and give change control suggestions, such as which data sets need to be reviewed by the control users, and the expected change impact points on the production business.
[0063] In one example, see Figure 5 The change diagnosis module 112 can analyze the impact of the change operation requested by the change request on the downstream data set of the target data set, and formulate relevant management and control strategies based on the impact, for example, specifying a management and control user to approve the change request, and providing the management and control user with the impact of the change operation requested by the change request on the business.
[0064] The change control module 113 may include a backup of the target dataset and the downstream datasets of the target dataset when the change request is allowed, and the backup is used to restore the target dataset and the downstream data of the target dataset to the state before the change when the change result of the change request does not meet the requirements. Exemplarily, when the control level of at least one downstream dataset of the target dataset is greater than or equal to the threshold, before executing the change operation requested by the change request, the relevant backup is executed. When the control levels of all downstream datasets of the target dataset are less than the threshold, before executing the change operation requested by the change request, the relevant backup is not executed.
[0065] In some embodiments, the change control module 113 can adopt different change control strategies for different control levels of downstream data sets of the target data set, so as to strictly control the changes of target data with high control levels of downstream data sets. Among them, the version of the target data before the change can be saved and executed synchronously, and the full backup B of the target data set before the change is output, and the script after the change is executed to obtain the change result A, so as to facilitate the rapid recovery of subsequent problems. No change control restrictions are imposed on non-production businesses.
[0066] In one example, you can set the change results and backup as follows Figure 6 As shown in , when the change result does not meet the requirements, Figure 7As shown, the backup can be used for restoration, thereby achieving rapid rollback of changes.
[0067] In some embodiments, the backup has a life cycle, and the length of the life cycle can be set according to the required preservation time of the backup.
[0068] The above example introduces a system architecture provided by an embodiment of the present application. Next, in combination with the system architecture, the data processing method provided by an embodiment of the present application is introduced.
[0069] In some embodiments, Figure 8 As shown, this method can be divided into three stages, namely before change, during change and after change.
[0070] In the pre-change stage, user 130 can create a data set, configure the control level, control user, etc. of the data set, and identify the upstream and downstream data sets of the data set, and then record the control level, control user, and upstream and downstream data sets of the data set into the control table. In an example, user 130 can be a data development engineer. Before the change, the data set identification module 111 can generate a production environment determination result table, upstream and downstream data sets, etc. through the data lineage information and the production environment and control user tags of each data development engineer.
[0071] In the pre-change stage, user 130 can issue a change request, and can use the change diagnosis module 112 to obtain the control strategy of the change request by configuring the control level, control users, etc. In one example, user 130 can be a data development engineer. Before the change, the data development engineer provides the modified script / job based on the business intent, and the change diagnosis module 112 gives the control level and different process requirements of the change based on the production environment judgment result table. For example: the change request of the data set in the development environment can be executed directly, and the core high-risk change can only be executed after the unanimous consent of the stakeholders (other approval flows can be set according to the business control process). In one example, user 130 can be a data development engineer. Before the change, the data development engineer provides the modified script / job content based on the business intent. The change diagnosis module 112 simulates the impact of the modification content on the downstream business based on the modification point, and gives the metadata change point, data volume change point and other content, and gives risk assessment suggestions based on business rules.
[0072] In the change stage, the change strategy formulated before the change can be used to execute the change operation requested by the change request to obtain the change result. The user 130 can issue a backup instruction. In response to the backup instruction, the backup operation can be executed to obtain data backup. When changing, the change control module 113 automatically performs a full backup of the change content, including scripts, jobs, etc., and executes the script and job content before the change to perform a full backup of the data.
[0073] In the post-change stage, user 130 can verify the change results and perform post-change management. If the change results meet the requirements, the change is considered successful and the change results are retained. If the change results do not meet the requirements, the change is considered failed and the data backup is used for recovery. Exemplarily, user 130 can be a data development engineer. Exemplarily, if the change results do not meet the requirements, it means that the change has unexpected behavior, and it will be automatically rolled back, and the backup script, job and backup data will be replaced in full; if the change does not have any unexpected new behavior, the data backup can be retained for a period of time and then deleted.
[0074] Next, the process of the data processing method provided by the embodiment of the present application is introduced. The method is specifically executed by the management platform 110. The management platform 110 is used to manage multiple data sets, wherein each of the multiple data sets records at least one data, and the data sets have a management level. The multiple data sets may be data sets stored by the infrastructure 120. Fig. 9 As shown, the method includes the following steps.
[0075] Step 901 : The management platform 110 obtains a change request, where the change request is used to request to perform a change operation on a target data set among the multiple data sets.
[0076] The change request may be issued by user 130. User 130 may request to change the target data set through the change request. The target data set refers to the data set to which the change request is directed. In one example, the change operation may be a modification operation. In one example, the change operation may be a write operation. In one example, the change operation may be a delete operation.
[0077] In step 902 , the management platform 110 obtains the control level of each data set in at least one data set among the multiple data sets; wherein all or part of the data recorded in each data set in the at least one data set comes from the target data set.
[0078] In some embodiments, the control level of a data set may be configured by the developer, manager, or creator of the data set. For details, please refer to the above description and will not be repeated here.
[0079] The at least one data set is a downstream data set of the target data set. In some embodiments, the at least one data set is all downstream data sets of the target data set. In some embodiments, the at least one data set is a data set whose control level is greater than or equal to a threshold value among the downstream data sets of the target data set.
[0080] In some embodiments, the threshold value may be pre-set according to the classification of the control level. For example, the control level may be divided into high, medium, and low, and the threshold value may be set to high or medium.
[0081] Step 903 : The management platform 110 controls the change request based on the control level of each data set in the at least one data set.
[0082] The management platform 110 may reject or allow the change request based on the control level of each data set in the at least one data set.
[0083] In some embodiments, in step 903, the management platform 110 may first identify a first data set in the at least one data set, and the control level of the first data set is greater than or equal to the above threshold. Exemplarily, the first data set may be the data set with the highest control level in the at least one data set. Then, the management platform 110 may issue a change control request to the control user of the first data set, and the change control request is used to request the control user to make a change instruction for the change request. Among them, the change instruction indicates to allow or reject the change request. The control user of the data set may make a change instruction based on the impact of the change request on the data set. The management platform 110 receives the change instruction issued by the control user, and allows or rejects the change request based on the change instruction. Among them, when the change instruction indicates to allow the change request, the control platform 110 allows the change request. When the change instruction indicates to reject the change request, the control platform 110 rejects the change request.
[0084] In an example of this embodiment, the change control request includes the impact of the change operation on the data in the first data set. In this example, the management platform 110 can evaluate the impact of the change operation requested by the change request on the data in the first data set, and carry the impact in the change control request. In this way, there is no need for the control user to evaluate the impact of the change operation requested by the change request on the data in the first data set, which reduces the work of the control user and improves the user experience.
[0085] In some embodiments, the management platform 110 may allow the change request when the control level of each data set in the at least one data set is less than the above threshold. The fact that the control level of each data set in the at least one data set is less than the above threshold indicates that the importance of the business that can be affected by the change request is relatively low. In this case, the management platform 110 may directly allow the change request, thereby improving the change efficiency.
[0086] In some embodiments, the management platform 110 may also save a backup of the target data set and the at least one data set when the change request is allowed; then, the management platform 110 performs the change operation on the target data set to obtain a change result; when the change result does not meet the requirements, the backup is used to restore the target data set and the at least one data set. In this embodiment, a full backup of the data set affected by the change request can be performed, and when the change result does not meet expectations (for example, affects the security or stability of the business), it can be restored in time to ensure the security and stability of the business.
[0087] In summary, the data processing method provided in the embodiment of the present application uses the control level of the downstream data set of the changed target data set to control the change, thereby ensuring the security and stability of key businesses and improving the efficiency of data changes.
[0088] Based on the above description, the embodiment of the present application further provides a management platform 1000. The management platform 1000 is used to manage multiple data sets, wherein each of the multiple data sets records at least one data, and the data sets have a management level. Fig.10 As shown, the management platform 1000 includes:
[0089] A first acquisition module 1010 is used to acquire a change request, where the change request is used to request to perform a change operation on a target data set among the multiple data sets;
[0090] A second acquisition module 1020 is used to acquire the control level of each data set in at least one data set among the multiple data sets; wherein all or part of the data recorded in each data set in the at least one data set comes from the target data set;
[0091] The control module 1030 is configured to control the change request based on the control level of each data set in the at least one data set.
[0092] Among them, the first acquisition module 1010, the second acquisition module 1020 and the control module 1030 can all be implemented by software, or can be implemented by hardware. Exemplarily, the implementation of the first acquisition module 1010 is introduced below by taking the first acquisition module 1010 as an example. Similarly, the implementation of the second acquisition module 1020 and the control module 1030 can refer to the implementation of the first acquisition module 1010.
[0093] As an example of a software functional unit, the first acquisition module 1010 may include code running on a computing instance. Among them, the computing instance may include at least one of a physical host (computing device), a virtual machine, and a container. Further, the above-mentioned computing instance may be one or more. For example, the first acquisition module 1010 may include code running on multiple hosts / virtual machines / containers. It should be noted that the multiple hosts / virtual machines / containers used to run the code can be distributed in the same region or in different regions. Furthermore, the multiple hosts / virtual machines / containers used to run the code can be distributed in the same availability zone AZ or in different AZs, and each AZ includes a data center or multiple data centers with close geographical locations. Among them, usually a region can include multiple AZs.
[0094] Similarly, multiple hosts / virtual machines / containers used to run the code can be distributed in the same VPC or in multiple VPCs. Usually, a VPC is set up in a region. For cross-region communication between two VPCs in the same region and between VPCs in different regions, a communication gateway needs to be set up in each VPC to achieve interconnection between VPCs through the communication gateway.
[0095] As an example of a hardware functional unit, the first acquisition module 1010 may include at least one computing device, such as a server, etc. Alternatively, the first acquisition module 1010 may also be a device implemented using an application-specific integrated circuit (ASIC) or a programmable logic device (PLD). The PLD may be a complex programmable logical device (CPLD), a field programmable gate array (FPGA), a generic array logic (GAL) or any combination thereof.
[0096] The multiple computing devices included in the first acquisition module 1010 can be distributed in the same region or in different regions. The multiple computing devices included in the first acquisition module 1010 can be distributed in the same AZ or in different AZs. Similarly, the multiple computing devices included in the first acquisition module 1010 can be distributed in the same VPC or in multiple VPCs. The multiple computing devices can be any combination of computing devices such as servers, ASICs, PLDs, CPLDs, FPGAs, and GALs.
[0097] It should be noted that, in other embodiments, the first acquisition module 1010 may be used to execute Fig. 9 In any step of the method shown, the second acquisition module 1020 can be used to perform Fig. 9 In any step of the method shown, the control module 1030 can be used to perform Fig. 9 The steps implemented by the first acquisition module 1010, the second acquisition module 1020 and the control module 1030 can be specified as needed, and the first acquisition module 1010, the second acquisition module 1020 and the control module 1030 can be respectively implemented Fig. 9 The different steps in the method shown are used to implement the overall functions of the management platform 1000 .
[0098] The present application also provides a computing device 1100. Fig.11 As shown, the computing device 1100 includes: a bus 1102, a processor 1104, a memory 1106, and a communication interface 1108. The processor 1104, the memory 1106, and the communication interface 1108 communicate through the bus 1102. The computing device 1100 can be a server or a terminal device. It should be understood that the present application does not limit the number of processors and memories in the computing device 1100.
[0099] The bus 1102 may be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Fig.11 The bus 1102 may include a path for transmitting information between various components of the computing device 1100 (eg, the memory 1106, the processor 1104, and the communication interface 1108).
[0100] The processor 1104 may include any one or more of a central processing unit (CPU), a graphics processing unit (GPU), a microprocessor (MP), or a digital signal processor (DSP).
[0101] The memory 1106 may include a volatile memory, such as a random access memory (RAM). The memory 1106 may also include a non-volatile memory, such as a read-only memory (ROM), a flash memory, a hard disk drive (HDD), or a solid state drive (SSD).
[0102] The memory 1106 stores executable program codes, and the processor 1104 executes the executable program codes to respectively implement the functions of the first acquisition module 1010, the second acquisition module 1020, and the control module 1030, thereby implementing Fig. 9 That is, the memory 1106 stores the method for executing Fig. 9 Instructions for the method shown.
[0103] The communication interface 1108 uses a transceiver module such as, but not limited to, a network interface card or a transceiver to implement communication between the computing device 1100 and other devices or a communication network.
[0104] The embodiment of the present application also provides a computing device cluster. The computing device cluster includes at least one computing device. The computing device can be a server, such as a central server, an edge server, or a local server in a local data center. In some embodiments, the computing device can also be a terminal device such as a desktop computer, a laptop computer, or a smart phone.
[0105] like Fig.12 As shown, the computing device cluster includes at least one computing device 1100. The memory 1106 in one or more computing devices 1100 in the computing device cluster may store the same Fig. 9 Instructions for the method shown.
[0106] In some possible implementations, the memory 1106 of one or more computing devices 1100 in the computing device cluster may also store a program for executing Fig. 9In other words, the combination of one or more computing devices 1100 can jointly execute instructions for executing Fig. 9 Instructions for the method shown.
[0107] It should be noted that the memory 1106 in different computing devices 1100 in the computing device cluster may store different instructions, which are respectively used to execute part of the functions of the management platform 1000. That is, the instructions stored in the memory 1106 in different computing devices 1100 may implement the functions of one or more modules among the first acquisition module 1010, the second acquisition module 1020 and the management and control module 1030.
[0108] In some possible implementations, one or more computing devices in the computing device cluster may be connected via a network, which may be a wide area network or a local area network. Fig.13 A possible implementation is shown. Fig.13 As shown, two computing devices 1100A and 1100B are connected via a network. Specifically, the network is connected via a communication interface in each computing device. In this type of possible implementation, the memory 1106 in the computing device 1100A stores instructions for executing the functions of the first acquisition module 1010. At the same time, the memory 1106 in the computing device 1100B stores instructions for executing the functions of the second acquisition module 1020 and the control module 1030.
[0109] It should be understood that Fig.13 The functions of the computing device 1100A shown in FIG. 1100A may also be completed by multiple computing devices 1100. Similarly, the functions of the computing device 1100B may also be completed by multiple computing devices 1100.
[0110] The present application embodiment also provides another computing device cluster. The connection relationship between the computing devices in the computing device cluster can be similar to that of Fig.12 and Fig.13 The difference is that the memory 1106 in one or more computing devices 1100 in the computing device cluster may store the same memory for executing Fig. 9 Instructions for the method shown.
[0111] In some possible implementations, the memory 1106 of one or more computing devices 1100 in the computing device cluster may also store a program for executing Fig. 9 In other words, the combination of one or more computing devices 1100 can jointly execute instructions for executing Fig. 9 Instructions for the method shown.
[0112] The present application also provides a computer program product including instructions. The computer program product may be software or a program product including instructions that can be run on a computing device or stored in any available medium. When the computer program product is run on at least one computing device, the at least one computing device executes Fig. 9 The method shown.
[0113] The present application also provides a computer-readable storage medium. The computer-readable storage medium may be any available medium that can be stored by a computing device or a host migration device such as a data center that includes one or more available media. The available medium may be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state hard disk). The computer-readable storage medium includes instructions that instruct the computing device to execute Fig. 9 The method shown.
[0114] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the protection scope of the technical solutions of the embodiments of the present application.
Claims
1. A data processing method, characterized in that: The method is applied to a management platform, and the management platform is used to manage multiple data sets, wherein each of the multiple data sets records at least one data, and the data sets have a management level; the method includes: Obtaining a change request, where the change request is used to request to perform a change operation on a target data set among the multiple data sets; Acquire the control level of each data set in at least one data set among the multiple data sets; wherein all or part of the data recorded in each data set in the at least one data set comes from the target data set; The change request is governed based on a governance level of each of the at least one data set.
2. The method according to claim 1, characterized in that The controlling the change request based on the control level of each data set in the at least one data set comprises: Identifying a first data set in the at least one data set; wherein a control level of the first data set is greater than or equal to a threshold; Sending a change control request to a control user of the first data set; the change control request is used to request the control user to make a change instruction in response to the change request; Receiving a change instruction issued by the control user; Based on the change indication, the change request is allowed or denied.
3. The method according to claim 2, characterized in that The change control request includes the impact of the change operation on the data in the first data set.
4. The method according to claim 1, characterized in that: The controlling the change request based on the control level of each data set in the at least one data set comprises: When the control level of each data set in the at least one data set is less than a threshold, the change request is allowed.
5. The method according to any one of claims 1 to 4, characterized in that The method further comprises: If the change request is allowed, saving a backup of the target data set and the at least one data set; Execute the change operation on the target data set to obtain a change result; When the change result does not meet the requirement, the target data and the at least one data set are restored using the backup.
6. A management platform, characterized in that: The management platform is used to manage multiple data sets, wherein each of the multiple data sets records at least one data, and the data sets have a management level; the management platform includes: A first acquisition module, configured to acquire a change request, wherein the change request is used to request to perform a change operation on a target data set among the multiple data sets; A second acquisition module is used to acquire the control level of each data set in at least one data set among the multiple data sets; wherein all or part of the data recorded in each data set in the at least one data set comes from the target data set; A management and control module is used to manage the change request based on the management level of each data set in the at least one data set.
7. The management platform according to claim 6, characterized in that: The control module is used to: Identifying a first data set in the at least one data set; wherein a control level of the first data set is greater than or equal to a threshold; Sending a change control request to a control user of the first data set; the change control request is used to request the control user to make a change instruction in response to the change request; Receiving a change instruction issued by the control user; Based on the change indication, the change request is allowed or denied.
8. A computing device cluster, characterized in that: comprising at least one computing device, each computing device comprising a processor and a memory; The processor of the at least one computing device is configured to execute instructions stored in the memory of the at least one computing device, so that the computing device cluster executes the method according to any one of claims 1 to 5.
9. A computer-readable storage medium, characterized in that: The method comprises computer program instructions. When the computer program instructions are executed by a computing device cluster, the computing device cluster performs the method according to any one of claims 1 to 5.
10. A computer program product comprising instructions, characterized in that When the instructions are executed by a computer device cluster, the computer device cluster is caused to perform the method according to any one of claims 1 to 5.