A big data resource processing method based on cloud database service

Abstract

The present invention discloses a big data resource processing method based on cloud database service, which specifically relates to the field of data management technology. The method obtains basic attribute information attached to a data access request initiated by a tenant using the cloud database service, extracts the state change characteristics of the data access request and the execution deviation index of the data access strategy; constructs a cross-tenant logical data domain access relationship change model, and analyzes the data access relationship change pattern; constructs an instability evaluation function of the data access strategy based on the execution deviation index, and identifies potential resource scheduling failure points; generates resource scheduling control instructions based on the access relationship change pattern and the potential resource scheduling failure points; and reconfigures the big data resource processing logic in the cloud database service based on the resource scheduling control instructions, thereby realizing dynamic adaptation and stable mapping of the data access context, and improving the stability and consistency of big data resource processing in the cloud database service.

Description

Technical Field

[0001] The present invention relates to the field of data management technology, and more specifically, to a big data resource processing method based on cloud database services. Background Art

[0002] In the existing cloud database service environment, distributed resource scheduling relies on a static mapping mechanism between tenant access behavior and data policies, which makes it difficult to cope with complex scenarios where context frequently drifts under multi-tenant dynamic access modes.

[0003] During the high-concurrency processing of big data resources, the data access contexts of different tenants change frequently and have blurred boundaries, which makes it impossible for the resource scheduling system to accurately maintain a stable correspondence between data processing strategies and logical data domains, seriously affecting the resource processing consistency and policy execution stability of cloud databases in multi-tenant environments. Summary of the Invention

[0004] In order to overcome the above-mentioned defects of the prior art, an embodiment of the present invention provides a big data resource processing method based on cloud database services to solve the problems raised in the above-mentioned background technology.

[0005] To achieve the above object, the present invention provides the following technical solutions:

[0006] A method for processing big data resources based on cloud database services, comprising the following steps:

[0007] S1: Obtain basic attribute information attached to the data access request initiated by the tenant using the cloud database service;

[0008] S2: Based on basic attribute information, identify changes in data access policies for processing data access requests, and extract state change characteristics of data access requests and execution deviation indicators of data access policies;

[0009] S3: Based on the status change trend characteristics of data access requests, a cross-tenant logical data domain access relationship change model is constructed to analyze the status change trend characteristics of data access requests and the data access relationship change pattern between the target logical data domains;

[0010] S4: Based on the execution deviation index, a data access policy instability evaluation function is constructed to evaluate the stability of the data access policy and identify potential resource scheduling failure points.

[0011] S5: Based on the access relationship change pattern and potential resource scheduling failure points, perform logical domain reconstruction and data access strategy mapping path optimization to generate resource scheduling control instructions;

[0012] S6: Based on resource scheduling control instructions, reconfigure the big data resource processing logic in the cloud database service.

[0013] In a preferred embodiment, the basic attribute information includes tenant identification information, submission time of the data access request, location of the data resource pointed to by the data access request, access operation type specified by the data access request, and logical data domain information of the tenant from which the data access request originates.

[0014] In a preferred embodiment, S2 is specifically:

[0015] Based on basic attribute information, determine how the data access policy executed by the data access request changes at different times or under different resource location conditions;

[0016] Extracting the state change trend characteristics of data access requests based on the changes in the data access policies executed by the data access requests at different times or under different resource location conditions;

[0017] Based on the changes in the data access strategy executed by the data access request at different times or under different resource location conditions, the deviation between the actual execution status and the expected execution status of the data access strategy is determined, and the execution deviation index of the data access strategy is quantified and generated.

[0018] In a preferred embodiment, S3 is specifically:

[0019] Based on the state change trend characteristics of data access requests, a cross-tenant logical data domain access relationship change model is constructed; the cross-tenant logical data domain access relationship change model includes the correspondence between the state change trend of data access requests and the target logical data domain, as well as the change path of data access requests between different tenant logical data domains;

[0020] Based on the cross-tenant logical data domain access relationship change model, the state change trend characteristics of data access requests and the data access relationship change pattern between the target logical data domains are analyzed; the data access relationship change pattern includes the regular change pattern of data access requests over time and the change pattern of data access requests transferred or remapped between different resource locations.

[0021] In a preferred embodiment, S4 is specifically:

[0022] According to the execution deviation index of the data access strategy, the instability evaluation function of the data access strategy is constructed;

[0023] Based on the instability evaluation function of the data access policy, the stability of the data access policy executed by the data access request is quantitatively evaluated to generate a stability index of the data access policy;

[0024] According to the stability index of the data access policy, the data resource location corresponding to the data access request is analyzed to identify the potential resource scheduling failure points corresponding to the abnormal execution of the data access policy.

[0025] In a preferred embodiment, S5 is specifically:

[0026] Determine the mapping adjustment range of the data resource location in the target logical data domain based on the state change trend characteristics of the data access request and the data access relationship change pattern between the target logical data domain;

[0027] Based on potential resource scheduling failure points, determine the location of data resources that cause abnormal data access policy execution;

[0028] Based on the mapping adjustment range of the data resource location and the data resource location with abnormal data access policy execution, a mapping reconstruction rule for the data resource location is established;

[0029] Based on the mapping reconstruction rules of the data resource location, the mapping path of the data access strategy is optimized and resource scheduling control instructions are generated; the resource scheduling control instructions include data resource location adjustment information and data access strategy mapping path adjustment information.

[0030] In a preferred embodiment, S6 is specifically:

[0031] Based on resource scheduling control instructions, determine the location range of data resources that need to be configured and the execution path corresponding to the data access policy;

[0032] Based on the data resource location range that needs to be changed, reconfigure the storage distribution and access mapping of the data resource location in the target logical data domain, and update the correspondence between the data resource location and the tenant identification information;

[0033] Based on the execution path corresponding to the data access policy, adjust the data access control logic corresponding to the data access request and rebuild the data mapping rules between the data access policy and the target logical data domain;

[0034] Update the data access processing logic in the cloud database service based on the adjusted correspondence and data mapping rules.

[0035] The technical effects and advantages of the big data resource processing method based on cloud database service of the present invention are as follows:

[0036] By obtaining the basic attribute information of tenant data access requests, we can achieve comprehensive perception of data access context and enhance the accurate identification of tenant behavior; by identifying changes in data access policies and extracting state change characteristics and execution deviation indicators, we can dynamically grasp the drift of data access policies; building a cross-tenant logical data domain access relationship change model and analyzing the data access relationship change pattern between the state change trend characteristics of data access requests and the target logical data domain, which helps to accurately grasp the mapping drift process between policies and data domains; by establishing an instability assessment function and identifying potential resource scheduling failure points, the accuracy of anomaly perception is improved; combining access relationship patterns with potential resource scheduling failure points to reconstruct logical domains and optimize mapping paths, we can improve the mapping consistency between policies and data domains; reconfiguring resource processing logic through control instructions realizes dynamic adaptation and optimization of big data resource processing behavior, effectively improving the consistency, isolation and scheduling efficiency of big data resource processing in cloud database environments. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1 This is a schematic diagram of a big data resource processing method based on cloud database services in the present invention. DETAILED DESCRIPTION

[0038] The following will provide a clear and complete description of the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0039] Example

[0040] Figure 1 The present invention provides a method for processing big data resources based on cloud database services, which includes the following steps:

[0041] S1: Obtain basic attribute information attached to the data access request initiated by the tenant using the cloud database service;

[0042] S2: Based on basic attribute information, identify changes in data access policies for processing data access requests, and extract state change characteristics of data access requests and execution deviation indicators of data access policies;

[0043] S3: Based on the status change trend characteristics of data access requests, a cross-tenant logical data domain access relationship change model is constructed to analyze the status change trend characteristics of data access requests and the data access relationship change pattern between the target logical data domains;

[0044] S4: Based on the execution deviation index, a data access policy instability evaluation function is constructed to evaluate the stability of the data access policy and identify potential resource scheduling failure points.

[0045] S5: Based on the access relationship change pattern and potential resource scheduling failure points, perform logical domain reconstruction and data access strategy mapping path optimization to generate resource scheduling control instructions;

[0046] S6: Based on resource scheduling control instructions, reconfigure the big data resource processing logic in the cloud database service.

[0047] S1: Obtain basic attribute information associated with the data access request initiated by the tenant using the cloud database service, including:

[0048] The cloud database service receives data access requests initiated by tenants through the cloud database service system. Data access requests initiated by tenants carry basic attribute information, including tenant identification information, the time the data access request was submitted, the location of the data resource to which the data access request refers, the type of access operation specified by the data access request, and information about the logical data domain of the tenant from whom the data access request originated. The above information is uniquely identified within the cloud database service system and verified by the cloud database service's identity authentication system. The cloud database service's identity authentication system implements an identity authentication mechanism for each tenant. For example, when a tenant submits a data access request, the cloud database service system first verifies the tenant's account information, organizational information, account key, or token to ensure the legitimacy of the tenant submitting the data access request.

[0049] The cloud database service system controls data access permissions based on the tenant's authentication results, the data resource location targeted by the data access request, and the access operation type specified in the data access request. For each data access request, the cloud database service system first matches the permission control rules in the predefined data access policy library. The permission control rules in the cloud database service system's data access policy library specify the permission levels each tenant has for different data resource locations and access operation types. For example, a tenant can only access data resources within its own logical data domain or the logical data domains of other tenants to whom it is authorized. Unauthorized cross-tenant access is prohibited.

[0050] Tenant identification information is the unique identification information of each tenant in the cloud database service, including the tenant's name, account information, and organizational information. For example, for a tenant, the tenant's name is the company name, the tenant's account information is the account name used when the company registered for the cloud database service, and the tenant's organizational information is the corporate or unit organizational information provided when the company registered in the cloud database service system. Each tenant's identity information is obtained through the above methods.

[0051] The submission time of a data access request refers to the timestamp information when each tenant submits a data access request to the cloud database service system through the cloud database service. Timestamp information is accurate to the year, month, day, hour, minute, and second, and is recorded and stored in a standardized manner using a unified time standard. For example, when a tenant submits a data access request to the cloud database service, the cloud database service system will record the precise time information of the data access request's arrival. For example, the arrival time may be recorded as the year, month, day, hour, minute, and second, and used as the submission time of the data access request, thereby accurately recording the arrival time of each data access request.

[0052] The data resource location targeted by a data access request is the location of the data resource being accessed within the target logical data domain in the cloud database service specified by the tenant when submitting the data access request. This includes the location identification information of the database instance, data storage location, logical data storage area, or logical partition within the cloud database service where the data resource resides. For example, when a tenant submits a data access request specifying access to a data table located in a specific database instance, the data resource location targeted by the data access request includes the database instance information where the data table resides, the location information of the logical data domain where the data table resides, and the identification information of the data table itself. This is how the location of the data resource to be accessed by each data access request is determined.

[0053] The access operation type specified in a data access request is the operation that each tenant specifies on the target data resource when submitting a data access request. This includes operations such as reading, writing, modifying, or deleting the target data resource. For example, if a tenant submits a data access request through the cloud database service and specifies a read operation on the data resource, the access operation type specified in the data access request is a read operation. If a write operation is specified, the access operation type specified in the data access request is a write operation. Similarly, each data access request must specify a specific access operation type when submitted.

[0054] The logical data domain information of the tenant from whom the data access request originates refers to the location information of the logical area within the cloud database service system that is allocated to the tenant submitting the data access request to store and manage the tenant's data. The cloud database service divides and manages specific logical data domains for each tenant, each with clear identification and boundaries. For example, if a tenant is assigned a specific logical data domain by the cloud database service system, which is the tenant's exclusive data area, the logical data domain information will include the logical data domain's unique identification information, the logical data domain's data boundaries, and the structure and storage method of the data resources within the logical data domain. This information clearly identifies the location of the logical data domain of the tenant submitting the data access request.

[0055] S2: Based on basic attribute information, identify changes in data access policies for processing data access requests, extract state change characteristics of data access requests and execution deviation indicators of data access policies, including:

[0056] Based on basic attribute information, determine how the data access policy executed by the data access request changes at different times or under different resource location conditions;

[0057] The cloud database service system obtains the basic attribute information associated with data access requests initiated by tenants using the cloud database service. Based on this basic attribute information, the cloud database service system performs an initial data access policy matching operation for each data access request. Specifically, based on the tenant's identification information, the location of the data resource targeted by the data access request, and the access operation type specified in the data access request, the cloud database service system determines the initial data access policy applicable to the current data access request from the predefined data access policy library in the cloud database service system. This initial data access policy includes policy information such as data resource access permission rules, the range of permitted access operation types, and resource scheduling priorities.

[0058] When determining the initial data access policy for each data access request, the cloud database service system matches and verifies the scope of the permission control rules against the data access request. If the data resource location specified by the data access request exceeds the scope defined by the permission control rules, the cloud database service system will reject the data access request. For example, if a tenant submits a data access request that specifies a data resource location outside the tenant's own logical data domain and is not authorized by other tenants, the cloud database service system will immediately terminate processing of the data access request and log the abnormal access event.

[0059] The cloud database service system continuously tracks changes in the data access policies executed by data access requests over time and across resource locations. During each policy change detection process, the cloud database service system re-verifies the legitimacy of permission scopes to prevent unauthorized expansion of permission scopes during policy upgrades or changes. For example, when data access policies change due to resource migration or policy adjustments, the cloud database service system re-compares the mapping between the data resource locations involved in the new policy and tenant permissions to ensure that the new policy adheres to the existing permission control framework.

[0060] For example, a tenant submits a data access request. The tenant identification information is a company name and the corresponding account and organizational information. The data access request submission time is recorded as a timestamp of the specific year, month, day, hour, minute, and second. The data resource location targeted by the data access request is a data table within a specific logical area under a database instance in the cloud database service. The access operation type specified in the data access request is a read operation on the data resource. The logical data domain information of the tenant from whom the data access request originated is identified as a specific logical area and its boundary information. Based on the above basic attribute information, the cloud database service system determines the initial data access policy for the data access request as allowing data read permissions, setting the read operation priority to general priority, and matching the target logical data domain location.

[0061] After determining the initial data access policy, the cloud database service system analyzes data access policy changes for data access requests. Specifically, the cloud database service system compares the initial data access policy of a tenant-initiated data access request with a predefined policy mapping library within the cloud database service system, under different time and resource location conditions, to determine whether the data access policy of the data access request has changed. Data access policy changes can include upgrades or downgrades, or expansions or reductions in permissions. Specifically, the cloud database service system performs data access policy matching at different time points and records the differences in data access policy at each match. For example, if data access requests are matched at different times of the day, and the policy matching result in the morning indicates read permission and high priority, while the matching result in the afternoon indicates normal priority, then the data access policy is considered to have changed. Under different resource location conditions, if the data resource location targeted by the data access request has changed due to data migration or expansion within the cloud database service system, the cloud database service system performs data access policy matching for both the pre- and post-adjustment data resource locations, and records and determines the data access policy changes caused by the data resource location adjustment.

[0062] Extracting the state change trend characteristics of data access requests based on the changes in the data access policies executed by the data access requests at different times or under different resource location conditions;

[0063] Based on the data access policy changes determined above, the cloud database service system extracts state change trend characteristics of data access requests. Specifically, the cloud database service system identifies the state changes that a submitted data access request undergoes throughout its lifecycle, from submission to completion, based on the data access policy changes. This state change process includes multiple states, such as the waiting state after the data access request is received by the cloud database service system, the execution state after the data access policy is matched, the resource allocation state during the access operation, and the resource release state after the operation is completed. The cloud database service system records all states of the data access request in chronological order of submission, forming a state change sequence. Based on this state change sequence, the cloud database service system analyzes the state change patterns of the data access request to identify and extract state change trend characteristics during the data access request processing. State change trend characteristics are reflected in the regular characteristics of periodic or sudden changes in the data access request from the waiting state, execution state, resource allocation state, and resource release state along the timeline.

[0064] For example, the cloud database service system records the waiting state time, execution state duration, resource allocation state occurrence time, and resource release state duration of each data access request submitted by a tenant in a single day. Through statistical analysis, it determines the average duration and frequency of each state and extracts the changing trend characteristics of the data access request states. For example, when access requests are submitted in large numbers within a specific time period, the duration of the resource allocation state increases significantly, reflecting the cyclical change trend of the data access request states.

[0065] Based on the changes in the data access policy executed by the data access request at different times or under different resource location conditions, determine the deviation between the actual execution status of the data access policy and the expected execution status, and quantify and generate the execution deviation index of the data access policy;

[0066] Based on the initially determined data access policy, the cloud database service system generates an expected execution status, including the cloud database service system's expected ideal resource allocation, response time, and completion time for data access requests from receipt to completion. The cloud database service system monitors the actual execution of data access requests, recording the actual completion time, actual number of resources allocated, and actual response time. By comparing the actual execution status item by item with the expected execution status, the difference between the actual response time and the expected response time, the difference between the actual number of resources allocated and the expected number of resources allocated, and the difference between the actual completion time and the expected completion time are calculated to generate an execution deviation index for the data access policy. This execution deviation index reflects the degree of deviation between the actual execution of the data access policy and the expected execution.

[0067] S3: Based on the status change trend characteristics of data access requests, a cross-tenant logical data domain access relationship change model is constructed. The data access relationship change pattern between the status change trend characteristics of data access requests and the target logical data domain is analyzed, including:

[0068] Based on the status change trend characteristics of data access requests, a cross-tenant logical data domain access relationship change model is constructed;

[0069] The cloud database service system establishes a cross-tenant logical data domain access relationship change model based on the state change trend characteristics of data access requests. This model includes the correspondence between the state change trend of data access requests and the target logical data domain, as well as the change path of data access requests between different tenant logical data domains.

[0070] When building a cross-tenant logical data domain access relationship change model, the cloud database service system first determines the corresponding relationship between the state change trend of the data access request and the target logical data domain. Specifically, it determines the mapping relationship between each state of the data access request during processing and the target logical data domain corresponding to the data access request. Following the state change sequence of the data access request submitted by the tenant, the cloud database service system records the target logical data domain location corresponding to the data access request in different states, such as waiting state, execution state, resource allocation state, and resource release state. This information includes the initial storage location of the data resource location, the storage location after the data resource location change, and the time of the change. Based on the state change trend characteristics and the state change sequence of the data access request, the cloud database service system establishes a mapping relationship between each state and the target logical data domain.

[0071] For example, a tenant submits multiple data access requests to the cloud database service system in a single day. The cloud database service system records state change information for each data access request. When recording state change information, the cloud database service system sequentially records the specific database instance and specific data resource location in the target logical data domain to which the data access request originally corresponded when the data access request transitioned from the waiting state to the resource allocation state. It then records whether the data resource location of the data access request in the resource allocation state has changed due to resource optimization adjustments. For example, if the data resource location originally pointed to by the data access request has changed from the initial data resource location to a new data resource location due to load balancing adjustments, the cloud database service system records the data resource location changes of all data access requests in different states to form a complete mapping record, establishing a corresponding relationship between the state change trend of the data access request and the target logical data domain.

[0072] The cross-tenant logical data domain access relationship change model also includes the path of data access requests changing between different tenant logical data domains. When establishing the path of data access requests changing between different tenant logical data domains, the cloud database service system first determines the information of each tenant's logical data domain, including the unique identification information of the tenant's logical data domain, the data boundary information of the tenant's logical data domain, and the storage method of the data resources within the tenant's logical data domain. Based on the tenant identification information and the tenant's logical data domain information recorded when the data access request was submitted, the cloud database service system determines the tenant logical data domain where each data access request initially resides. During data access request processing, the cloud database service system determines whether the data resource location targeted by the data access request belongs to a logical data domain of a tenant other than the tenant submitting the data access request. If the data resource location targeted by the data access request belongs to a logical data domain of another tenant, the cloud database service system records the path information of the data access request from the initial tenant logical data domain to the logical data domain of the other tenant, including the starting and ending locations of the transition between the tenant logical data domains, as well as the path steps during the transition.

[0073] For example, when a tenant submits a data access request, the logical data domain initially located in the tenant's own exclusive logical data domain. During the data access request processing, the cloud database service system discovers that the data resource location pointed to by the data access request belongs to another tenant's logical data domain. The cloud database service system records the data access request's changes across logical data domains. For example, the data access request enters another tenant's logical data domain through a data access policy mapping path from the submitting tenant's logical data domain, recording the boundary information of the tenant's logical data domain, the path steps to enter the other logical data domain, and the data resource mapping or permission transfer between tenants. The cloud database service system records the above path change process to form a change path for the data access request between different tenants' logical data domains.

[0074] Based on the cross-tenant logical data domain access relationship change model, analyze the status change trend characteristics of data access requests and the data access relationship change pattern between the target logical data domains;

[0075] The data access relationship change pattern includes the regular change pattern of data access requests over time and the change pattern of data access requests being transferred or remapped between different resource locations.

[0076] When analyzing regular patterns of change over time, the cloud database service system uses the state change trend characteristics of data access requests as a basis to identify periodic changes in the resource allocation status or execution status of data access requests within specific time periods. For example, when recording the state change trends of tenants' data access requests, the cloud database service system discovered that during specific time periods each day, such as morning or afternoon, data access requests entered the resource allocation state more frequently and for a longer duration than during other time periods, demonstrating a regular pattern of change in data access requests over time.

[0077] When analyzing the changing patterns of data access requests as they are transferred or remapped between different resource locations, the cloud database service system records all change information regarding the transfer of data access requests from the initially designated data resource location to the new data resource location, including the specific change path of the data resource location, the time of the change, and the data access policy conditions that triggered the change. For example, a data access request submitted by a tenant originally points to a data table in a specific database instance. Over time, due to resource optimization requirements, the data resource location to which the data access request points shifts from the data table in the initial database instance to another data table in another database instance. The cloud database service system records the starting location, end location, time point, specific path, and other change information of the transfer process, reflecting the changing patterns of data access requests as they are transferred or remapped between different resource locations.

[0078] S4: Based on the execution deviation index, a data access policy instability evaluation function is constructed to evaluate the stability of the data access policy and identify potential resource scheduling failure points, including:

[0079] According to the execution deviation index of the data access strategy, the instability evaluation function of the data access strategy is constructed;

[0080] The execution deviation indicators of data access policies include the difference between the actual response time and the expected response time, the difference between the actual resource allocation quantity and the expected resource allocation quantity, and the difference between the actual completion time and the expected completion time.

[0081] Through normalization, the cloud database service system converts deviation indicators into standardized deviation indicators that can be uniformly evaluated. Normalization compares each deviation indicator with the corresponding ideal indicator, calculates the difference between the deviation indicator and the ideal indicator, and converts it into a standardized value between zero and one. A value of zero indicates no deviation, while a value closer to one indicates greater deviation. A data access policy instability assessment function is constructed based on the normalized standardized deviation indicators. The data access policy instability assessment function is constructed by weighting the standardized response time deviation indicator, resource allocation quantity deviation indicator, and completion time deviation indicator. Each deviation indicator is assigned a different weight coefficient based on its impact on the stability of the data access policy. Specifically, if the response time deviation indicator has a greater impact on the stability of the data access policy, the response time deviation indicator is assigned a higher weight coefficient; if the resource allocation quantity deviation indicator and completion time deviation indicator have a smaller impact, they are each assigned a lower weight coefficient. The weighted standardized deviation indicators are then summed to obtain the instability assessment function value. The expression of the instability evaluation function of the data access strategy is: the product of the standardized response time deviation index and the weight coefficient corresponding to the response time deviation index plus the product of the standardized resource allocation quantity deviation index and the weight coefficient corresponding to the resource allocation quantity deviation index plus the product of the standardized completion time deviation index and the weight coefficient corresponding to the completion time deviation index.

[0082] Based on the instability evaluation function of the data access policy, the stability of the data access policy executed by the data access request is quantitatively evaluated to generate a stability index of the data access policy;

[0083] The cloud database service system quantitatively evaluates the stability of the data access policy executed by the data access request based on the data access policy's instability assessment function to generate a data access policy stability index. Specifically, the cloud database service system calculates the instability assessment function value for multiple consecutive data access requests, obtaining the stability assessment result of the data access policy for each data access request throughout its entire lifecycle. The instability assessment function values ​​for multiple data access requests are then statistically analyzed, including calculating the average of the instability assessment function values ​​for each data access request, to obtain a stability index that reflects the overall stability of the data access policy. A higher stability index indicates a lower stability of the data access policy; a lower stability index indicates a higher overall stability of the data access policy.

[0084] Based on the stability indicators of the data access policy, the data resource locations corresponding to the data access requests are analyzed to identify potential resource scheduling failure points corresponding to abnormal data access policy execution;

[0085] The cloud database service system performs resource location anomaly analysis on data access requests whose data access policy stability index is higher than the preset threshold. Specifically: For data access requests whose data access policy stability index is higher than the preset threshold, the cloud database service system will retrospectively analyze the resource location changes, data resource location mapping process, and data resource location adjustment history during the data access request processing; by comparing the data resource location with the normal and stable data access request, it will determine the data resource location that caused the abnormal data access policy execution, and determine the location and type of potential resource scheduling failure points related to it. The preset threshold is determined based on the statistical analysis results of the historical stability index of the data access policy in the cloud database service system, based on the range of stable operation status indicators in the historical operation data, and appropriately adjusted in combination with the business needs for security, reliability, and performance flexibility in different application scenarios.

[0086] S5: Based on access relationship change patterns and potential resource scheduling failure points, perform logical domain reconstruction and data access strategy mapping path optimization, and generate resource scheduling control instructions, including:

[0087] Determine the mapping adjustment range of the data resource location in the target logical data domain based on the state change trend characteristics of the data access request and the data access relationship change pattern between the target logical data domain;

[0088] The cloud database service system conducts a comprehensive analysis of the data resource locations in the target logical data domain based on the state change trend characteristics of the data access request and the data access relationship change pattern between the target logical data domain. Specifically, the cloud database service system obtains the data resource locations corresponding to the data access request in each processing state, and records the data resource locations of the obtained data access requests one by one in chronological order based on the submission time of the data access request to form a data resource location change sequence. The cloud database service system analyzes the data resource location change sequence to determine the frequency, scale, and path of adjustments to the data resource location in the target logical data domain, and determines the mapping adjustment range of the data resource location. The mapping adjustment range refers to the range of data resource locations that need to be adjusted in the target logical data domain, including the starting position, end position, and all intermediate positions involved in the data resource.

[0089] For example, a cloud database service system analyzes changes in the location of data resources for multiple consecutive data access requests from tenants and discovers that the storage location of data tables within a specific database instance changes repeatedly over a short period of time. By analyzing the data resource location changes, it is determined that the data resource location adjustment range is concentrated within a specific data partition within the specific database instance. By recording the starting storage location and the new storage location after each change, a mapping adjustment range is formed, including all data resource locations involved between the initial data storage partition in the database instance and the new data storage partition after the change, and recorded as the data resource location mapping adjustment range.

[0090] Based on potential resource scheduling failure points, determine the location of data resources that cause abnormal data access policy execution;

[0091] Based on the stability indicators of the data access policy, the cloud database service system identifies data access requests with stability indicators exceeding a predetermined threshold and identifies them as abnormal data access requests. It then analyzes potential resource scheduling failure points corresponding to these abnormal data access requests. Specifically, it traces the data access request processing process and records the change history of the data resource locations involved in the data access request, including the initial allocation location of the data resource location, the location of any changes during the process, the time when the changes occurred, and the location and time of the first resource scheduling failure.

[0092] For example, a data resource location accessed during the initial submission phase of a data access request is a table in a specific database instance. As the request progresses, the location of the data resource changes multiple times due to frequent resource scheduling, ultimately leading to execution anomalies. The cloud database service system records these frequently changing data resource locations as potential sources of data access policy execution anomalies. It also records the location identifier, process, and time of each change, ultimately locating the data resource location that caused the anomaly.

[0093] Based on the mapping adjustment range of the data resource location and the data resource location with abnormal data access policy execution, a mapping reconstruction rule for the data resource location is established;

[0094] The cloud database service system establishes data resource location mapping reconstruction rules based on the mapping adjustment range of data resource locations in the target logical data domain and the data resource locations that cause data access policy execution anomalies. Data resource location mapping reconstruction rules are a method for adjusting data resource locations, including the migration sequence, the migration target, the migration trigger conditions, and the new resource access rules after the migration is complete.

[0095] Specifically, the cloud database service system first defines the mapping relationship between the mapping adjustment range of the data resource location and the data resource location that caused the abnormal data access policy execution, and then determines the order for reconstructing the data resource location mapping. Based on the data access frequency between the data resource location within the mapping adjustment range and the data resource location that caused the abnormal data access policy execution, the cloud database service system calculates the degree of association between the data resource location and the data resource location that caused the abnormal data access policy execution: first, determine the number of times the data resource location and the data resource location that caused the abnormal data access policy execution are accessed simultaneously, divide the number of simultaneous accesses by the total number of accesses to obtain a numerical value for the degree of association. Based on the numerical value of the degree of association, the data resource location to be migrated is determined first. The higher the numerical value of the degree of association, the higher the priority migration level; if the numerical values ​​of the degree of association of multiple data resource locations that need to be migrated are the same, the specific migration order is determined based on the access frequency of the data access request, and finally the locations of subsequent migrations are determined in sequence.

[0096] The target location for data resource migration is another database instance or logical data domain in the cloud database service system with a high resource access policy stability indicator and low load. Low load refers to the resource utilization rate of the database instance or logical data domain being below a pre-set threshold. Data resource location migration is triggered when the data resource location's correlation exceeds a pre-set threshold or when the stability indicator of multiple consecutive data access requests exceeds a pre-set threshold. After completing the data resource location migration, the cloud database service system first updates the data access policy mapping path, specifically updating the mapping relationship between the data resource location targeted by tenant data access requests so that subsequent tenant data access requests can be directed to the new, migrated data resource location. New resource access rules are then redefined, including: updating the access mapping relationship between the migrated data resource location and tenants, specifically determining each tenant's access rights and the scope of accessible data resources for the migrated data resource location; optimizing the access path, specifically automatically updating the routing path for data access requests between the migrated data resource location and tenants to reduce the path length of data access requests; and redistributing access rights, specifically updating and redefining the specific content of tenant access rights to the migrated data resource location based on each tenant's data access requirements and historical access records.

[0097] Based on the mapping reconstruction rules of data resource locations, the mapping path of data access strategy is optimized and resource scheduling control instructions are generated;

[0098] The mapping path of a data access policy is the path that a data access request follows to access data resource locations within the target logical data domain. This path includes the starting location of the data request, the intermediate paths, and the final data resource location accessed. The cloud database service system uses mapping and reconstruction rules to determine the new path for data access requests, thereby avoiding potential resource scheduling failure points.

[0099] Specifically, the cloud database service system optimizes the path of each data access request based on the data resource location mapping reconstruction rules, adjusting the data access path that originally passed through the potential resource scheduling failure point to the new data resource location. The cloud database service system gradually executes the data resource location migration process according to the mapping reconstruction rules. After the migration is complete, the cloud database service system determines the readjusted mapping path for the data access request. This means that the data access request no longer accesses the original failure location, but instead points to the new data resource location after the migration.

[0100] Ultimately, the cloud database service system generates a complete resource scheduling control instruction, including data resource location adjustment information and data access strategy mapping path adjustment information. The data resource location adjustment information records the data resource location before migration, the new data resource location after migration, and the migration trigger conditions. The data access strategy mapping path adjustment information records the optimized starting location, intermediate nodes, and final data resource location of the mapping path for the data access request.

[0101] S6: Based on resource scheduling control instructions, reconfigure the big data resource processing logic in the cloud database service, including:

[0102] Based on resource scheduling control instructions, determine the location range of data resources that need to be configured and the execution path corresponding to the data access policy;

[0103] The cloud database service system determines the data resource location range for the target logical data domain requiring configuration changes based on the data resource location adjustment information contained in the resource scheduling control directive. The system obtains the initial location of the data resource before migration and the target location after migration from the data resource location adjustment information, and uses the initial and target locations as endpoints to determine the data resource location range for the configuration change. If the resource scheduling control directive includes multiple migration paths, the cloud database service system determines the initial and target locations involved in each migration path, ultimately forming a complete data resource location range for all migration paths.

[0104] For example, if a resource scheduling control instruction indicates that a specific data table in a specific database instance needs to be moved to a new location in another database instance due to multiple consecutive data access request exceptions, the cloud database service system will record the data resource location of the specific data table in the specific database instance and the new data table location in the other database instance, forming the data resource location range of the configuration change. If there are multiple data tables or multiple database instances, the cloud database service system will determine the initial and target locations of each involved, recording them one by one to form the data resource location range.

[0105] Based on the data resource location range that needs to be changed, reconfigure the storage distribution and access mapping of the data resource location in the target logical data domain, and update the correspondence between the data resource location and the tenant identification information;

[0106] The cloud database service system reconfigures the storage distribution mode and access mapping mode of the data resource location in the target logical data domain based on the data resource location range that requires configuration changes. The storage distribution mode refers to the distribution structure and distribution pattern of data resources within a database instance or logical data domain, and the access mapping mode refers to the correspondence between a tenant's data access request and the data resource location. The configuration method includes: the cloud database service system determines the optimal storage location of each data resource location within the logical data domain based on the number of data resources within the data resource location range, the frequency of data access requests, and the efficiency of resource utilization. Specifically, the load of each data resource location is sorted according to the historical data access frequency, and the load of each data resource location is distributed in descending order to the location of the database instance or logical data domain with the lower current load to achieve load balancing.

[0107] After the storage location is reconfigured, the cloud database service system updates the mapping between the data resource location and the tenant's identification information. Based on the tenant's identification information, the cloud database service system identifies the mapping between each tenant's original data resource location and the reconfigured new data resource location. This mapping is then updated and saved in the cloud database service system's internal relationship mapping library. The relationship mapping library records the updated target location for tenant data access requests, ensuring that subsequent tenant data access requests are accurately mapped to the new, migrated data resource location.

[0108] Based on the execution path corresponding to the data access policy, adjust the data access control logic corresponding to the data access request and rebuild the data mapping rules between the data access policy and the target logical data domain;

[0109] The cloud database service system adjusts the data access control logic corresponding to the data access request based on the execution path corresponding to the data access policy. The execution path corresponding to the data access policy is the path of the data resource location that the data access request actually passes through, including the initial request location, the intermediate path location, and the final data resource location. The data access control logic is the logical rule that controls how the data access request flows in a specific path. The adjustment method is as follows: the cloud database service system determines a new data access policy mapping path based on the data access policy mapping path adjustment information in the resource scheduling control instruction, that is, replaces the old path that the data access request originally passed through the potential resource scheduling failure point with the newly migrated data resource location path. Based on the new data access path, the cloud database service system adjusts the original data access control logic, including updating the access order of the data request path, the access rights of the path nodes, and the resource allocation logic after the data access request reaches the path node.

[0110] After adjusting the data access control logic, the cloud database service system reconstructs the data mapping rules between the data access policy and the target logical data domain. Data mapping rules refer to the correspondence between the data access policy and the location of data resources within the logical data domain. Specifically, the cloud database service system determines the access rights, access scope, and resource scheduling priorities between the new data resource location and the data access policy. Based on these rules, the cloud database service system adjusts the mapping relationship between the data access policy and the target logical data domain, forming the data mapping rules between the data access policy and the target logical data domain.

[0111] Update the data access processing logic in the cloud database service based on the adjusted correspondence and data mapping rules;

[0112] The cloud database service system updates the data access processing logic within the cloud database service based on the adjusted correspondence between the data resource location and tenant identification information, as well as the data mapping rules between the data access policy and the target logical data domain. This update method includes: Based on all the aforementioned adjustments, the cloud database service system rebuilds the processing flow rules for tenant data access requests, determining how tenant data access requests are received, executed, resourced, and completed based on the new data resource location and mapping path.

[0113] For example, when the cloud database service system receives a new data access request, it determines the processing path of the data access request based on the updated data access processing logic, including the data resource location and resource allocation process, to ensure that the data access request is accurately processed according to the latest resource configuration, mapping relationship and control logic, completing the entire life cycle of the data access request.

[0114] The above formulas are all dimensionless and numerical calculations. The formulas are obtained by collecting a large amount of data and performing software simulation to obtain the most recent real situation. The preset parameters and thresholds in the formulas are set by technicians in this field according to actual conditions.

[0115] The above embodiments can be implemented in whole or in part via software, hardware, firmware, or any other combination. When implemented using software, the above embodiments can be implemented in whole or in part in the form of a computer program product. The computer program product comprises one or more computer instructions or computer programs. When loaded or executed on a computer, the processes or functions described in the embodiments of this application are fully or partially performed. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired means (e.g., infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium accessible by a computer or a data storage device such as a server or data center that contains a collection of one or more available media. The available medium can be magnetic media (e.g., floppy disks, hard disks, tapes), optical media (e.g., DVDs), or semiconductor media. The semiconductor media can be a solid-state drive.

[0116] Those skilled in the art will appreciate that the modules and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0117] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and modules described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0118] In the several embodiments provided in this application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of the modules is only a logical function division. In actual implementation, there may be other division methods, such as multiple modules or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or modules, which can be electrical, mechanical or other forms.

[0119] The modules described as separate components may or may not be physically separate, and the components shown as modules may or may not be physical modules, and may be located in one place or distributed across multiple network modules. Some or all of the modules may be selected to achieve the purpose of this embodiment according to actual needs.

[0120] In addition, each functional module in each embodiment of the present application may be integrated into one processing module, or each module may exist physically separately, or two or more modules may be integrated into one module.

[0121] If the functions are implemented in the form of software function modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0122] The above description is merely a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the claims.

[0123] Finally: The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A method for processing big data resources based on cloud database services, characterized in that: The steps include: S1: Obtain basic attribute information attached to the data access request initiated by the tenant using the cloud database service; S2: Based on basic attribute information, identify changes in data access policies for processing data access requests, and extract state change characteristics of data access requests and execution deviation indicators of data access policies; S3: Based on the status change trend characteristics of data access requests, a cross-tenant logical data domain access relationship change model is constructed to analyze the status change trend characteristics of data access requests and the data access relationship change pattern between the target logical data domains; S4: Based on the execution deviation index, a data access policy instability evaluation function is constructed to evaluate the stability of the data access policy and identify potential resource scheduling failure points. S5: Based on the access relationship change pattern and potential resource scheduling failure points, perform logical domain reconstruction and data access strategy mapping path optimization to generate resource scheduling control instructions; S6: Reconfigure the big data resource processing logic in the cloud database service based on resource scheduling control instructions; Based on resource scheduling control instructions, determine the location range of data resources that need to be configured and the execution path corresponding to the data access policy; Based on the data resource location range that needs to be changed, reconfigure the storage distribution and access mapping of the data resource location in the target logical data domain, and update the correspondence between the data resource location and the tenant identification information; Based on the execution path corresponding to the data access policy, adjust the data access control logic corresponding to the data access request and rebuild the data mapping rules between the data access policy and the target logical data domain; Update the data access processing logic in the cloud database service based on the adjusted correspondence and data mapping rules.

2. A method for processing big data resources based on cloud database services according to claim 1, characterized in that: Basic attribute information includes tenant identification information, submission time of the data access request, location of the data resource pointed to by the data access request, type of access operation specified by the data access request, and logical data domain information of the tenant from which the data access request originates.

3. A method for processing big data resources based on cloud database services according to claim 2, characterized in that: S2, specifically: Based on basic attribute information, determine how the data access policy executed by the data access request changes at different times or under different resource location conditions; Extracting the state change trend characteristics of data access requests based on the changes in the data access policies executed by the data access requests at different times or under different resource location conditions; Based on the changes in the data access strategy executed by the data access request at different times or under different resource location conditions, the deviation between the actual execution status and the expected execution status of the data access strategy is determined, and the execution deviation index of the data access strategy is quantified and generated.

4. A method for processing big data resources based on cloud database services according to claim 3, characterized in that: S3, specifically: Based on the state change trend characteristics of data access requests, a cross-tenant logical data domain access relationship change model is constructed; the cross-tenant logical data domain access relationship change model includes the correspondence between the state change trend of data access requests and the target logical data domain, as well as the change path of data access requests between different tenant logical data domains; Based on the cross-tenant logical data domain access relationship change model, the state change trend characteristics of data access requests and the data access relationship change pattern between the target logical data domains are analyzed; the data access relationship change pattern includes the regular change pattern of data access requests over time and the change pattern of data access requests transferred or remapped between different resource locations.

5. A method for processing big data resources based on cloud database services according to claim 4, characterized in that: S4, specifically: According to the execution deviation index of the data access strategy, the instability evaluation function of the data access strategy is constructed; Based on the instability evaluation function of the data access policy, the stability of the data access policy executed by the data access request is quantitatively evaluated to generate the stability index of the data access policy; According to the stability index of the data access policy, the data resource location corresponding to the data access request is analyzed to identify the potential resource scheduling failure points corresponding to the abnormal execution of the data access policy.

6. A method for processing big data resources based on cloud database services according to claim 5, characterized in that: S5, specifically: Determine the mapping adjustment range of the data resource location in the target logical data domain based on the state change trend characteristics of the data access request and the data access relationship change pattern between the target logical data domain; Based on potential resource scheduling failure points, determine the location of data resources that cause abnormal data access policy execution; Based on the mapping adjustment range of the data resource location and the data resource location with abnormal data access policy execution, a mapping reconstruction rule for the data resource location is established; Based on the mapping reconstruction rules of the data resource location, the mapping path of the data access strategy is optimized and resource scheduling control instructions are generated; the resource scheduling control instructions include data resource location adjustment information and data access strategy mapping path adjustment information.

Images