Data processing method and device, electronic equipment, storage medium and program product

Abstract

The invention provides a data processing method and device, electronic equipment, a storage medium and a program product. The method comprises the following steps: acquiring first information of first data which is not processed currently, wherein the first information comprises a reason for the first data which is not processed currently, first resource information which can be scheduled currently, and second resource information required for processing the first data; when the unprocessed reason indicates that the submission of the first data fails, determining a target partition corresponding to the reason of the submission failure according to the reason of the submission failure and the first resource information, and determining a first correction strategy for the second resource information according to the resource information of the target partition; determining a second correction strategy for the second resource information according to the first data and the first resource information when the reason of failure in processing indicates that the first data is submitted successfully and queues to wait for processing; and displaying the first correction strategy or the second correction strategy. The reliability of data processing is improved.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a data processing method, apparatus, electronic device, storage medium, and program product. Background Technology

[0002] With the rapid development of high-performance computing technology, the complexity and scale of its tasks have increased significantly, making the need for accuracy and efficiency in scheduling computing resources increasingly important.

[0003] In related technologies, when an anomaly occurs during the execution of a computing task, the data is usually processed manually. That is, the user needs to manually modify the relevant data (e.g., resource parameters) based on the returned error information so that the computing task can run normally.

[0004] However, the above process requires manual troubleshooting and modification based on experience, which may result in repeated modifications to meet the requirements, leading to low reliability of data processing. Summary of the Invention

[0005] This application provides a data processing method, apparatus, electronic device, storage medium, and program product to solve the technical problem of low reliability in data processing.

[0006] Firstly, this application provides a data processing method, including:

[0007] Obtain first information about the first data that has not yet been processed. The first information includes the reason why the data has not been processed, the first resource information that is currently available for scheduling, and the second resource information required to process the first data.

[0008] When the reason for failure indicates that the first data submission failed, the target partition corresponding to the reason for the submission failure is determined based on the reason for the submission failure and the first resource information, and the first correction strategy for the second resource information is determined based on the resource information of the target partition.

[0009] When the reason for failure to process indicates that the first data was successfully submitted and is queued for processing, a second correction strategy for the second resource information is determined based on the first data and the first resource information.

[0010] Display the first correction strategy or the second correction strategy.

[0011] In this embodiment, by acquiring first information about the first data that has not yet been processed, including the reason for the failure, the first available schedulable resource information, and the first resource information required for processing; when the reason indicates submission failure, a target partition is determined based on the failure reason and the first resource information, and a first correction strategy for the second resource information is determined based on the resource information of the target partition; when the reason indicates queuing, a second correction strategy for the second resource information is determined based on the first data and the first resource information. In the above method, the electronic device can dynamically generate targeted resource configuration correction strategies by adopting different analysis paths according to different anomaly types (submission failure or queuing) encountered during data processing, effectively avoiding the problem of low processing efficiency caused by ambiguous anomaly information in traditional solutions. It does not rely on user's manual experience analysis, significantly improving the success rate and efficiency of data processing. Furthermore, by directly associating repair suggestions with specific configuration parameters and resource queues, this method realizes an integrated processing flow from problem diagnosis to correction guidance, enabling rapid resolution of resource scheduling and execution problems. In addition, this method is applicable to various scales and complexities of operation scenarios in supercomputing internet, improving data processing reliability while increasing data processing success rate and resource utilization.

[0012] Optionally, the method described above, based on the reason for the submission failure and the first resource information, determines the target partition corresponding to the reason for the submission failure, including:

[0013] Determine the reason type corresponding to the reason for the submission failure. The reason type is used to indicate the resource that caused the first data submission failure.

[0014] The target partition is determined based on the reason type corresponding to the submission failure and the first resource information.

[0015] In this embodiment of the application, the problem resource is accurately located by cause type, so as to achieve accurate matching of target partition and improve processing efficiency.

[0016] Optionally, the method described above includes:

[0017] Determine the failure parameters when the first data submission fails;

[0018] Obtain the data association library, which includes multiple failure parameters and information describing each failure parameter.

[0019] In the data association database, match the information describing the failure parameters corresponding to the failure parameters when the first data submission fails to obtain the reason for the submission failure.

[0020] In this embodiment of the application, the failure parameters are automatically parsed using a data association library to quickly obtain understandable reasons for failure and reduce the cost of manual interpretation.

[0021] Optionally, as described above, the target partition is determined based on the reason type corresponding to the submission failure and the first resource information, including:

[0022] Based on the reason type corresponding to the submission failure and the first resource information, determine at least one first partition that can process the first data;

[0023] In at least one first partition, determine the target partition corresponding to the cause type.

[0024] In this embodiment of the application, a two-level screening mechanism is used to determine the target partition, ensuring that the selected partition can both process data and match the cause of the anomaly, thereby improving the rationality and reliability of the target partition selection.

[0025] Optionally, the method described above, based on the resource information of the target partition, determines a first correction strategy for the second resource information, including:

[0026] From the resource information in the target partition, obtain the schedulable target resource information that can process the first data, where the first resource information includes the target resource information;

[0027] Based on the target resource information and the second resource information, a first correction strategy for the second resource information is determined. The first correction strategy is used to configure resources for the second resource information of the first data. The first correction strategy includes the resource parameters that need to be corrected.

[0028] In this embodiment, a specific parameter correction strategy is generated based on the target partition resource information, providing a direct and operable resource configuration scheme and avoiding blind trial and error.

[0029] Optionally, the method described above, based on the first data and the first resource information, determines a second correction strategy for the second resource information, including:

[0030] Determine the waiting parameters corresponding to the first data waiting to be processed. The waiting parameters are used to indicate the reason why the first data is waiting to be processed.

[0031] Based on the waiting parameters, the first data, and the first resource information, a second correction strategy for the second resource information is determined. The second correction strategy is used to configure resources for the second resource information of the first data. The second correction strategy includes the resource queue to be corrected and the corresponding waiting time information.

[0032] In this embodiment, a queue optimization strategy is generated by combining waiting parameters and resource information to provide a clear optimization path for queuing operations and reduce waiting time.

[0033] Optionally, before obtaining the first information of the first data that has not yet been processed, the method as described above may further include:

[0034] Obtain information on the first currently schedulable resource, the first data that has not been processed, and the second resource information required to process the first data;

[0035] Based on the first data, the first resource information, and the second resource information, the submission status of the first data is determined. The submission status is used to indicate whether the first data can be processed at present.

[0036] When the submission status indicates that the first data could not be processed, determine the reason for the failure to process the data corresponding to the submission status.

[0037] In this embodiment of the application, the submission status and cause of the exception are automatically determined through a systematic process, thereby automating the process from problem detection to cause diagnosis and improving processing efficiency.

[0038] Secondly, this application provides a data processing apparatus, comprising:

[0039] The acquisition module is used to acquire first information about the first data that has not yet been processed. The first information includes the reason why the data has not been processed, the first resource information that is currently available for scheduling, and the second resource information required to process the first data.

[0040] The first determining module is used to determine the target partition corresponding to the reason for the failure to submit the first data when the reason for the failure to be processed indicates that the submission of the first data has failed, based on the reason for the failure to submit and the first resource information, and to determine the first correction strategy for the second resource information based on the resource information of the target partition.

[0041] The second determining module is used to determine a second correction strategy for the second resource information based on the first data and the first resource information when the reason for failure to process indicates that the first data was successfully submitted and is queued for processing.

[0042] The display module is used to display the first correction strategy or the second correction strategy.

[0043] Optionally, in the apparatus described above, the first determining module is specifically used to determine the target partition corresponding to the reason for the submission failure based on the reason for the submission failure and the first resource information, wherein the first determining module is used to:

[0044] Determine the reason type corresponding to the reason for the submission failure. The reason type is used to indicate the resource that caused the first data submission failure.

[0045] The target partition is determined based on the reason type corresponding to the submission failure and the first resource information.

[0046] Optionally, in the apparatus described above, the apparatus further includes a first processing module, which is specifically used for:

[0047] Determine the failure parameters when the first data submission fails;

[0048] Obtain the data association library, which includes multiple failure parameters and information describing each failure parameter.

[0049] In the data association database, match the information describing the failure parameters corresponding to the failure parameters when the first data submission fails to obtain the reason for the submission failure.

[0050] Optionally, in the apparatus described above, the target partition is determined based on the reason type corresponding to the submission failure and the first resource information. The first determining module is specifically used for:

[0051] Based on the reason type corresponding to the submission failure and the first resource information, determine at least one first partition that can process the first data;

[0052] In at least one first partition, determine the target partition corresponding to the cause type.

[0053] Optionally, in the apparatus described above, a first correction strategy for the second resource information is determined based on the resource information of the target partition, wherein the first determining module is specifically used for:

[0054] From the resource information in the target partition, obtain the schedulable target resource information that can process the first data, where the first resource information includes the target resource information;

[0055] Based on the target resource information and the second resource information, a first correction strategy for the second resource information is determined. The first correction strategy is used to configure resources for the second resource information of the first data. The first correction strategy includes the resource parameters that need to be corrected.

[0056] Optionally, in the apparatus described above, based on the first data and the first resource information, a second correction strategy for the second resource information is determined, wherein the second determining module is specifically used for:

[0057] Determine the waiting parameters corresponding to the first data waiting to be processed. The waiting parameters are used to indicate the reason why the first data is waiting to be processed.

[0058] Based on the waiting parameters, the first data, and the first resource information, a second correction strategy for the second resource information is determined. The second correction strategy is used to configure resources for the second resource information of the first data. The second correction strategy includes the resource queue to be corrected and the corresponding waiting time information.

[0059] Optionally, in the apparatus described above, before acquiring the first information of the first data that has not yet been processed, the apparatus further includes a second processing module, which is specifically used for:

[0060] Obtain information on the first currently schedulable resource, the first data that has not been processed, and the second resource information required to process the first data;

[0061] Based on the first data, the first resource information, and the second resource information, the submission status of the first data is determined. The submission status is used to indicate whether the first data can be processed at present.

[0062] When the submission status indicates that the first data could not be processed, determine the reason for the failure to process the data corresponding to the submission status.

[0063] Thirdly, this application provides an electronic device, including: a processor, and a memory communicatively connected to the processor;

[0064] The memory stores the instructions that the computer executes;

[0065] The processor executes computer-executable instructions stored in memory to implement the method described in the first aspect.

[0066] Fourthly, this application provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a computer, are used to implement the method described in the first aspect.

[0067] The computer-readable storage medium provided in this application embodiment can execute the technical solutions in the above method embodiments, and its beneficial effects are similar, so they will not be described again here.

[0068] Fifthly, this application provides a computer program product, including a computer program, which, when executed by a computer, is used to implement the method of the first aspect.

[0069] The computer program product provided in this application embodiment can execute the technical solutions in the above method embodiments, and its beneficial effects are similar, so they will not be described again here.

[0070] The data processing method, apparatus, electronic device, storage medium, and program product provided in this application obtain first information about first data that has not yet been processed. This first information includes the reason for the failure to process, currently schedulable first resource information, and first resource information required for processing. When the reason indicates submission failure, a target partition is determined based on the failure reason and the first resource information, and a first correction strategy for second resource information is determined based on the target partition resource information. When the reason indicates queuing, a second correction strategy for second resource information is determined based on the first data and the first resource information. In this method, the electronic device can dynamically generate targeted resource configuration correction strategies by adopting different analysis paths according to different anomaly types encountered during data processing (submission failure or queuing), effectively avoiding the low processing efficiency problem caused by ambiguous anomaly information in traditional solutions. It eliminates the need for user-based manual analysis, significantly improving the success rate and efficiency of data processing. Furthermore, by directly associating repair suggestions with specific configuration parameters and resource queues, this method achieves an integrated processing flow from problem diagnosis to correction guidance, enabling rapid resolution of resource scheduling and execution problems. Furthermore, this method is applicable to various operational scenarios of different scales and complexities in supercomputing internet, improving the reliability of data processing while increasing data processing success rate and resource utilization. Attached Figure Description

[0071] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0072] Figure 1 This is a schematic diagram of the system architecture provided for an embodiment of this application;

[0073] Figure 2 A flowchart illustrating a data processing method provided in an embodiment of this application;

[0074] Figure 3 A schematic diagram illustrating a method for determining the reason for failure to process a submission status, as provided in an embodiment of this application;

[0075] Figure 4 A flowchart illustrating another data processing method provided in an embodiment of this application;

[0076] Figure 5 A flowchart illustrating another data processing method provided in an embodiment of this application;

[0077] Figure 6 A schematic diagram of a job monitoring server provided in an embodiment of this application;

[0078] Figure 7A schematic diagram of the resource scheduling process provided in the embodiments of this application;

[0079] Figure 8 A schematic diagram illustrating the process of generating suggested prompts for embodiments of this application;

[0080] Figure 9 This is a schematic diagram of the structure of a data processing device provided in an embodiment of this application;

[0081] Figure 10 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application.

[0082] The accompanying drawings illustrate specific embodiments of this application, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the concept in any way, but rather to illustrate the concept of this application to those skilled in the art through reference to particular embodiments. Detailed Implementation

[0083] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0084] In supercomputing internet platforms, exception handling during computation task scheduling faces new challenges. Traditional methods suffer from three main problems: poor readability of error messages, reliance on human experience for solutions, and a lack of targeted remediation guidance. These issues lead to low execution efficiency of computation tasks and low utilization of computing resources.

[0085] In existing technologies, computational task anomalies are typically handled manually. Users need to adjust task parameters or seek technical support based on the brief error messages returned by the system and their personal experience.

[0086] However, this approach has obvious limitations: although there is an inherent connection between task anomaly information, resource status, and parameter configuration, due to the dispersed nature of the information presentation, manual analysis alone often fails to accurately pinpoint the root cause of the problem and is prone to yielding inappropriate solutions.

[0087] Specifically, the technical challenge of traditional exception handling methods lies in their inability to effectively correlate and analyze multi-dimensional scheduling information. For example, they cannot associate error codes with real-time resource status, nor can they distinguish between two different types of problems: task submission failure and task queuing.

[0088] Anomaly analysis typically requires comprehensive consideration of scheduling information from multiple dimensions. Users must repeatedly compare and analyze multiple independent information sources, such as error messages, resource status, and task parameters.

[0089] Traditional processing methods have the following inherent drawbacks: First, various types of scheduling information are independent of each other, lacking an effective correlation analysis mechanism; second, fragmented state information makes it difficult to construct a complete problem context; and finally, a single processing method cannot meet the differentiated analysis needs of different types of anomalies. These factors make it difficult to guarantee both the accuracy and efficiency of anomaly handling.

[0090] Furthermore, users need to manually develop analysis plans for different types of anomalies. Because the complete context of various anomalies cannot be predicted, and fixed processing procedures are difficult to adapt to dynamically changing cluster states, the troubleshooting effect is limited and the cost is high.

[0091] Another approach involves relying on technical support personnel for real-time analysis and manual judgment. However, technical personnel can usually only observe scattered and isolated states, making it difficult to grasp the complete task execution context and the relationships between various states. They often need to balance processing efficiency with the accuracy of problem localization.

[0092] To address this, this application proposes a data processing method that acquires relevant information about currently unprocessed tasks (including the cause of the anomaly, available resource information, and task resource requirements), and adopts differentiated processing strategies based on the anomaly type: for task submission failures, the method determines the target resource partition and generates a parameter correction scheme based on the cause of failure and resource status; for tasks waiting in a queue, the method generates a scheduling optimization scheme based on task characteristics and resource status. By combining multi-dimensional information collection with differentiated analysis strategies, the essential characteristics of different anomaly stages are effectively distinguished, achieving precise handling and automated guidance for computational task anomalies. This method can adapt to the needs of computing environments of different scales, avoiding the inefficiency of manual analysis while providing accurate solutions, significantly improving the processing efficiency and success rate of computational tasks.

[0093] Figure 1 For a schematic diagram of the system architecture provided in this application embodiment, please refer to [link / reference]. Figure 1 This includes electronic devices and terminal devices. Electronic devices can be any device with on-device computing capabilities, such as servers and terminal devices.

[0094] The terminal device and the electronic device can communicate with each other. The terminal device can send relevant data about computing tasks to the electronic device, and the electronic device can process the received data. After obtaining the processing result, the electronic device can send the processing result back to the terminal device and display it on the terminal device so that the user can perform relevant operations. In this embodiment, the electronic device can be a scheduling system for a high-performance computing cluster. The electronic device and the terminal device can be the same device, which can obtain relevant data during its own operation and process the data.

[0095] The technical solution of this application and how the technical solution of this application solves the above-mentioned technical problems are described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of this application will now be described with reference to the accompanying drawings.

[0096] Figure 2 This is a flowchart illustrating a data processing method provided in an embodiment of this application. Specifically, as shown... Figure 2 As shown, the method includes the following steps:

[0097] S201. Obtain the first information of the first data that has not yet been processed.

[0098] The execution subject of this application embodiment can be an electronic device or a data processing device installed in the electronic device. The data processing device can be implemented by software or by a combination of software and hardware.

[0099] The first information includes the reason why it cannot be processed at present, the first resource information that can be scheduled at present, and the second resource information required to process the first data.

[0100] The first data can be the data corresponding to the computing task to be submitted by the user, that is, the first data can be the job that the server cluster needs to process.

[0101] In some embodiments, in a high-performance computing cluster, the interaction between the user and the computing cluster is completed through a specific access point. Specifically, users typically interact with the high-performance computing cluster through a login node and cannot directly access the computing nodes. The ways in which users submit the first data include, but are not limited to, the following three forms: (1) Interactive submission: Users submit computing tasks in real time through a command-line interface, which is suitable for debugging and immediate computing needs; (2) Batch submission: Users submit batch computing tasks through job scripts, which is suitable for large-scale scientific computing; (3) API interface submission: Users submit tasks programmatically through an application programming interface, which is suitable for integrated workflows.

[0102] The first information is used to process the first data. That is, the first information can provide context information for the processing of the first data, provide data support for subsequent resource allocation and task scheduling, thereby ensuring that the first data can be successfully processed and optimizing the overall processing efficiency.

[0103] The reason for failure to process is used to indicate the root cause of the failure to process the first piece of data. The reasons for failure to process can be submission failure and waiting in the queue.

[0104] In some embodiments, after the electronic device obtains the reason why the first data failed to be processed, the electronic device can obtain an identifier corresponding to the reason why the data failed to be processed. For example, the identifier can be an error code, i.e., a feature used to indicate that the current first data submission failed.

[0105] The first resource information can be the set of idle computing resources in the current computing cluster. The first resource information can include the resource configuration information, load status and queue information of each computing node.

[0106] In some embodiments, the electronic device can obtain the resource configuration information of the computing cluster, parse the configuration information to obtain the resource information of each partition in the computing cluster, and integrate the resource information such as the number of idle computing nodes, the number of idle processor cores, the capacity of idle memory and the number of idle accelerator cards in each partition into the first resource information based on the resource information of each partition.

[0107] The second resource information can be the resource configuration information required by the first data, that is, the second resource information can be the resource requirement information pre-configured by the user. The second resource information may include the number of processor cores requested, memory capacity, and number of accelerator cards, etc.

[0108] In some embodiments, the electronic device can obtain user-submitted jobs (first data) through an interface, preprocess the first data (e.g., format conversion), and parse the preprocessed first data to obtain user-defined resource requirement information, i.e., second resource information.

[0109] S202. When the reason for failure to process indicates that the first data submission failed, the target partition corresponding to the reason for the submission failure is determined according to the reason for the submission failure and the first resource information, and the first correction strategy for the second resource information is determined according to the resource information of the target partition.

[0110] The reason for submission failure is used to indicate the constraints that the computation task cannot be processed. For example, if the reason for failure indicates that the first data submission failed, the reason for submission failure could be resource request limit exceeded, node number limit exceeded, etc.

[0111] In some embodiments, the electronic device may obtain the reason for submission failure based on the following implementation: determining the failure parameters when the first data submission fails; obtaining a data association library; and matching the information describing the failure parameters corresponding to the failure parameters when the first data submission fails in the data association library to obtain the reason for submission failure.

[0112] The data association library includes multiple failure parameters and information describing each failure parameter.

[0113] Failure parameters can be parameters related to submission failure; for example, failure parameters can be error codes returned by the scheduling system.

[0114] The information describing the failure parameters can be a structured description corresponding to the error code. For example, the information describing the failure parameters may include constraints, resource types, etc.

[0115] For example, if the failure parameter is 1001, the corresponding information describing the failure parameter could be that too many processor cores have been requested; if the failure parameter is 1002, the corresponding information describing the failure parameter could be that a resource queue request error has occurred.

[0116] In some embodiments, the data association library can be a pre-defined database of reasons for submission failure. That is, the electronic device can obtain the data association library by establishing the association between the failure parameters and the corresponding information describing the failure parameters based on the reasons for submission failures during the user's job submission process in historical time periods. In this way, through such a structured data association library, the electronic device can quickly convert the original failure parameters returned by the scheduling system into submission failure reasons with clear semantics, providing accurate input basis for subsequent target partition determination and correction strategy generation.

[0117] In some embodiments, after the electronic device obtains the failure parameters corresponding to the first data, the electronic device can sequentially match the failure parameters corresponding to the first data with the failure parameter list in the data association library to obtain the information describing the failure parameters corresponding to the successfully matched failure parameters, that is, the reason for the submission failure.

[0118] For example, the data association library includes parameters A and B. The information corresponding to the failure parameter corresponding to parameter A is that too many processor cores have been requested, and it is recommended to reduce the number of processor cores. The information corresponding to the failure parameter corresponding to parameter B is that the resource queue request does not meet the requirements, and it is recommended to choose another resource queue. If the failure parameter corresponding to the first data is parameter A, then the reason for the submission failure is that too many processor cores have been requested, and it is recommended to reduce the number of processor cores.

[0119] The target partition can be the partition containing currently available computing resources whose resource configuration meets the primary data processing requirements.

[0120] In some embodiments, the electronic device may determine the target partition corresponding to the reason for submission failure based on the reason for submission failure and the first resource information in the following manner: determining the reason type corresponding to the reason for submission failure; determining the target partition based on the reason type corresponding to the reason for submission failure and the first resource information.

[0121] The reason type indicates the resource that caused the first data submission to fail. For example, the reason type can include resource specification exceeding limits, insufficient available resources, and resource mismatch. Resource specification exceeding limits can include exceeding the available limit for the number of processor cores, exceeding the limit for memory capacity per node, or exceeding the borrowing limit for the number of required compute nodes. Insufficient available resources can mean that the current partition has no compute resources. Resource mismatch can mean that the specified resource type is incompatible with the resource type of the current partition.

[0122] In some embodiments, the electronic device may determine the target partition based on the reason type corresponding to the reason for the submission failure and the first resource information, by: determining at least one first partition capable of processing the first data based on the reason type corresponding to the reason for the submission failure and the first resource information; and determining the target partition corresponding to the reason type in the at least one first partition.

[0123] The first partition can be a set of candidate partitions determined from the first resource information according to preset filtering conditions.

[0124] In some embodiments, the electronic device can determine the constraints associated with the resource based on the reason type corresponding to the reason for the submission failure, and filter out the candidate partitions that meet the constraints in the first resource information, i.e., the first partition. Specifically, the candidate partitions that meet the constraints can be filtered out in the first resource information based on the resource configuration information of each partition, the schedulable status of the resources in each partition, and the access permissions of each partition.

[0125] After the electronic device identifies multiple first partitions, it can determine the target partition from among them based on preset conditions. The preset conditions are pre-defined selection criteria for the target partition. For example, the preset conditions include, but are not limited to, the matching degree of resource specifications and the current load status of each first partition. In this way, through the above filtering strategy, the most suitable target partition can be accurately matched for tasks that fail to be submitted, and an accurate basis can be provided for subsequent correction of resource parameters.

[0126] Specifically, for reasons related to exceeding resource limits, partitions with sufficient resource capacity to meet the corrected requirements and low load are prioritized; for reasons related to insufficient resource availability, partitions with the required resource combination and currently idle are prioritized; for reasons related to policy and permission restrictions, alternative partitions with full access permissions are selected; and finally, the optimal partition is selected as the target partition by comprehensively evaluating multiple dimensions such as the resource matching degree, real-time load status, and expected waiting time of each first partition.

[0127] In some embodiments, the electronic device may determine a first correction strategy for the second resource information based on the resource information of the target partition, according to the following implementation: obtaining schedulable target resource information capable of processing the first data from the resource information in the target partition; and determining the first correction strategy for the second resource information based on the target resource information and the second resource information.

[0128] The first resource information includes target resource information.

[0129] The first correction strategy is used to configure resources based on the second resource information of the first data. The first correction strategy includes the resource parameters that need to be corrected.

[0130] The target resource information can be a set of schedulable resources in the target partition that meet the primary data processing requirements.

[0131] In some embodiments, the electronic device can determine the target resource information based on the real-time idle resource quantity of the target partition, the resource load trend of the partition, and the current scheduling rules of the partition. The real-time idle resource quantity can be the amount of resources that are not currently occupied, the resource load trend can be the amount of resources to be released within a preset time period, and the scheduling rules can include the maximum allowed runtime, the upper limit of the number of concurrent jobs, etc. In this way, the set of available resources that can meet both the first data processing requirements and the current scheduling conditions can be accurately extracted from the resource information of the target partition, forming structured target resource information, which provides an accurate resource basis for the generation of subsequent correction strategies.

[0132] For example, if the range of the target resource information is greater than or equal to the range of the second resource information, the first correction strategy can be "maintain the original request" or "moderately optimize". For instance, when resources are abundant, the task can be allowed to use all the resources it requests. If the range of the target resource information is less than the range of the second resource information, the first correction strategy is "downward correction", which includes, but is not limited to: reducing the number of computing cores requested, reducing the memory capacity requested, or extending the estimated task runtime to match the lower resource quota. Furthermore, the first correction strategy will clearly identify the specific resource parameters that need to be corrected and their suggested values ​​or correction ranges.

[0133] In some embodiments, the electronic device may establish a resource parameter comparison mechanism to compare and analyze the resource request parameters (including the number of requested nodes, memory capacity, processor type, etc.) in the second resource information with the resource quota parameters and real-time availability in the target resource information item by item; and generate differentiated correction schemes based on the comparison results.

[0134] Specifically, when the resource request value exceeds the target resource limit, a specific parameter modification instruction is generated to adjust the request value to within the limit range. When the resource request value is lower than the recommended configuration of the target partition, parameter upgrade suggestions for performance optimization are provided. When the resource type does not match, a resource type conversion scheme is provided. Combining historical execution data and performance optimization goals, the correction scheme is further optimized, and a structured first correction strategy containing specific modified parameters, modification basis, and expected effects is output to ensure that the corrected resource configuration not only meets the actual constraints of the target partition but also optimizes the processing efficiency of the first data.

[0135] For example, when a resource request value is detected to exceed the limit (e.g., requesting 32 nodes but the limit is 16 nodes), specific parameter modification instructions may be generated (e.g., the first correction strategy is "change nodes=32 to --nodes=16"); when the resource request value is lower than the recommended configuration (e.g., requesting 64GB of memory but the partition is configured with 256GB), performance optimization suggestions will be provided (e.g., the first correction strategy is "increase to 128GB to reduce I / O wait"). These first correction strategies are all fine-tuned by combining historical execution data and performance optimization goals, and the final output is a structured solution containing specific modified parameters, technical basis and expected effects, which not only ensures that the resource configuration meets the actual constraints of the partition, but also significantly improves data processing efficiency and resource utilization.

[0136] In some embodiments, when the electronic device determines the first correction strategy for the second resource information, the electronic device can classify the reasons for submission failure and generate differentiated correction strategies based on the classification results. Specifically, the electronic device can classify the reasons for submission failure (error codes) according to the severity of the error codes. For example, based on predefined error code classification rules, the reasons for submission failure (error codes) can be divided into three levels, including "serious error", "warning" and "prompt".

[0137] Among them, "Critical Error" is used to indicate a mandatory restriction problem that causes the job to be unprocessed, "Warning" is used to indicate a potential configuration problem that affects the job's execution efficiency or success rate, and "Tips" are used to indicate runtime reference information that does not affect job submission.

[0138] After determining the classification results, the electronic device can obtain the corresponding first correction strategy based on the classification level. Taking the three levels of "critical error", "warning", and "hint" as examples, if it is a critical error, a mandatory correction suggestion is generated. This suggestion provides a limited set of correction solutions that must be implemented and imposes submission restrictions on the user interface. If it is a warning, an optimization correction suggestion is generated. This suggestion explains the potential problem and provides recommended optimization solutions, but allows the user to ignore it. If it is a hint, reference information is generated, which only provides context to assist in decision-making and does not contain operation instructions. In this way, through the above classification process, it is ensured that critical issues are forcibly resolved to improve the success rate of the job, while optimizing the user experience by reducing interference with non-critical issues.

[0139] S203. When the reason for failure to process indicates that the first data was successfully submitted and is queued for processing, a second correction strategy for the second resource information is determined based on the first data and the first resource information.

[0140] In some embodiments, the electronic device may determine a second correction strategy for the second resource information based on the first data and the first resource information in the following manner: determining a waiting parameter corresponding to the waiting processing of the first data; and determining a second correction strategy for the second resource information based on the waiting parameter, the first data, and the first resource information.

[0141] The waiting parameter is used to indicate the reason why the first data is waiting to be processed. That is, there is a mapping relationship between the waiting parameter and the reason for waiting to be processed. In this embodiment of the application, the waiting parameter can be a structured error code or status code.

[0142] In some embodiments, the electronic device can obtain a preset rule base, in which the reason for the waiting process corresponding to the waiting parameter corresponding to the first data waiting process is matched. The preset rule base is a pre-set rule base that includes multiple waiting parameters and information on the waiting processes corresponding to each waiting parameter.

[0143] In some embodiments, after the electronic device determines the waiting parameters, it can determine a second correction strategy for the second resource information based on the characteristics of the first data itself (e.g., task type, urgency, etc.) and the first resource information. Specifically, if the waiting parameter indicates that the reason for the waiting is insufficient resources and the urgency of the first data is low, the suggested direction of the second correction strategy is to reduce the number of resource requests, for example, the second correction strategy is to suggest reducing the number of processor cores requested from 16 cores to 10 cores, with an expected reduction in waiting time of 1 hour. If the urgency of the first data is high, the suggested direction of the second correction strategy is to switch resource queues, for example, the second correction strategy is to suggest requesting cluster resource queue A, with an expected waiting time of less than 1 hour.

[0144] The second correction strategy is used to configure resources based on the second resource information of the first data. The second correction strategy includes the resource queue to be corrected and the corresponding waiting time information.

[0145] S204, Display the first correction strategy or the second correction strategy.

[0146] After the electronic device determines the first correction strategy or the second correction strategy, the electronic device can send the correction strategy to the terminal device and display it on the display interface of the terminal device. The user can modify the first data and the second resource information according to the displayed correction strategy and submit the modified first data and the second resource information. After receiving the modified data from the user, the electronic device processes it according to the above embodiment.

[0147] In some embodiments, the electronic device can monitor the user's behavior information regarding the correction strategy and the submission status of the modified data submitted by the user in real time. Specifically, if the user adopts the correction strategy and submits successfully, the information of adopting the suggestion and submitting successfully can be logged. If the user adopts the correction strategy but the submission fails or is pending processing, the information of adopting the suggestion but the submission fails or is pending processing can be logged. If the user does not adopt the correction strategy, the information of not adopting the suggestion and the corresponding specific reason can be logged.

[0148] In this embodiment, the electronic device can employ reinforcement learning to continuously iterate and optimize the correction strategy. Specifically, user adoption rate and job success rate can be used as core reward functions to construct a reinforcement learning model. This model dynamically adjusts the generation strategy of repair suggestions by taking user feedback logs as input. For example, for correction strategies with high user adoption rate and high job success rate, the model will increase their recommendation priority in similar scenarios. For strategies marked as "incompatible with requirements," the model will optimize the rule base based on feedback reasons (e.g., adding matching verification between job type and resource requirements, or adjusting the weight parameters of partition load) to make the correction strategy more in line with the resource dynamics of the supercomputing platform and the actual needs of users. In this way, the correction strategy can be continuously optimized in practical applications, forming a virtuous cycle of "strategy generation - user feedback - model optimization - strategy iteration," gradually improving the intelligence level of job anomaly handling and user experience.

[0149] The data processing method, apparatus, electronic device, storage medium, and program product provided in this application obtain first information about first data that has not yet been processed. This first information includes the reason for the failure to process, currently schedulable first resource information, and first resource information required for processing. When the reason indicates submission failure, a target partition is determined based on the failure reason and the first resource information, and a first correction strategy for second resource information is determined based on the target partition resource information. When the reason indicates queuing, a second correction strategy for second resource information is determined based on the first data and the first resource information. In this method, the electronic device can dynamically generate targeted resource configuration correction strategies by adopting different analysis paths according to different anomaly types encountered during data processing (submission failure or queuing), effectively avoiding the low processing efficiency problem caused by ambiguous anomaly information in traditional solutions. It eliminates the need for user-based manual analysis, significantly improving the success rate and efficiency of data processing. Furthermore, by directly associating repair suggestions with specific configuration parameters and resource queues, this method achieves an integrated processing flow from problem diagnosis to correction guidance, enabling rapid resolution of resource scheduling and execution problems. Furthermore, this method is applicable to various operational scenarios of different scales and complexities in supercomputing internet, improving the reliability of data processing while increasing data processing success rate and resource utilization.

[0150] Based on any of the above embodiments, the method for determining the reason for failure to process corresponding to the submission status in the above processing method before the electronic device obtains the first information of the first data that has not been processed will be described in detail.

[0151] Figure 3 This is a schematic diagram illustrating a method for determining the reason for a submission status failure, provided in an embodiment of this application. Please refer to... Figure 3 The method includes the following steps:

[0152] S301. Obtain the first schedulable resource information, the first data that has not been processed, and the second resource information required to process the first data.

[0153] In some embodiments, the electronic device can obtain currently schedulable first resource information based on resource information in the computing cluster, and can receive first data and second resource information submitted by the user, and monitor the user's submission behavior data in real time. When an abnormal user submission behavior is detected, the corresponding first data is determined as first data that has not been processed, and the second resource information required to determine the first data is also determined.

[0154] S302. Determine the submission status of the first data based on the first data, the first resource information, and the second resource information.

[0155] The submission status indicates whether the first data can be processed at present.

[0156] In some embodiments, the submission status can be a submission success status and a submission failure status. The submission success status is used to indicate that the first data has passed the preliminary verification and can be processed, and the submission failure status is used to indicate that the first data has failed the verification and cannot be processed. The submission success status may also include a submission success and not waiting for processing status and a submission success but waiting for processing status.

[0157] In this context, the "submitted successfully and not waiting for processing" status indicates that the second resource information required by the first data can be satisfied by the currently schedulable first resource information and conforms to the scheduling policy. Therefore, the computing cluster can directly allocate resources and begin execution. This status typically corresponds to "started running" in the task status. The "submitted successfully but waiting for processing" status indicates that the first data has been successfully received, but due to insufficient system resources, resource fragmentation, or scheduling policies, it cannot be immediately allocated all the required resources and needs to enter a waiting queue. This status typically corresponds to "queuing" in the task status.

[0158] S303. When the submission status indicates that the first data cannot be processed, determine the reason for the failure to process the data corresponding to the submission status.

[0159] In some embodiments, when the submission status indicates that the first data cannot be processed at present, it may include a submission failure status and a submission success but waiting for processing status. If it is a submission failure status, the reason for failure to process the submission status indicates that the submission of the first data failed. If it is a submission success but waiting for processing status, the reason for failure to process the submission status indicates that the first data was successfully submitted and is waiting in the queue for processing.

[0160] The data processing method, apparatus, electronic device, storage medium, and program product provided in this application determine the submission status of the first data by acquiring currently schedulable first resource information, first data that has not been processed, and the required second resource information. When the status indicates that the data has not been processed, the root cause is accurately determined. In this method, the electronic device finely distinguishes between "submission failed" and "submission successful but awaiting processing" submission statuses and further determines the specific cause, enabling rapid problem localization after an initial scheduling attempt. This effectively avoids the problems of repeated blind retries and resource waste caused by unclear causes in traditional scheduling systems, eliminating the need for full resource matching simulation calculations and significantly improving diagnostic efficiency. The correction strategy (first or second correction strategy) based on the accurately generated cause greatly improves the success rate of task resource configuration optimization and user satisfaction. Furthermore, this diagnostic method is applicable to computing clusters of different sizes and diverse task types, reducing the cost of manual troubleshooting for maintenance personnel. By improving the intelligence of cluster resource scheduling, the overall throughput and efficiency of the computing cluster are improved.

[0161] Figure 4 This is a flowchart illustrating another data processing method provided in an embodiment of this application. Please refer to... Figure 4 The method includes the following steps:

[0162] S401. Obtain the first schedulable resource information, the first data that has not been processed, and the second resource information required to process the first data.

[0163] S402. Determine the submission status of the first data based on the first data, the first resource information, and the second resource information.

[0164] The submission status indicates whether the first data can be processed at present.

[0165] S403. When the submission status indicates that the first data cannot be processed, determine the reason for the failure to process the data corresponding to the submission status.

[0166] It should be noted that the execution process of S401-S403 can be found in S301-S303, and will not be repeated here.

[0167] S404. Obtain the first information of the first data that has not yet been processed.

[0168] The first information includes the reason why it cannot be processed at present, the first resource information that can be scheduled at present, and the second resource information required to process the first data.

[0169] It should be noted that the execution process of S404 above can be found in S201, and will not be repeated here.

[0170] S405. When the reason for failure to process indicates that the first data submission failed, determine the reason type corresponding to the reason for the submission failure.

[0171] The reason type is used to indicate the resource that caused the first data submission to fail.

[0172] S406. Based on the reason type corresponding to the reason for the submission failure and the first resource information, determine at least one first partition that can process the first data.

[0173] S407. In at least one first partition, determine the target partition corresponding to the cause type.

[0174] S408. Obtain the schedulable target resource information that can process the first data from the resource information in the target partition.

[0175] The first resource information includes target resource information.

[0176] S409. Based on the target resource information and the second resource information, determine the first correction strategy for the second resource information.

[0177] The first correction strategy is used to configure the second resource information of the first data, and the first correction strategy includes the resource parameters to be corrected.

[0178] It should be noted that the execution process of S405-S409 above can be found in S202, and will not be repeated here.

[0179] S410. When the reason for failure to process indicates that the first data was successfully submitted and is waiting in the queue for processing, determine the waiting parameter corresponding to the first data waiting to be processed.

[0180] The waiting parameter indicates the reason why the first data is waiting to be processed.

[0181] S411. Based on the waiting parameters, the first data, and the first resource information, determine the second correction strategy for the second resource information.

[0182] The second correction strategy is used to configure resources based on the second resource information of the first data. The second correction strategy includes the resource queue to be corrected and the corresponding waiting time information.

[0183] In some embodiments, the electronic device can predict the first data submitted by the user through a preset model. Specifically, in the case of submission failure, the electronic device can predict the resource adjustment plan for the current first data based on the first data submitted by the user and the required second resource information, historical data with similar characteristics to the first data and the required resource information. For the first data that is successfully submitted but queued for processing, the electronic device can predict the queuing time and adjustment plan for the current first data based on the cluster's resource information, including the running resource information and the currently idle first resource information, as well as the first data and the required second resource information. In this way, the user can modify or optimize in advance according to the resource adjustment plan to improve the success rate of processing the data submitted by the user during the submission and queuing stages.

[0184] The data processing method provided in this application is similar in principle and beneficial effects to the above embodiments, and will not be described again here.

[0185] The following specific examples illustrate the data processing methods.

[0186] Figure 5 This is a flowchart illustrating another data processing method provided in an embodiment of this application. Please refer to... Figure 5 The specific process is as follows: The user submits a computation job (S501); the user's submission behavior is monitored (S502); it is determined whether the job submission is successful (S503). If the submission is successful, a job record is generated (S504); it is determined whether the job has been successfully allocated resources (S505). If the allocation is successful, the job is run (S506); if the allocation fails, the failure details are captured and a second correction strategy is generated (S507); if the submission fails, the failure details are captured (S508), and based on the reason for the submission failure, a guidance prompt, i.e., the first correction strategy, is analyzed (S509); after obtaining the first correction strategy or the second correction strategy, the correction strategy is displayed to the user (S510).

[0187] In some embodiments, after fully studying the existing information prompting mechanism of the scheduling system, this solution designs a user-friendly interactive implementation method for the submission and queuing stages of the job lifecycle. That is, for jobs that fail to submit, the solution captures the failure details (e.g., the number of requested nodes exceeds the maximum limit of available resources), analyzes and generates guidance prompts based on this, and directly displays the specific reasons to the user through the interactive interface. For example, the interactive interface directly displays the number of requested nodes and the maximum number of nodes that are restricted to the user, and intelligently generates guidance prompts based on the error code generated by this error, suggesting that the user reduce the number of nodes to a certain limit.

[0188] For successfully submitted jobs, resources are allocated to them. If resource allocation is successful, the job enters the running state. If resource allocation fails (i.e., the job enters the queue), the behavior of each scheduling cycle is dynamically monitored, and the detailed reasons for resource allocation failure are captured (e.g., the user's concurrent job limit of 10 has been reached, causing this allocation failure). Targeted guidance prompts are analyzed and generated, suggesting that if the user urgently needs to run the job, they can pause the currently running job or contact customer service to grant greater permissions.

[0189] Below, in conjunction with Figure 6 The job monitoring server will be explained through specific examples.

[0190] Figure 6 For a schematic diagram of the job monitoring server provided in the embodiments of this application, please refer to [link / reference]. Figure 6 The system comprises a client and a server. Users can submit jobs on the client (terminal device), including interactive, batch, and API job submission types. The server includes a listening server and a resource scheduling server. The listening server monitors various job submission requests from clients in real time and sends job submission information to the resource scheduling server. Electronic devices can then allocate resources and run jobs through the resource scheduling server. It's important to note that multiple interfaces are involved. To listen for all types of job submission requests, the listening location can be set directly on the scheduling system server, or the client can listen for each type of job submission to uniformly capture all submission behavior, ensuring that the submission behavior of every single job is recorded.

[0191] Below, in conjunction with Figure 7 The resource scheduling process is illustrated through specific examples.

[0192] Figure 7 For a schematic diagram of the resource scheduling process provided in the embodiments of this application, please refer to [link / reference]. Figure 7 In this embodiment of the application, it is necessary to capture resource scheduling behavior and show the user the reason for failure. Since resource scheduling is a periodic behavior in the scheduling service, a mechanism for capturing the details of the reasons for failed job allocation attempts can be added to the scheduling thread to provide detailed explanations for jobs that failed to submit and jobs that failed to allocate resources and were queued.

[0193] The specific process is as follows: Electronic devices perform resource scheduling through the scheduling system (S701); it is determined whether the resource allocation was successful (S702). If the allocation is successful, the job is run, i.e., the computation task submitted by the user is executed (S703). If the allocation fails, a failure log is printed (S704), and failure details are simultaneously obtained through listening logs (S705), and returned to the client, i.e., the failure result and reason are fed back to the user, so that the user can adjust the job configuration (S706); after the resource scheduling of this job is completed, the resource scheduling of the next job is performed (S707). It is worth noting that obtaining failure details does not necessarily require direct collection of log files on the server side.

[0194] Below, in conjunction with Figure 8 The process of generating suggestion prompts is explained through specific examples.

[0195] Figure 8 For a schematic diagram illustrating the process of generating suggested prompts in the embodiments of this application, please refer to [link / reference]. Figure 8 The electronic device generates guidance prompts based on the failure error codes: After the job is scheduled and allocated resources on the server, if it fails, it will return an error code to the client. Based on the returned error code, the guidance message is matched from the template, and finally, the user is shown information suggesting modifications to the job.

[0196] The specific process is as follows: If the job resource allocation fails (S801), a structured error code is returned to the client (S802). The electronic device can obtain the error code and match it with a predefined rule base (S803). After obtaining the reason for the submission failure, a suggestion prompt, namely the first correction strategy (S804), is generated and displayed to the terminal to facilitate the user to adjust the job configuration (S805). In this way, the user can directly adjust the job resource request or configuration according to this intelligent prompt and resubmit, thereby significantly improving the efficiency of problem solving and user experience, forming an efficient closed loop from system diagnosis to user correction.

[0197] Figure 9 This is a schematic diagram of the structure of a data processing device provided in an embodiment of this application, as shown below. Figure 9 As shown, the device 90 includes: an acquisition module 91, a first determination module 92, a second determination module 93, and a display module 94.

[0198] The acquisition module 91 is used to acquire first information of the first data that has not been processed. The first information includes the reason why it has not been processed, the first resource information that is currently schedulable, and the second resource information required to process the first data.

[0199] The first determining module 92 is used to determine the target partition corresponding to the reason for the failure to submit the first data when the reason for the failure to submit the first data indicates that the submission failed, based on the reason for the failure and the first resource information, and to determine the first correction strategy for the second resource information based on the resource information of the target partition.

[0200] The second determining module 93 is used to determine a second correction strategy for the second resource information based on the first data and the first resource information when the reason for failure to process indicates that the first data was successfully submitted and is queued for processing.

[0201] Display module 94 is used to display the first correction strategy or the second correction strategy.

[0202] The data processing apparatus provided in this application embodiment can execute the technical solution shown in the above method embodiment. Its implementation principle and beneficial effects are similar, and will not be described again here.

[0203] In one possible implementation, the first determining module 92 is specifically used for:

[0204] Determine the reason type corresponding to the reason for the submission failure. The reason type is used to indicate the resource that caused the first data submission failure.

[0205] The target partition is determined based on the reason type corresponding to the submission failure and the first resource information.

[0206] In one possible implementation, the apparatus further includes a first processing module, which is specifically used for:

[0207] Determine the failure parameters when the first data submission fails;

[0208] Obtain the data association library, which includes multiple failure parameters and information describing each failure parameter.

[0209] In the data association database, match the information describing the failure parameters corresponding to the failure parameters when the first data submission fails to obtain the reason for the submission failure.

[0210] In one possible implementation, the first determining module 92 is specifically used for:

[0211] Based on the reason type corresponding to the submission failure and the first resource information, determine at least one first partition that can process the first data;

[0212] In at least one first partition, determine the target partition corresponding to the cause type.

[0213] In one possible implementation, the first determining module 92 is specifically used for:

[0214] From the resource information in the target partition, obtain the schedulable target resource information that can process the first data, where the first resource information includes the target resource information;

[0215] Based on the target resource information and the second resource information, a first correction strategy for the second resource information is determined. The first correction strategy is used to configure resources for the second resource information of the first data. The first correction strategy includes the resource parameters that need to be corrected.

[0216] In one possible implementation, the second determining module 93 is specifically used for:

[0217] Determine the waiting parameters corresponding to the first data waiting to be processed. The waiting parameters are used to indicate the reason why the first data is waiting to be processed.

[0218] Based on the waiting parameters, the first data, and the first resource information, a second correction strategy for the second resource information is determined. The second correction strategy is used to configure resources for the second resource information of the first data. The second correction strategy includes the resource queue to be corrected and the corresponding waiting time information.

[0219] In one possible implementation, the apparatus further includes a second processing module, which is specifically used for:

[0220] Obtain information on the first currently schedulable resource, the first data that has not been processed, and the second resource information required to process the first data;

[0221] Based on the first data, the first resource information, and the second resource information, the submission status of the first data is determined. The submission status is used to indicate whether the first data can be processed at present.

[0222] When the submission status indicates that the first data could not be processed, determine the reason for the failure to process the data corresponding to the submission status.

[0223] The data processing apparatus provided in this application embodiment can execute the technical solution shown in the above method embodiment. Its implementation principle and beneficial effects are similar, and will not be described again here.

[0224] Figure 10 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application, such as... Figure 10 As shown, the electronic device 10 may include at least one processor 11 and a memory 12.

[0225] The memory 12 is used to store programs. Specifically, the program may include program code, which includes computer-executable instructions.

[0226] The memory 12 may include random access memory (RAM) and may also include non-volatile memory, such as at least one disk storage.

[0227] The processor 11 is used to execute computer execution instructions stored in the memory 12 to implement the methods described in the foregoing method embodiments. The processor 11 may be a CPU, an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of this application.

[0228] Optionally, the electronic device 10 may also include a communication interface 13. In specific implementations, if the communication interface 13, memory 12, and processor 11 are implemented independently, then the communication interface 13, memory 12, and processor 11 can be interconnected via a bus to complete communication between them. The bus can be an industry standard architecture (ISA) bus, a peripheral component (PCI) bus, or an extended industry standard architecture (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc., but this does not imply that there is only one bus or one type of bus.

[0229] Optionally, in a specific implementation, if the communication interface 13, memory 12 and processor 11 are integrated on a single chip, then the communication interface 13, memory 12 and processor 11 can communicate through an internal interface.

[0230] The electronic device in this embodiment can be used to execute the technical solutions of the above method embodiments. The specific implementation methods and technical effects are similar, and will not be repeated here.

[0231] This application provides a computer-readable storage medium, which may include various media capable of storing computer-executable instructions, such as a USB flash drive, a portable hard drive, a read-only memory (ROM), RAM, a disk, or an optical disk. Specifically, the computer-readable storage medium stores computer-executable instructions, which, when executed by a computer, cause the technical solution shown in the above method embodiment to be executed. The specific implementation and technical effects are similar and will not be repeated here.

[0232] This application provides a computer program product, including a computer program. When the computer program is executed by a computer, the technical solution shown in the above method embodiment is executed. The specific implementation method and technical effect are similar, and will not be repeated here.

[0233] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, as some steps may be performed in other orders or simultaneously according to this application. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are all optional embodiments, and the actions and modules involved are not necessarily essential to this application.

[0234] It should be further noted that although the steps in the flowchart are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowchart may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the sub-steps or stages of other steps.

[0235] It should be understood that the above-described device embodiments are merely illustrative, and the device of this application can also be implemented in other ways. For example, the division of units / modules in the above embodiments is only a logical functional division, and there may be other division methods in actual implementation. For example, multiple units, modules, or components may be combined, or integrated into another system, or some features may be ignored or not executed.

[0236] Furthermore, unless otherwise specified, the functional units / modules in the various embodiments of this application can be integrated into one unit / module, or each unit / module can exist physically separately, or two or more units / modules can be integrated together. The integrated units / modules described above can be implemented in hardware or as software program modules.

[0237] If the integrated unit / module is implemented as a software program module and sold or used as an independent product, it can be stored in a computer-readable storage device (CMD). Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a memory and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned memory includes various media capable of storing program code, such as a USB flash drive, read-only memory (ROM), random access memory (RAM), portable hard drive, magnetic disk, or optical disk.

[0238] In the above embodiments, the descriptions of each embodiment have their own emphasis. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments. The technical features of the above embodiments can be combined arbitrarily. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as the combination of these technical features does not contradict each other, it should be considered within the scope of this specification.

[0239] Other embodiments of this application will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this application that follow the general principles of this application and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this application are indicated by the following claims.

[0240] It should be understood that this application is not limited to the precise structure described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this application is limited only by the appended claims.

Claims

1. A data processing method, characterized in that, include: Obtain first information about the first data that has not yet been processed, including the reason why it has not been processed, first information about the first resources that are currently available for scheduling, and second information about the resources required to process the first data. When the reason for failure indicates that the submission of the first data failed, the target partition corresponding to the reason for the submission failure is determined according to the reason for the submission failure and the first resource information, and a first correction strategy for the second resource information is determined according to the resource information of the target partition. When the reason for failure to process indicates that the first data was successfully submitted and is queued for processing, a second correction strategy for the second resource information is determined based on the first data and the first resource information. Display the first correction strategy or the second correction strategy.

2. The method according to claim 1, characterized in that, Based on the reason for the submission failure and the first resource information, determine the target partition corresponding to the reason for the submission failure, including: Determine the reason type corresponding to the reason for the submission failure, wherein the reason type is used to indicate the resource that caused the first data submission failure; The target partition is determined based on the reason type corresponding to the submission failure and the first resource information.

3. The method according to claim 2, characterized in that, The method includes: Determine the failure parameters when the first data submission fails; Obtain the data association library, which includes multiple failure parameters and information describing each failure parameter. In the data association database, information describing the failure parameters corresponding to the failure parameters when the first data submission fails is matched to obtain the reason for the submission failure.

4. The method according to claim 2, characterized in that, Based on the reason type corresponding to the submission failure and the first resource information, the target partition is determined, including: Based on the reason type corresponding to the submission failure and the first resource information, at least one first partition capable of processing the first data is determined. In the at least one first partition, the target partition corresponding to the cause type is determined.

5. The method according to any one of claims 1-4, characterized in that, Based on the resource information of the target partition, a first correction strategy for the second resource information is determined, including: From the resource information in the target partition, obtain schedulable target resource information that can process the first data, wherein the first resource information includes the target resource information; Based on the target resource information and the second resource information, a first correction strategy for the second resource information is determined. The first correction strategy is used to configure resources for the second resource information of the first data. The first correction strategy includes the resource parameters to be corrected.

6. The method according to claim 1, characterized in that, Based on the first data and the first resource information, a second correction strategy for the second resource information is determined, including: Determine the waiting parameters corresponding to the first data waiting to be processed, wherein the waiting parameters are used to indicate the reason for the first data waiting to be processed; Based on the waiting parameters, the first data, and the first resource information, a second correction strategy for the second resource information is determined. The second correction strategy is used to configure resources for the second resource information of the first data. The second correction strategy includes the resource queue to be corrected and the corresponding waiting time information.

7. The method according to claim 1, characterized in that, Before obtaining the first information about the first data that has not yet been processed, the method further includes: Obtain information on currently schedulable first resources, first data that has not been processed, and second resource information required to process the first data; Based on the first data, the first resource information, and the second resource information, the submission status of the first data is determined, and the submission status is used to indicate whether the first data can be processed at present; When the submission status indicates that the first data cannot be processed, determine the reason for the failure to process the data corresponding to the submission status.

8. A data processing apparatus, characterized in that, include: The acquisition module is used to acquire first information about the first data that has not been processed. The first information includes the reason why the data has not been processed, the first resource information that is currently available for scheduling, and the second resource information required to process the first data. The first determining module is configured to, when the reason for failure indicates that the submission of the first data has failed, determine the target partition corresponding to the reason for the submission failure based on the reason for the submission failure and the first resource information, and determine a first correction strategy for the second resource information based on the resource information of the target partition; The second determining module is used to determine a second correction strategy for the second resource information based on the first data and the first resource information when the reason for failure indicates that the first data was successfully submitted and is queued for processing. The display module is used to display the first correction strategy or the second correction strategy.

9. An electronic device, characterized in that, include: A processor, and a memory communicatively connected to the processor; The memory stores computer-executed instructions; The processor executes computer execution instructions stored in the memory to implement the method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, are used to implement the method as described in any one of claims 1 to 7.

11. A computer program product, characterized in that, Includes a computer program that, when executed by a processor, implements the method described in any one of claims 1-7.

Images