Batch program processing method and device, equipment and medium
By acquiring the running status and load status of batch programs and using a fault diagnosis model for automated updates, the problem of low efficiency in batch program updates is solved, and efficient and accurate determination and updating of program anomaly data is achieved.
Patent Information
- Application Number
- CN202511251006.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-09-03
- Publication Date
- 2025-11-21
AI Technical Summary
Existing batch processing programs require manual analysis and handling when malfunctions occur, resulting in low update efficiency.
By acquiring the running status and load status of batch programs, and using a trained fault diagnosis model for automated updates, multi-dimensional determination and improved accuracy of program anomaly data can be achieved.
It enables automated updates of batch programs, improving update efficiency and accuracy while reducing manual intervention time.
Smart Images
Figure CN120994223A_ABST
Abstract
Description
Technical Field
[0001] The embodiments of the present invention relate to the field of financial technology, and in particular to a batch processing method, apparatus, device and medium. Background Technology
[0002] Current business processes are typically handled through batch programs. When these programs malfunction, manual analysis and troubleshooting are usually required to fix the issues before the programs can resume operation. Therefore, improving the efficiency of batch program updates is crucial. Summary of the Invention
[0003] This invention provides a batch program processing method, apparatus, device, and medium to achieve automated updates of batch programs and improve the efficiency of batch program updates.
[0004] According to one aspect of the present invention, a batch processing method is provided, comprising:
[0005] Get the current batch running status of the current batch program for the current business;
[0006] If the current batch running status is a batch running success status, then obtain the business feedback data of the current business, and determine the current program running status of each current subroutine in the current batch program based on the business feedback data.
[0007] If there is an abnormal program running state in the current program running state, then obtain the current load state of each data node where the current subroutine is located;
[0008] The current program running status and current load status of each current subroutine are input into the trained fault diagnosis model to obtain the program anomaly data of the current batch program, and the current batch program is updated according to the program anomaly data.
[0009] According to another aspect of the present invention, a batch processing apparatus is provided, comprising:
[0010] The current batch execution status determination module is used to obtain the current batch execution status of the current batch program for the current business.
[0011] The current program running status determination module is used to obtain the business feedback data of the current business if the current batch running status is a batch running success status, and determine the current program running status of each current subroutine in the current batch program based on the business feedback data.
[0012] The current load status determination module is used to obtain the current load status of each data node where the current subroutine is located if there is an abnormal program running state in the current program running state.
[0013] The program update module is used to input the current program running status and current load status of each current subroutine into the trained fault diagnosis model, obtain the program abnormality data of the current batch program, and update the current batch program according to the program abnormality data.
[0014] According to another aspect of the present invention, an electronic device is provided, comprising:
[0015] One or more processors;
[0016] Memory, used to store one or more programs;
[0017] When one or more programs are executed by one or more processors, the one or more processors are able to execute any of the batch program processing methods provided in the embodiments of the present invention.
[0018] According to another aspect of the present invention, a computer-readable storage medium is provided, which stores computer instructions for causing a processor to execute any of the batch program processing methods provided in the embodiments of the present invention.
[0019] According to another aspect of the present invention, a computer program product is provided, including a computer program / instructions, which, when executed by a processor, implement any of the batch program processing methods provided in the embodiments of the present invention.
[0020] This invention provides a batch processing scheme. It obtains the current batch running status of the current batch program for the current business; if the current batch running status is successful, it obtains the business feedback data for the current business and determines the current program running status of each current subroutine in the current batch program based on the business feedback data; if there is an abnormal program running status in the current program running status, it obtains the current load status of the data node where each current subroutine is located; it inputs the current program running status and current load status of each current subroutine into a trained fault diagnosis model to obtain the program abnormal data of the current batch program, and updates the current batch program based on the program abnormal data. The above solution, when the current batch execution status is successful, determines the current execution status of each subroutine in the current batch program based on business feedback data. If an abnormal execution status exists, the current load status of the data node where each subroutine is located is obtained. The current execution status and load status of each subroutine are input into a trained fault diagnosis model to obtain the program's abnormal data. The batch program is then updated based on this abnormal data, achieving automated batch program updates and improving update efficiency. Furthermore, by inputting the current execution status and load status of each subroutine into the trained fault diagnosis model to obtain the program's abnormal data, this embodiment of the invention achieves multi-dimensional determination of program abnormal data, improving the accuracy of the determined abnormal data and consequently enhancing the accuracy of subsequent batch program updates based on this abnormal data.
[0021] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of the present invention, nor is it intended to limit the scope of the invention. Other features of the invention will become readily apparent from the following description. Attached Figure Description
[0022] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0023] Figure 1 This is a flowchart of a batch program processing method provided in Embodiment 1 of the present invention;
[0024] Figure 2 This is a flowchart of a batch program processing method provided in Embodiment 2 of the present invention;
[0025] Figure 3 This is a schematic diagram of the structure of a batch processing device provided in Embodiment 4 of the present invention;
[0026] Figure 4 This is a schematic diagram of the structure of an electronic device that implements a batch program processing method, as provided in Embodiment 5 of the present invention. Detailed Implementation
[0027] The present invention will now be described in further detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and not intended to limit it. Furthermore, it should be noted that, for ease of description, the accompanying drawings show only the parts relevant to the present invention, and not all of the structures.
[0028] Example 1
[0029] Figure 1 This is a flowchart of a batch program processing method provided in Embodiment 1 of the present invention. This embodiment is applicable to the case of automatically updating batch programs. The method can be executed by a batch program processing device, which can be implemented in software and / or hardware and can be configured in an electronic device that carries batch program processing functions.
[0030] See Figure 1 The batch processing method shown is applied to a batch processing system and includes:
[0031] S110. Obtain the current batch running status of the current batch program for the current business.
[0032] Here, "current business" refers to the business being processed at the current moment. "Current batch program" refers to the candidate batch program used to process the current business. "Candidate batch program" refers to the original batch program in the batch program processing system. The batch program processing system can be used to call and update candidate batch programs, etc.
[0033] The current batch execution status refers to the overall execution status of the current batch program. For example, the current batch execution status can be either a successful batch execution status or an interrupted batch execution status. A successful batch execution status indicates that the current batch program has executed successfully. An interrupted batch execution status indicates that the current batch program has been interrupted.
[0034] Specifically, the batch processing system monitors the overall running status of the current batch program in real time, and obtains the current batch running status of the current batch program for the current business when the current batch program stops running.
[0035] S120. If the current batch running status is a batch running success status, then obtain the business feedback data of the current business, and determine the current program running status of each current subroutine in the current batch program based on the business feedback data.
[0036] Business feedback data refers to data that provides feedback on the current business outcome. For example, business feedback data may include currently abnormal business outcomes from the current business results. The current business outcome refers to the processing result of the current business. Currently abnormal business outcomes refer to business outcomes that contain anomalies.
[0037] Here, "current subroutine" refers to the subroutine in the current batch program. "Current program running state" refers to the running state of the current subroutine. For example, the current program running state can be either an abnormal running state or a normal running state. An abnormal running state indicates that the current subroutine is running abnormally. A normal running state indicates that the current subroutine is running without abnormalities.
[0038] Specifically, when the current batch run status is "batch run successful", obtain the business feedback data of the current business; based on the business feedback data, determine the current program running status of each current subroutine in the current batch program.
[0039] S130. If there is an abnormal running state in the current program running state, then obtain the current load state of each data node where the current subroutine is located.
[0040] Among them, a data node can represent the server where the current subroutine resides. The current load status can represent the load status of the data node.
[0041] Specifically, if at least one program is in an abnormal running state in the current program running state, for any current subroutine, obtain the current load state of the data node where the current subroutine is located.
[0042] S140. Input the current program running status and current load status of each current subroutine into the trained fault diagnosis model to obtain the program abnormal data of the current batch program, and update the current batch program according to the program abnormal data.
[0043] The fault diagnosis model can be used to identify abnormal data in the current batch of programs. This embodiment of the invention does not specifically limit the network structure of the fault diagnosis model; it can be set by technicians based on experience or needs.
[0044] In this context, program exception data refers to the causes of program failures in the current batch of programs. For example, program exception data may include at least one of the following: program failure location, program failure category, and environmental failure data. Program failure location refers to the location in the current batch of programs where the failure occurred; that is, the program code experiencing the failure can be determined based on the program failure location. Program failure category refers to the type of program failure that occurred in the current batch of programs. Environmental failure data characterizes the program's operating environment in which the failure occurred; that is, the data nodes experiencing the anomaly can be identified based on the environmental failure data.
[0045] It should be noted that for any given subroutine, there is a corresponding current program running state and current load state.
[0046] Specifically, the current program running status and current load status corresponding to each current subroutine are input into the trained fault diagnosis model, and the program anomaly data of the current batch of programs are output; the current batch of programs are updated based on the program anomaly data.
[0047] For example, updating the current batch program may include updating the program code that has malfunctioned in the current batch program, and / or updating the data nodes that have malfunctioned in the current batch program.
[0048] In an optional embodiment, before updating the current batch program based on the program error data, the method further includes: backing up the current batch program to obtain a backup batch program; correspondingly, updating the current batch program based on the program error data includes: updating the current batch program with a backup batch program based on the program error data.
[0049] Among them, backup batch program refers to the batch program obtained by backing up the current batch program.
[0050] Specifically, the current batch program is backed up to the backup node to obtain the backup batch program; based on the program error data, the current batch program, which already has a backup batch program, is updated. The backup node can be identified as the server used to store the backup batch program.
[0051] Understandably, by backing up the current batch program to obtain a backup batch program, and then updating the current batch program with the backup batch program based on the program's abnormal data, the loss of the original data in the current batch program is avoided, and data corruption is prevented in the event that the current batch program update fails, thus providing the ability to roll back the batch program.
[0052] It should be noted that before updating the current batch program, the current batch program must be stopped to prevent the currently running batch subroutine from triggering a batch interruption when updating the current batch program.
[0053] This invention provides a batch processing scheme. It obtains the current batch running status of the current batch program for the current business; if the current batch running status is successful, it obtains the business feedback data for the current business and determines the current program running status of each current subroutine in the current batch program based on the business feedback data; if there is an abnormal program running status in the current program running status, it obtains the current load status of the data node where each current subroutine is located; it inputs the current program running status and current load status of each current subroutine into a trained fault diagnosis model to obtain the program abnormal data of the current batch program, and updates the current batch program based on the program abnormal data. The above solution, when the current batch execution status is successful, determines the current execution status of each subroutine in the current batch program based on business feedback data. If an abnormal execution status exists, the current load status of the data node where each subroutine is located is obtained. The current execution status and load status of each subroutine are input into a trained fault diagnosis model to obtain the program's abnormal data. The batch program is then updated based on this abnormal data, achieving automated batch program updates and improving update efficiency. Furthermore, by inputting the current execution status and load status of each subroutine into the trained fault diagnosis model to obtain the program's abnormal data, this embodiment of the invention achieves multi-dimensional determination of program abnormal data, improving the accuracy of the determined abnormal data and consequently enhancing the accuracy of subsequent batch program updates based on this abnormal data.
[0054] Based on the above technical solutions, this embodiment of the invention also provides an optional example. In one optional embodiment, the method further includes: if the current batch running state is a batch running interrupted state, then determining the current interruption category based on the current interruption alarm information of the current batch program; determining the corresponding current restart time threshold based on the current interruption category, and rescheduling the current batch program based on the current restart time threshold to obtain a scheduling result; and updating the current batch program based on the scheduling result.
[0055] The batch execution interruption status indicates that the overall execution of the current batch program has been interrupted. Current interruption alarm information can be used to indicate that the overall execution of the current batch program has been interrupted. For example, the current interruption alarm information may include the reason for the interruption. For instance, the current interruption alarm information can be used to determine the interruption type through sound.
[0056] The current interruption category refers to the reason for the interruption of the current batch program. For example, the current interruption category can be a program interruption category or an environment interruption category. A program interruption category indicates that the execution of the entire batch program is interrupted due to an exception in the current batch subroutine. An environment interruption category indicates that the execution of the entire batch program is interrupted due to an exception in the data node where the current batch subroutine resides. A current batch subroutine exception refers to an anomaly in the program code of the current batch subroutine, such as a code error. A data node exception refers to an anomaly in server performance. For example, a data node exception could be caused by excessive load on the data node, leading to data node crashes.
[0057] The current restart time threshold refers to the candidate restart time threshold corresponding to the current interruption category. The candidate restart time threshold is a pre-set time threshold that allows batch program restarts. This embodiment of the invention does not limit the size of the candidate restart time threshold; it can be set by technicians based on experience or needs, or determined through extensive experimentation. For example, the candidate restart time threshold corresponding to the program interruption category may differ from the candidate restart time threshold corresponding to the environment interruption category. For instance, the candidate restart time threshold corresponding to the program interruption category may be smaller than the candidate restart time threshold corresponding to the environment interruption category.
[0058] The scheduling result refers to the outcome of rescheduling and executing the current batch of programs. For example, the scheduling result can be either successful or unsuccessful. Successful scheduling indicates that the current batch of programs was successfully executed. Unsuccessful scheduling indicates that the current batch of programs failed to be executed.
[0059] Specifically, if the current batch running status is a batch running interrupted status, the current interruption alarm information of the current batch program is obtained, and the current interruption category is determined based on the current interruption alarm information of the current batch program; the current restart time threshold corresponding to the current interruption category is determined; within the current restart time threshold, the current batch program is rescheduled to obtain the scheduling result; and the current batch program is updated based on the scheduling result.
[0060] For example, if scheduling is successful, the overall running status of the current batch program will continue to be monitored, and the current batch running status will be obtained in real time; if scheduling fails, the reason for the scheduling failure will be determined, and the current batch program will be updated according to the reason for the scheduling failure.
[0061] Understandably, by determining the current interruption type and the corresponding current restart time threshold based on the current interruption alarm information, and then rescheduling the current batch of programs within the current restart time threshold to obtain the scheduling result, and updating the current batch of programs based on the scheduling result, the accuracy of the determined current restart time threshold is improved. When the current batch of programs experiences a running interruption, the restart of the current batch of programs is automatically realized, and restarts without time limits are avoided, thus improving the restart efficiency of the current batch of programs.
[0062] To improve the reliability of batch processing systems, this invention provides an embodiment that periodically monitors the batch processing system. In one optional embodiment, the method further includes: acquiring historical fault data of each candidate batch program in the batch processing system within a preset historical time period; clustering candidate subroutines in each candidate batch program based on the historical fault data to obtain candidate cluster groups; determining target cluster groups based on preset fault categories of the batch processing system; and determining the number of target subroutines in the target cluster groups; and determining the system state of the batch processing system based on the number of subroutines and a preset subroutines number threshold.
[0063] In this embodiment of the invention, the length of the preset historical period is not limited; it can be set by technicians based on experience or needs, or determined through extensive experimentation. Historical fault data refers to fault data of candidate batch programs within the preset historical period. Candidate subroutines refer to subroutines within the candidate batch programs. Candidate cluster groups refer to the groups obtained after clustering candidate subroutines. It should be noted that each candidate cluster group corresponds to a program fault category. This embodiment of the invention does not limit the clustering algorithm; it can be set by technicians based on experience or needs.
[0064] The preset fault category refers to a pre-set fault category that is related to the performance of the batch processing system. This embodiment of the invention does not impose any limitations on the setting of the preset fault category; it can be set by technical personnel based on experience or needs.
[0065] In this context, the target cluster group refers to a candidate cluster group that matches any preset fault category. The target subroutine refers to a candidate subroutine within the target cluster group. The number of programs refers to the total number of target subroutines. This embodiment of the invention does not impose any limitation on the size of the preset program number threshold; it can be set by technicians based on experience or needs, or determined repeatedly through numerous experiments.
[0066] The system state characterizes the performance status of the batch processing system. For example, the system state can be either a normal state or an abnormal state. A normal state indicates that the batch processing system's performance is normal. An abnormal state indicates that the batch processing system's performance is abnormal.
[0067] Specifically, the system acquires historical fault data for each candidate batch program within a preset historical time period in the batch program processing system, and clusters candidate subroutines in each candidate batch program based on the historical fault data to obtain candidate cluster groups, that is, candidate subroutines with the same program fault category are clustered into the same candidate cluster group; the candidate cluster group that matches the preset fault category is taken as the target cluster group; the number of target subroutines in the target cluster group is determined; and the system status of the batch program processing system is determined based on the number of programs and the preset program number threshold.
[0068] Understandably, by clustering candidate subroutines based on historical fault data within a preset historical period, candidate cluster groups are obtained. Target cluster groups are determined based on preset fault categories. The number of target subroutines in the target cluster groups is then determined. Finally, based on the number of subroutines and a preset threshold for the number of subroutines, the system status of the batch processing system is determined. This enables performance monitoring of the batch processing system and improves the accuracy of the determined system status.
[0069] For example, the system status of the batch program processing system is determined based on the number of programs and a preset program number threshold, including: if the number of programs is greater than or equal to the preset program number threshold, the system status of the batch program processing system is determined to be an abnormal system status; if the number of programs is less than the preset program number threshold, the system status of the batch program processing system is determined to be a normal system status.
[0070] Example 2
[0071] Figure 2 This is a flowchart of a batch program processing method provided in Embodiment 2 of the present invention. Based on the above embodiments, this embodiment further refines the operation of "determining the current program running state of each current subroutine in the current batch program based on business feedback data" into "determining the sub-business state of each current sub-business in the current business based on business feedback data; determining the current program running state of the corresponding current subroutine based on the sub-business state," thereby improving the mechanism for determining the current program running state. It should be noted that for parts not detailed in this embodiment, please refer to the descriptions in other embodiments.
[0072] See Figure 2 The batch processing method shown includes:
[0073] S210. Obtain the current batch running status of the current batch program for the current business.
[0074] S220. If the current batch operation status is a batch operation success status, then obtain the business feedback data of the current business, and determine the sub-business status of each current sub-business in the current business based on the business feedback data.
[0075] Here, "current sub-service" refers to a sub-service within the current business. "Sub-service status" refers to the current business state of the sub-service. For example, the sub-service status can be either a normal sub-service status or an abnormal sub-service status. A normal sub-service status indicates that the current sub-service has no abnormalities. An abnormal sub-service status indicates that the current sub-service has an abnormality.
[0076] For example, if the business feedback data only includes the current abnormal business result, then the sub-business status of the current sub-business corresponding to the current abnormal business result will be determined as the sub-business abnormal status, and the sub-business status of other current sub-businesses will be determined as the sub-business normal status.
[0077] S230. Based on the sub-business status, determine the current program running status of the current sub-program corresponding to the current sub-business.
[0078] In an optional embodiment, determining the current program running state of the current subroutine corresponding to the current sub-service based on the sub-service state includes: for any current sub-service, if the sub-service state of the current sub-service is an abnormal state, then determining the current program running state of the current subroutine corresponding to the current sub-service is an abnormal running state; if the sub-service state of the current sub-service is a normal state, then determining the current program running state of the current subroutine corresponding to the current sub-service is a normal running state.
[0079] Understandably, if the current sub-business is in an abnormal state, then the current subroutine corresponding to the current sub-business is determined to be in an abnormal running state; if the current sub-business is in a normal state, then the current subroutine corresponding to the current sub-business is determined to be in a normal running state, thus improving the accuracy of determining the current running state of the current subroutine.
[0080] S240. If there is an abnormal running state in the current program running state, then obtain the current load state of each data node where the current subroutine is located.
[0081] S250. Input the current program running status and current load status of each current subroutine into the trained fault diagnosis model to obtain the program abnormal data of the current batch program, and update the current batch program according to the program abnormal data.
[0082] This invention provides a batch processing scheme. By refining the process of determining the current program running status of each current subroutine in the current batch program based on business feedback data, it further refines this process to determine the sub-business status of each current sub-business within the current business based on business feedback data; and then, based on the sub-business status, determines the current program running status of the corresponding current subroutine, thus improving the mechanism for determining the current program running status. This scheme, by determining the sub-business status of each current sub-business within the current business based on business feedback data, and then determining the current program running status of the corresponding current subroutine based on the sub-business status, achieves the determination of the current program running status of the current subroutine based on business feedback data, thereby improving the accuracy of the determined current program running status.
[0083] Example 3
[0084] This invention provides an optional example based on the above embodiments. It should be noted that for parts not described in detail in this invention's embodiments, please refer to the descriptions in other embodiments.
[0085] In this embodiment of the invention, if the current business is a financial business, such as a resource interaction record generation business, business-related data associated with the financial business is obtained from the user attribute data stored in the database. Based on the business-related data and the current batch program, the current business result of the financial business is determined; business feedback data is obtained, and the current batch program is updated based on the business feedback data. The database can be used to store user attribute data of authorized users. User attribute data refers to the user's basic data. Business-related data refers to data associated with the current business.
[0086] For example, the batch processing method provided in this embodiment of the invention realizes real-time judgment and prediction of batch operation. Based on the overall operation status of the current batch program (i.e., the current batch operation status), it predicts whether the batch operation is normal. If a batch problem occurs, it immediately enters the batch operation diagnosis module to perform adaptive optimization analysis on the batch, provide solutions for different causes, and automatically generate and update code. For multiple batch problems occurring in a short period of time, cluster analysis is used to determine the system health level (i.e., determine the system status based on historical fault data), and developers and relevant departments are quickly notified by means of telephone, SMS, email, etc., to ensure that the problem can be resolved as soon as possible and prevent the problem from spreading.
[0087] An example of a self-correcting process for batch program execution monitoring technology is as follows: The batch execution status is monitored by tracking the batch execution and analyzing the execution logs in real time. If a batch execution is interrupted, it is first restarted based on a retry mechanism. Generally, the retry time threshold is fixed. However, for problems caused by the cluster environment (i.e., environment fault types), it is unclear whether the environment has returned to normal during retry, requiring manual judgment. This system method monitors the environment in real time and dynamically sets the retry time threshold (i.e., the candidate restart time threshold), realizing an adaptive batch interruption resolution system and method.
[0088] For example, the specific process for adaptive batch program updates is as follows: This process automatically deploys the updated batch programs to the specified runtime environment. First, a backup task is created to back up the original batch programs, supporting rollback of newly deployed programs. Next, a batch task is created to determine which batch program components will be installed subsequently. Then, a batch shutdown task is executed to stop the current batch jobs, preventing ongoing batch updates from triggering batch interruptions. Finally, a backup task is executed to back up the batch programs. Next, the batch programs are executed, deploying the determined batch program components to the server. Finally, a batch startup is performed to maintain normal batch operation.
[0089] The batch processing method provided in this invention enables batch prediction and self-optimization, improving the automation level of software development and maintenance; and it advances risk management nodes based on big data mining technology.
[0090] For example, in existing technologies, financial institutions' monthly resource interaction records (i.e., monthly bills) are developed in batches, and then generated at the beginning of each month based on historical data from the previous month. Each user's monthly resource interaction record displays fixed and identical metrics. When the monthly resource interaction records are pushed to users, data correction and updates are needed to address various application scenarios raised by users. Manual solutions are time-consuming and inefficient. The batch program processing system provided in this invention can automatically update batch programs based on user business feedback data. Subsequent monthly resource interaction records can be generated again based on the updated batch programs, improving efficiency. When generating monthly resource interaction records, the batch program processing system in this invention can analyze user business-related data across various dimensions and generate metrics for each user's monthly resource interaction record based on machine learning. Different users display different metrics, solving the problem of existing monthly resource interaction records having single metrics and fixed templates.
[0091] Continuing from the previous example, this embodiment of the invention also provides a business alarm system for the generation of monthly resource interaction records. In this scenario, the business alarm system is connected to a batch processing system to analyze the received business feedback data and determine the corresponding business alarm strategy. Specifically, this business alarm system can automatically cluster and analyze user feedback issues and generate work orders, which are then dispatched to the relevant departments for workflow processing. For issues in monthly resource interaction records that are reported in a short period of time, cluster analysis triggers system alerts, and different alert methods are adopted according to different situations, with technical adaptive logic adjustments. For suspected major issues or public opinion issues that trigger public opinion alerts, relevant departments can be notified to quickly activate relevant emergency plans, achieving rapid response to business feedback data, shortening work order processing time, improving the system capabilities of various departments, and significantly increasing the processing efficiency of monthly resource interaction record issues. For the generation of monthly resource interaction records, this embodiment of the invention also provides a product push system. In this scenario, the product push system is connected to a batch processing system to push financial products to corresponding users based on the received monthly resource interaction records.
[0092] It should be noted that the information collected in the embodiments of the present invention is information and data authorized by the user or fully authorized by all parties. The collection, storage, use, processing, transmission, provision, disclosure and application of the relevant data all comply with the relevant laws, regulations and standards of the relevant countries and regions, and necessary confidentiality measures have been taken. It does not violate public order and good morals, and corresponding operation entry points are provided for users to choose to authorize or refuse. If the user chooses to refuse, the process will proceed to the expert decision-making process.
[0093] Example 4
[0094] Figure 3 This is a schematic diagram of a batch program processing device provided in an embodiment of the present invention. This embodiment is applicable to the automatic updating of batch programs. The method can be executed by a batch program processing device, which can be implemented in software and / or hardware and can be configured in an electronic device that carries batch program processing functions.
[0095] like Figure 3 As shown, the device includes: a current batch operation status determination module 310, a current program operation status determination module 320, a current load status determination module 330, and a program update module 340.
[0096] The current batch running status determination module 310 is used to obtain the current batch running status of the current batch program of the current business;
[0097] The current program running status determination module 320 is used to obtain the business feedback data of the current business if the current batch running status is a batch running success status, and determine the current program running status of each current subroutine in the current batch program based on the business feedback data.
[0098] The current load status determination module 330 is used to obtain the current load status of each data node where the current subroutine is located if there is an abnormal program running state in the current program running state.
[0099] The program update module 340 is used to input the current program running status and current load status of each current subroutine into the trained fault diagnosis model, obtain the program abnormal data of the current batch program, and update the current batch program according to the program abnormal data.
[0100] This invention provides a batch processing scheme. It obtains the current batch running status of the current batch program for the current business; if the current batch running status is successful, it obtains the business feedback data for the current business and determines the current program running status of each current subroutine in the current batch program based on the business feedback data; if there is an abnormal program running status in the current program running status, it obtains the current load status of the data node where each current subroutine is located; it inputs the current program running status and current load status of each current subroutine into a trained fault diagnosis model to obtain the program abnormal data of the current batch program, and updates the current batch program based on the program abnormal data. The above solution, when the current batch execution status is successful, determines the current execution status of each subroutine in the current batch program based on business feedback data. If an abnormal execution status exists, the current load status of the data node where each subroutine is located is obtained. The current execution status and load status of each subroutine are input into a trained fault diagnosis model to obtain the program's abnormal data. The batch program is then updated based on this abnormal data, achieving automated batch program updates and improving update efficiency. Furthermore, by inputting the current execution status and load status of each subroutine into the trained fault diagnosis model to obtain the program's abnormal data, this embodiment of the invention achieves multi-dimensional determination of program abnormal data, improving the accuracy of the determined abnormal data and consequently enhancing the accuracy of subsequent batch program updates based on this abnormal data.
[0101] Optionally, the current program running status determination module 320 includes:
[0102] The sub-service status determination unit is used to determine the sub-service status of each current sub-service in the current service based on the service feedback data.
[0103] The current program running state determination unit is used to determine the current program running state of the current subroutine corresponding to the current current sub-service based on the sub-service state.
[0104] Optionally, the current program running state determination unit is specifically used for:
[0105] For any current sub-service, if the sub-service status of the current sub-service is a sub-service abnormal state, then the current program running state of the current subroutine corresponding to the current sub-service is determined to be a program abnormal running state.
[0106] If the sub-business status of the current sub-business is normal, then the current program running status of the current subroutine corresponding to the current sub-business is determined to be normal program running status.
[0107] Optionally, the device further includes:
[0108] The current interrupt category determination module is used to determine the current interrupt category based on the current interrupt alarm information of the current batch program if the current batch running state is a batch running interrupt state.
[0109] The scheduling result determination module is used to determine the corresponding current restart time threshold according to the current interrupt category, and to reschedule the current batch of programs according to the current restart time threshold to obtain the scheduling result;
[0110] The batch program update module is used to update the current batch program according to the scheduling result.
[0111] Optionally, the device further includes:
[0112] The candidate cluster group determination module is used to obtain historical fault data of each candidate batch program in the batch program processing system within a preset historical period, and to cluster the candidate subroutines in each candidate batch program according to the historical fault data to obtain candidate cluster groups.
[0113] The program quantity determination module is used to determine the target cluster group based on the preset fault categories of the batch program processing system, and to determine the program quantity of the target subroutine in the target cluster group;
[0114] The system status determination module is used to determine the system status of the batch program processing system based on the number of programs and a preset program number threshold.
[0115] Optionally, before updating the current batch program based on the program exception data, the device further includes:
[0116] A backup batch program determination module is used to back up the current batch program to obtain a backup batch program;
[0117] Correspondingly, program update module 340 includes:
[0118] The program update unit is used to update the current batch program that has backup batch programs based on the program anomaly data.
[0119] The batch processing apparatus provided in the embodiments of the present invention can execute the batch processing method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects for executing each batch processing method.
[0120] The technical solution of this invention involves the collection, storage, use, processing, transmission, provision and disclosure of current batch operation status, business feedback data, current load status and historical fault data, etc., which all comply with the provisions of relevant laws and regulations and do not violate public order and good morals.
[0121] Example 5
[0122] According to embodiments of the present invention, the present invention also provides an electronic device, a readable storage medium, and a computer program product.
[0123] Figure 4 This is a schematic diagram of the structure of an electronic device for implementing a batch processing method according to Embodiment 5 of the present invention. The electronic device 410 is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices (such as helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the invention described and / or claimed herein.
[0124] like Figure 4As shown, the electronic device 410 includes at least one processor 411 and a memory, such as a read-only memory (ROM) 412 or a random access memory (RAM) 413, communicatively connected to the at least one processor 411. The memory stores computer programs executable by the at least one processor. The processor 411 can perform various appropriate actions and processes based on the computer program stored in the ROM 412 or loaded from storage unit 418 into the RAM 413. The RAM 413 may also store various programs and data required for the operation of the electronic device 410. The processor 411, ROM 412, and RAM 413 are interconnected via a bus 414. An input / output (I / O) interface 415 is also connected to the bus 414.
[0125] Multiple components in electronic device 410 are connected to I / O interface 415, including: input unit 416, such as keyboard, mouse, etc.; output unit 417, such as various types of displays, speakers, etc.; storage unit 418, such as disk, optical disk, etc.; and communication unit 419, such as network card, modem, wireless transceiver, etc. Communication unit 419 allows electronic device 410 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.
[0126] Processor 411 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 411 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. Processor 411 performs the various methods and processes described above, such as batch processing methods.
[0127] In some embodiments, the batch processing method may be implemented as a computer program tangibly contained in a computer-readable storage medium, such as storage unit 418. In some embodiments, part or all of the computer program may be loaded and / or mounted on electronic device 410 via ROM 412 and / or communication unit 419. When the computer program is loaded into RAM 413 and executed by processor 411, one or more steps of the batch processing method described above may be performed. Alternatively, in other embodiments, processor 411 may be configured to execute the batch processing method by any other suitable means (e.g., by means of firmware).
[0128] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.
[0129] Computer programs used to implement the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the processor, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be performed. The computer programs may be executed entirely on a machine, partially on a machine, or as a standalone software package, partially on a machine and partially on a remote machine, or entirely on a remote machine or server.
[0130] In the context of this invention, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination thereof. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.
[0131] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).
[0132] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or computing systems that include middleware components (e.g., application servers), or computing systems that include frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.
[0133] A computing system can include clients and servers. Clients and servers are generally geographically separated and typically interact via communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a host product within the cloud computing service system to address the shortcomings of traditional physical hosts and virtual private servers, such as high management difficulty and weak business scalability.
[0134] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and this is not limited herein.
[0135] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.
Claims
1. A batch processing method, characterized in that, include: Get the current batch running status of the current batch program for the current business; If the current batch running status is a batch running success status, then obtain the business feedback data of the current business, and determine the current program running status of each current subroutine in the current batch program based on the business feedback data. If there is an abnormal program running state in the current program running state, then obtain the current load state of each data node where the current subroutine is located; The current program running status and current load status of each current subroutine are input into the trained fault diagnosis model to obtain the program anomaly data of the current batch program, and the current batch program is updated according to the program anomaly data.
2. The method according to claim 1, characterized in that, Determining the current program running status of each current subroutine in the current batch program based on the business feedback data includes: Based on the business feedback data, determine the sub-business status of each current sub-business in the current business; Based on the sub-service status, determine the current program running status of the current subroutine corresponding to the current sub-service.
3. The method according to claim 2, characterized in that, Determining the current program running state of the current subroutine corresponding to the current sub-service based on the sub-service state includes: For any current sub-service, if the sub-service status of the current sub-service is a sub-service abnormal state, then the current program running state of the current subroutine corresponding to the current sub-service is determined to be a program abnormal running state. If the sub-business status of the current sub-business is normal, then the current program running status of the current subroutine corresponding to the current sub-business is determined to be normal program running status.
4. The method according to claim 1, characterized in that, The method further includes: If the current batch running status is a batch running interrupted status, then the current interruption category is determined based on the current interruption alarm information of the current batch program; Based on the current interruption category, determine the corresponding current restart time threshold, and based on the current restart time threshold, reschedule the current batch of programs to obtain the scheduling result; The current batch program is updated based on the scheduling result.
5. The method according to claim 1, characterized in that, The method further includes: Historical fault data of each candidate batch program in the batch program processing system within a preset historical period are obtained, and candidate subroutines in each candidate batch program are clustered according to the historical fault data to obtain candidate cluster groups. Based on the preset fault categories of the batch processing system, a target cluster group is determined, and the number of target subroutines in the target cluster group is determined. The system state of the batch program processing system is determined based on the number of programs and the preset program number threshold.
6. The method according to any one of claims 1-5, characterized in that, Before updating the current batch program based on the program exception data, the method further includes: The current batch program is backed up to obtain a backup batch program; Accordingly, updating the current batch program based on the program exception data includes: Based on the program error data, update the current batch program that has backup batch programs.
7. A batch processing device, characterized in that, include: The current batch execution status determination module is used to obtain the current batch execution status of the current batch program for the current business. The current program running status determination module is used to obtain the business feedback data of the current business if the current batch running status is a batch running success status, and determine the current program running status of each current subroutine in the current batch program based on the business feedback data. The current load status determination module is used to obtain the current load status of each data node where the current subroutine is located if there is an abnormal program running state in the current program running state. The program update module is used to input the current program running status and current load status of each current subroutine into the trained fault diagnosis model, obtain the program abnormality data of the current batch program, and update the current batch program according to the program abnormality data.
8. An electronic device, characterized in that, include: One or more processors; Memory, used to store one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement a batch program processing method as described in any one of claims 1-6.
9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by the processor, the program implements a batch program processing method as described in any one of claims 1-6.
10. A computer program product comprising a computer program / instructions that, when executed by a processor, implement the steps of the batch program processing method according to any one of claims 1-6.