Parallel processing method and system for batch write-off tasks

Abstract

The present application relates to the technical field of parallel processing, and provides a parallel processing method and system for batch write-off tasks. The method comprises: extracting a plurality of pieces of query data of a preset write-off window, comprising external write-off codes and write-off services; performing file import on the basis of the data, and generating transaction files with unique identifiers; when file name identifiers are verified, parsing the external write-off codes, and generating fund files; generating pending write-off files by means of procurement-settlement mapping authentication; identifying request time identifiers to identify write-off conflicts, and generating conflicting file clusters; updating the configuration of a parallel processing module on the basis of the conflict condition; and utilizing the updated module to process the conflicting file clusters in parallel. The present application solves the technical problems of data conflict, irrational resource allocation and low processing efficiency caused by improper parallel processing in batch write-off tasks, and realizes real-time extraction, verification, parsing and authentication of write-off data, thereby ensuring the data accuracy and consistency.

Description

A Parallel Processing Method and System for Batch Reimbursement Tasks Technical Field

[0001] This application relates to the field of high-performance computing technology, specifically to the field of parallel processing technology, and in particular to a parallel processing method and system for batch verification tasks. Background Technology

[0002] With the rapid development of the automotive consumer service sector and the continuous expansion of online and offline business scale, the efficiency and accuracy of batch verification tasks have become key elements of enterprise operations. In this scenario, traditional batch verification methods have gradually revealed their limitations. Manual calculations are not only inefficient and consume significant human resources, but are also prone to errors due to human factors. Meanwhile, computer systems primarily based on serial processing cannot fully utilize the data processing capabilities of the computer system, resulting in low processing efficiency. Furthermore, improper data processing can easily lead to technical problems such as data conflicts and unreasonable resource allocation, further affecting the accuracy and consistency of verification data. Technical issues

[0003] The application provides a parallel processing method and system for batch reconciliation tasks, aiming to solve the technical problems of data conflicts, unreasonable resource allocation, and low processing efficiency caused by improper parallel processing in batch reconciliation tasks. Technical solutions

[0004] In view of the above problems, this application provides a parallel processing method and system for batch reimbursement tasks.

[0005] The first aspect disclosed in this application provides a parallel processing method for batch reconciliation tasks. The method includes: extracting multiple query data from a preset reconciliation window, wherein the multiple query data includes multiple external reconciliation codes and multiple reconciliation services; importing files based on the multiple external reconciliation codes and the multiple reconciliation services to generate multiple transaction files, wherein the multiple transaction files include multiple filename identifiers, and the filename identifiers are unique; performing template validation on the multiple filename identifiers; if the validation passes, parsing the fields of the multiple external reconciliation codes to generate multiple fund files; and based on the multiple transaction files... The system performs procurement-settlement mapping authentication with the multiple funding documents. If the authentication is successful, multiple documents to be reconciled are generated, each document having multiple request time identifiers. Conflict identification is performed on the multiple documents to be reconciled based on the multiple request time identifiers, generating multiple conflicting document clusters, each cluster having multiple conflict times. Resource configuration is updated for the parallel processing module based on the multiple conflicting document clusters and the multiple conflict times, resulting in an updated parallel processing module. The updated parallel processing module is then used to perform parallel processing on the multiple conflicting document clusters.

[0006] Another aspect of this application discloses a parallel processing system for batch reconciliation tasks. The system includes: a data extraction component for extracting multiple query data from a preset reconciliation window, wherein the multiple query data includes multiple external reconciliation codes and multiple reconciliation services; a file import component for importing files based on the multiple external reconciliation codes and the multiple reconciliation services to generate multiple transaction files, wherein the multiple transaction files include multiple filename identifiers, and the filename identifiers are unique; a field parsing component for performing template validation on the multiple filename identifiers; if the validation passes, performing field parsing on the multiple external reconciliation codes to generate multiple fund files; and a mapping authentication component for... The system performs procurement-settlement mapping authentication on the multiple transaction files and multiple fund files. If the authentication is successful, multiple files to be reconciled are generated, each file having multiple request time identifiers. A conflict identification component is used to identify reconciliation conflicts in the multiple files to be reconciled based on the multiple request time identifiers, generating multiple conflicting file clusters, each cluster having multiple conflict times. A configuration update component is used to update the resource configuration of the parallel processing module based on the multiple conflicting file clusters and the multiple conflict times, resulting in an updated parallel processing module. A parallel processing module is used to perform parallel processing on the multiple conflicting file clusters using the updated parallel processing module.

[0007] One or more technical solutions provided in this application have at least the following technical effects or advantages:

[0008] The aforementioned parallel processing method for batch reconciliation tasks extracts multiple query data from a preset reconciliation window, including external reconciliation codes and reconciliation services. This data is then used to import files, generating multiple transaction files each containing a unique filename identifier. These filename identifiers are then validated using a template to ensure correct formatting. Once validation passes, the external reconciliation codes are further parsed to generate a funds file. Next, the transaction and funds files are combined for procurement and settlement mapping authentication to ensure data accuracy and consistency, generating multiple files to be reconciled, each with a corresponding request time identifier. Reconciliation conflicts are then identified based on the obtained request time identifiers. Conflicting files are grouped into conflict-pending-reconciliation file clusters, and the conflict time for each cluster is recorded. To handle these conflicting files more effectively, the resource configuration of the parallel processing module is updated based on the conflict file clusters and conflict times. This results in a more efficient parallel processing module. Finally, the updated parallel processing module is used to process the conflict-pending-reconciliation file clusters in parallel, thereby improving the processing efficiency and accuracy of the reconciliation task.

[0009] The above description is merely an overview of the technical solution of this application. In order to better understand the technical means of this application and to implement it in accordance with the contents of the specification, and to make the above and other objects, features and advantages of this application more obvious and understandable, specific embodiments of this application are given below. Beneficial effects

[0010] One or more technical solutions provided in this application have at least the following technical effects or advantages:

[0011] The aforementioned parallel processing method for batch reconciliation tasks extracts multiple query data from a preset reconciliation window, including external reconciliation codes and reconciliation services. This data is then used to import files, generating multiple transaction files each containing a unique filename identifier. These filename identifiers are then validated using a template to ensure correct formatting. Once validation passes, the external reconciliation codes are further parsed to generate a funds file. Next, the transaction and funds files are combined for procurement and settlement mapping authentication to ensure data accuracy and consistency, generating multiple files to be reconciled, each with a corresponding request time identifier. Reconciliation conflicts are then identified based on the obtained request time identifiers. Conflicting files are grouped into conflict-pending-reconciliation file clusters, and the conflict time for each cluster is recorded. To handle these conflicting files more effectively, the resource configuration of the parallel processing module is updated based on the conflict file clusters and conflict times. This results in a more efficient parallel processing module. Finally, the updated parallel processing module is used to process the conflict-pending-reconciliation file clusters in parallel, thereby improving the processing efficiency and accuracy of the reconciliation task. Attached Figure Description

[0012] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0013] Figure 1 is a flowchart illustrating a parallel processing method for batch reimbursement tasks in one embodiment.

[0014] Figure 2 is a system architecture diagram of a batch reconciliation task parallel processing system in one embodiment.

[0015] Figure labeling: Data extraction component 1, file import component 2, field parsing component 3, mapping authentication component 4, conflict identification component 5, configuration update component 6, parallel processing component 7. Embodiments of the present invention

[0016] This application provides a parallel processing method and system for batch reconciliation tasks, which solves the technical problems of data conflicts, unreasonable resource allocation, and low processing efficiency caused by improper parallel processing in batch reconciliation tasks.

[0017] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of this application without creative effort are within the scope of protection of this application.

[0018] It should be noted that the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or server that includes a series of steps or units is not necessarily limited to those steps or units that are explicitly listed, but may include other steps or modules that are not explicitly listed or that are inherent to such process, method, product, or device. Example

[0019] As shown in Figure 1, this application provides a parallel processing method for batch reconciliation tasks, the method comprising:

[0020] Extract multiple query data from a preset verification window, wherein the multiple query data includes multiple external verification codes and multiple verification services;

[0021] As business scales up, the number of batch reconciliation tasks increases dramatically. Traditional serial processing methods cannot handle the demands of large-scale tasks, resulting in low processing efficiency and failing to meet the requirements of efficient enterprise operations. Parallel processing methods, on the other hand, can process multiple tasks simultaneously, significantly improving processing speed and helping enterprises quickly complete reconciliation work.

[0022] In this embodiment, the system terminal retrieves multiple query data items related to reconciliation from a preset reconciliation window. These data items include external reconciliation codes and reconciliation services. External reconciliation codes are crucial information used to identify and track reconciliation operations, while reconciliation services refer to the specific reconciliation business content. By extracting this query data, necessary information and basis can be provided for subsequent reconciliation processing.

[0023] Based on the multiple external verification codes and the multiple verification services, files are imported to generate multiple transaction files, wherein the multiple transaction files include multiple file name identifiers, and the file name identifiers are unique;

[0024] In one embodiment, based on multiple external verification codes and verification services, the system terminal imports relevant files to generate a series of transaction files. These transaction files each contain a unique filename identifier, ensuring that each file can be accurately identified and distinguished. For example, when the verification service is a maintenance cancellation, a unique identifier can be created using the format of store name, product name, settlement price, and timestamp. This not only facilitates subsequent data processing and management but also improves the efficiency and accuracy of the verification process.

[0025] Template validation is performed on the multiple file name identifiers. If the validation passes, the fields of the multiple external verification codes are parsed to generate multiple fund files.

[0026] In one embodiment, the system terminal performs template validation on the filename identifiers of the generated multiple transaction files. This is primarily to ensure that the filenames conform to preset specifications and are all valid transaction files. Only after the validation based on the preset template indicator matrix passes will the system terminal further process these transaction files. The processing method involves parsing the external write-off codes, extracting key information, and generating multiple funds files based on this information. The funds files record the fund flows related to the write-off and are crucial for subsequent write-off processing.

[0027] Furthermore, this application provides template validation, including:

[0028] Multiple preset template indicator matrices are extracted based on the multiple file name identifiers, wherein the multiple preset template indicator matrices include store name, product name and settlement price;

[0029] Using the multiple preset template indicator matrices as indexes, indicators are extracted from the multiple transaction files to generate multiple transaction indicator matrices;

[0030] Template verification is performed based on multiple preset template indicator matrices and the multiple transaction indicator matrices. If the store name, product name, and settlement price are all successfully verified, the verification is successful.

[0031] Preferably, the system terminal first defines the structure and key fields of the preset template indicator matrix, such as store name, product name, and settlement price. These fields are predefined according to the needs and rules of the enterprise's reconciliation business. Then, the system iterates through all generated filename identifiers, comparing them one by one to find the preset template indicator matrix corresponding to each filename identifier. Next, for each matched preset template indicator matrix, the system terminal extracts these successfully matched preset template indicator matrices. This results in multiple preset template indicator matrices. These preset template indicator matrices include store name, product name, and settlement price. Subsequently, the system terminal uses these preset template indicator matrices as indexes to extract relevant indicators from the transaction files one by one, and forms a transaction indicator matrix based on the structure of the preset template indicator matrices. Finally, the system terminal compares and verifies the preset template indicator matrices with the transaction indicator matrices to ensure that the length, format, and other information such as store name, product name, and settlement price match correctly. Only when all this key information is successfully verified does the system terminal consider the verification successful. This step is to ensure the accuracy and consistency of the transaction files, providing reliable data support for subsequent reconciliation processing.

[0032] Based on the multiple transaction files and the multiple fund files, a procurement-settlement mapping authentication is performed. If the authentication is successful, multiple files to be reconciled are generated, wherein the multiple files to be reconciled have multiple request time identifiers.

[0033] In one embodiment, the system terminal performs procurement-settlement mapping authentication based on multiple transaction and fund files. This authentication process primarily ensures the data consistency and accuracy between the transaction and fund files. Authentication is only considered successful if the transaction and fund data match perfectly in both the procurement and settlement stages and pass size checks. Once authentication is successful, the system terminal generates multiple pending verification files based on these files. These pending verification files not only contain all the key information about previous transactions and funds but are also assigned one or more request time identifiers, which identify the file's generation time or the chronological order of request processing. In summary, this process involves matching and verifying transaction and fund data, and generating pending verification files accordingly for subsequent verification processing.

[0034] Furthermore, this application provides a method for procurement-settlement mapping authentication based on the multiple transaction files and the multiple fund files, which further includes:

[0035] Perform a one-to-one mapping between the multiple transaction files and the multiple fund files to generate multiple transaction file-fund file mapping relationships;

[0036] Based on the multiple transaction file-fund file mapping relationships, the purchase amount and settlement amount are extracted to generate multiple purchase-settlement mapping relationships;

[0037] Preferably, the system terminal iterates through each transaction file and fund file, searching for key fields between them. These fields include, for example, order number, product name, and timestamp, used to determine the correspondence between files. By comparing these fields, the system terminal can establish a one-to-one mapping between transaction files and fund files. Once a matching field is found, the system terminal generates a mapping record, associating the transaction file identifier with the corresponding fund file identifier. During the mapping process, if a transaction file or fund file is found to have no matching identifier or field, or if multiple matches occur, the system terminal will handle the exception accordingly, including recording errors, skipping unmatched files, and prompting the user to perform manual matching. This process is repeated until all files are matched, generating multiple transaction file-fund file mapping relationships. These mapping relationships ensure an accurate correspondence between transaction data and fund data. Subsequently, the system terminal iterates through each mapping record. For each record, it obtains the corresponding transaction file and fund file identifiers. Then, it parses the obtained transaction file and fund file identifiers, extracting multiple purchase amounts and settlement amounts, and using these to form multiple purchase-settlement mapping relationships. These mapping relationships not only reflect the flow of funds between procurement and settlement, but also provide the necessary data foundation for subsequent reconciliation work. In summary, this process involves establishing a one-to-one correspondence between transaction documents and financial documents, and extracting the corresponding procurement and settlement amounts for subsequent reconciliation processing.

[0038] Determine whether the purchase amount is less than or equal to the settlement amount in the multiple purchase-settlement mapping relationships. If so, the authentication is successful, and the multiple purchase-settlement mapping relationships are treated as multiple pending verification documents.

[0039] If not, the authentication will fail.

[0040] Preferably, the system terminal checks each of the generated purchase-settlement mapping relationships one by one. During this process, the purchase amount and settlement amount in each mapping relationship are compared to ensure that the purchase amount does not exceed the settlement amount. If the purchase amount for all mapping relationships meets this condition (purchase amount less than or equal to settlement amount), the system terminal determines that the authentication is successful and treats these mapping relationships as pending reconciliation documents. These pending reconciliation documents will be used in subsequent reconciliation processes. However, if it is found that the purchase amount in any mapping relationship is greater than the settlement amount, the authentication fails because there are data errors or anomalies in these mapping relationships, requiring further verification and adjustment. In summary, this process verifies the consistency between the purchase amount and the settlement amount, ensuring that they meet the basic requirements for reconciliation.

[0041] Based on the multiple request time identifiers, the multiple files to be rewritten are identified for rewriting conflicts, and multiple conflicting file clusters to be rewritten are generated, wherein the multiple conflicting file clusters to be rewritten have multiple conflict times;

[0042] In one embodiment, the system terminal identifies reconciliation conflicts based on the request time identifiers of the files to be reconciled. This is primarily to determine which files may have overlapping or conflicting times during the reconciliation process. If two or more files to be reconciled have overlapping request time identifiers, they are grouped into a conflicting file cluster and assigned a conflict time identifier. This process is repeated multiple times until there are no conflicting times, and the remaining request time identifiers are assigned to non-conflicting internal nodes. The conflict time refers to the point in time between two request time identifiers, representing the time period during which these files may conflict during the reconciliation process. Through this process, the system terminal can clearly identify which files to be reconciled require special attention and processing to avoid conflicts or errors during the reconciliation process.

[0043] Furthermore, this application provides a method for identifying rescission conflicts of the plurality of files to be rescinded based on the plurality of request time identifiers, and generating a plurality of conflicting file clusters to be rescinded. The method further includes:

[0044] A first conflict identification unit is constructed by randomly extracting one request time identifier from the plurality of request time identifiers. The first conflict identification unit is used to identify the request time identifiers that meet the conflict window among the plurality of request time identifiers and assign them to the internal nodes of the first conflict identification unit. The remaining plurality of request time identifiers are assigned to the first set to be assigned.

[0045] Another request time identifier is randomly extracted from the first set to be divided to construct a second conflict identification unit. The second conflict identification unit is used to identify the request time identifiers in the first set to be divided that meet the conflict window, and assign them to the internal nodes of the second conflict identification unit. The remaining multiple request time identifiers are then assigned to the second set to be divided.

[0046] After multiple divisions, a request time identifier is randomly extracted from the (N-1)th set to be divided to construct the Nth conflict identification unit. The Nth conflict identification unit is used to identify the request time identifiers in the (N-1)th set to be divided that meet the conflict window, and assign them to the Nth conflict identification internal node. The remaining multiple request time identifiers are assigned to non-conflicting internal nodes.

[0047] Preferably, the system terminal randomly selects a request time identifier from all request time identifiers and constructs a first conflict identification unit based on the extracted request time identifier. The core of this unit is a pre-defined conflict judgment logic, which defines how to determine whether other request time identifiers conflict with the unit's base request time identifier. The conflict judgment logic selects a time identifier and compares it with the remaining time identifiers to determine if a conflict exists. The first conflict identification unit contains this conflict judgment logic and information about the base request time identifier. Subsequently, the remaining request time identifiers are iterated over. In each iteration, the system terminal uses the first conflict identification unit to check the current request time identifier to determine if it meets the time overlap requirement. Request time identifiers that meet the condition are assigned to the internal nodes of the first conflict identification unit, while those that do not are assigned to the first set to be partitioned. Afterward, the system terminal randomly selects another request time identifier from this set to be partitioned to construct a second conflict identification unit. Similarly, this unit checks whether other request time identifiers in the set to be partitioned meet the conflict window condition; those that meet the condition are assigned to the internal nodes of the second conflict identification unit, and the rest enter the second set to be partitioned. This process is repeated multiple times. Each time, a request timestamp is randomly selected from the previous set of files to be partitioned to construct a new conflict identification unit, until the last set of files to be partitioned is processed. The last conflict identification unit assigns the request timestamps that meet the criteria to the corresponding conflict identification internal nodes. The remaining request timestamps that are not assigned to any conflict identification internal node are considered conflict-free and are assigned to non-conflicting internal nodes. In this way, the system terminal can efficiently identify all files with time conflicts and organize them into different conflict-related file clusters, facilitating subsequent processing and analysis.

[0048] A reconciliation conflict identification module is constructed based on the first conflict identification unit, the second conflict identification unit, and the Nth conflict identification unit. The reconciliation conflict identification module is used to identify the multiple files to be reconciled, and to obtain the first conflict identification internal node, the second conflict identification internal node, the Nth conflict identification internal node, and the non-conflicting internal node.

[0049] Preferably, the system terminal integrates the functions and logic of the constructed first conflict identification unit, second conflict identification unit, and so on up to the Nth conflict identification unit, to form a unified reconciliation conflict identification module. This module integrates the core algorithms and judgment rules of all conflict identification units, enabling efficient and accurate conflict identification of documents to be reconciled. When using this module for identification, it can automatically classify the documents to be reconciled into corresponding conflict identification internal nodes or non-conflict internal nodes based on the judgment results of each conflict identification unit. Ultimately, the system terminal will obtain the first conflict identification internal node, the second conflict identification internal node, up to the Nth conflict identification internal node, and those non-conflict internal nodes that have not been classified into any conflict node. This process not only improves the efficiency and accuracy of conflict identification but also provides a clear classification and basis for subsequent processing.

[0050] Furthermore, this application provides multiple conflict-pending document clusters, and the method also includes:

[0051] The multiple request time identifiers stored in the first conflict identification internal node, the second conflict identification internal node, and the Nth conflict identification internal node are respectively regarded as N conflict-pending file clusters;

[0052] The M request time identifiers in the non-conflicting internal nodes are respectively regarded as M conflicting file clusters to be revoked;

[0053] The N conflicting file clusters and the M conflicting file clusters are referred to as the plurality of conflicting file clusters.

[0054] Optionally, the system terminal extracts N request time identifiers from each conflict identification internal node and uses these request time identifiers as N conflict-to-be-verified file clusters. These conflict-to-be-verified file clusters contain files that overlap or conflict in time. Subsequently, the system terminal extracts M conflict-to-be-verified file clusters in the same way. Then, these conflict-to-be-verified file clusters obtained from conflict identification internal nodes and non-conflicting internal nodes are integrated to form a set of multiple conflict-to-be-verified file clusters. This set provides a clear data foundation for subsequent processing and analysis.

[0055] Based on the multiple conflict-pending file clusters and the multiple conflict times, the resource configuration of the parallel processing module is updated to obtain the updated parallel processing module.

[0056] In one embodiment, the system terminal updates the resource configuration of the parallel processing module based on multiple conflicting file clusters and multiple conflict times. This process involves identifying the resource consumption of the conflicting file clusters through the resource configuration network layer and reallocating processing resources based on the identification results. For example, this could involve increasing or decreasing the number of processing units or adjusting their priorities, thereby dynamically adjusting the processing capacity of the parallel processing module. This ensures that the parallel processing module can process these conflicting files more efficiently, avoiding resource shortages or waste. After the resource configuration update, the system terminal receives an updated parallel processing module with improved capabilities for handling conflicting files, thus improving the efficiency and accuracy of the entire verification process.

[0057] Furthermore, this application provides a method for updating the resource configuration of a parallel processing module based on the multiple conflict-pending file clusters and the multiple conflict times, thereby obtaining an updated parallel processing module. The method further includes:

[0058] Based on the multiple conflict times, the multiple conflict-to-be-verified file clusters are serialized in ascending order to generate a sequence of conflict-to-be-verified file clusters.

[0059] The information entropy is calculated by traversing the sequence of conflicting files to be reconciled, and an information entropy data sequence is generated.

[0060] Preferably, the system terminal sorts multiple conflict-pending file clusters according to the chronological order of their conflict times, generating a sequence of conflict-pending file clusters. This sequence ensures that the file clusters are arranged according to their conflict times, which helps in subsequent analysis and processing of the conflict situation. Subsequently, the system terminal creates an empty information entropy data sequence to store the information entropy value of each conflict-pending file cluster. Then, it iterates through the sequence of conflict-pending file clusters sorted by conflict time. For each conflict-pending file cluster in the sequence, information entropy calculation is performed. For the currently traversed conflict-pending file cluster, the system terminal obtains all request time identifiers and related data contained within it. Based on this data, the information entropy of the conflict-pending file cluster is calculated. The information entropy calculation is based on the probability distribution of the data. The system terminal counts the frequency of different request time identifiers in the file cluster and calculates the information entropy based on these frequencies. After calculating the information entropy value of the current conflict-pending file cluster, the system knows to store it in the previously initialized information entropy data sequence. After calculating the information entropy of the current conflicting file cluster to be resolved, the system terminal continues to traverse the next file cluster in the sequence and repeats the above steps until the entire sequence has been traversed. Once all conflicting file clusters to be resolved have been traversed, the system terminal obtains a complete information entropy data sequence, containing the information entropy value for each file cluster. This sequence provides crucial data support for subsequent analysis and decision-making, helping the system terminal better understand the conflict situation and take appropriate action.

[0061] The information entropy data in the information entropy data sequence is sequentially input into the resource configuration network layer for resource consumption identification, thereby generating a resource consumption data sequence.

[0062] Based on the resource consumption data sequence, the resource configuration of the parallel processing module is updated to generate the updated parallel processing module.

[0063] Preferably, the system terminal sequentially inputs each information entropy data point from the information entropy data sequence into the resource allocation network layer. This network layer is pre-trained and is a model specifically designed to analyze data and predict resource consumption. It can assess the amount of resources required to process these conflicting file clusters based on the input information entropy data. The resource allocation network layer processes each information entropy data point and generates corresponding resource consumption data. This resource consumption data reflects the computational resources, storage resources, and other types of resources required to process the corresponding conflicting file clusters. Subsequently, the system terminal updates the resource allocation of the parallel processing module based on this resource consumption data. This means that the system terminal dynamically adjusts the resource allocation in the parallel processing module according to the resource consumption of each conflicting file cluster. For example, for file clusters with high resource consumption, the system terminal increases the number of processing units or improves the performance of the processing units; while for file clusters with low resource consumption, it reduces the resource allocation to improve resource utilization efficiency. After such a resource allocation update, the system terminal obtains an updated parallel processing module. This module has a more reasonable resource configuration, which can better adapt to the processing needs of different conflicting file clusters, thereby improving the efficiency and performance of the entire processing flow.

[0064] Furthermore, this application provides a method for configuring training resources for network layers, which also includes:

[0065] Acquire multiple sample information entropy data and multiple sample resource consumption data as training datasets;

[0066] The training dataset is divided equally to obtain n sets of training datasets;

[0067] The framework built on the feedforward neural network is trained in a supervised manner using the n sets of training datasets in sequence, and the model network parameters are updated according to the previous set of training output results until the output results meet the preset output accuracy, thereby generating the resource configuration network layer that has been trained.

[0068] Optionally, the system terminal retrieves multiple sample entropy data and corresponding sample resource consumption data from a historical database to form a training dataset. This entropy data reflects the complexity and uncertainty of the data, while the resource consumption data represents the amount of resources required to process it. The training dataset is then evenly divided into n training datasets. This is done to train the resource allocation network layer multiple times using different data subsets, improving its generalization ability and stability. Next, a feedforward neural network-based framework is initialized. This framework includes an input layer, hidden layers, and an output layer, as well as connection weights and bias terms between layers. The number of hidden layers and the number of neurons in each layer are set based on the complexity of the problem and the characteristics of the data. Then, the system terminal loads the first set of n training datasets, which includes the entropy data and corresponding resource consumption data of multiple samples. The entropy data from the first training dataset is then used as input and propagated forward through the feedforward neural network. The data passes from the input layer through the hidden layers and finally reaches the output layer, generating predicted resource consumption data. Furthermore, the system terminal compares the predicted resource consumption data with the actual resource consumption data, calculates the value of the loss function, and uses this value to measure the difference between the predicted and actual values. Then, based on the value of the loss function, the system terminal uses the backpropagation algorithm to calculate the gradient, i.e., the derivative of the loss function with respect to the network parameters. The calculated gradient is then used to update the network parameters of the feedforward neural network, including weights and bias terms. This process aims to reduce the value of the loss function, thereby gradually optimizing the prediction performance of the feedforward neural network. After completing the training of the first set of data, the next set of training datasets is loaded, and the above process is repeated for a new round of training and parameter updates. Afterward, the system terminal sequentially loads the remaining n-2 sets of training datasets, repeating the process of forward propagation, loss calculation, backpropagation, and parameter updates. In each iteration, the feedforward neural network adjusts its network parameters based on the training results of the previous set of data to gradually improve prediction accuracy. After each iteration, the system terminal evaluates the output accuracy of the feedforward neural network on the current training dataset. If the output accuracy reaches or exceeds a preset accuracy threshold, the training process ends; otherwise, the next iteration continues. When the training process meets the termination condition, i.e., the preset accuracy is reached, the system terminal saves the final network parameters and generates the trained resource configuration network layer. This network layer can accurately predict the amount of resources required to process the input information entropy data, providing strong support for subsequent resource configuration updates.

[0069] The update parallel processing module is used to process the multiple conflicting file clusters in parallel.

[0070] In one embodiment, after acquiring the updated parallel processing module, the system terminal uses this module to process multiple conflicting file clusters in parallel. Parallel processing means processing multiple file clusters simultaneously, which can significantly improve processing efficiency. The updated parallel processing module is updated based on the previous resource configuration, optimizing resource allocation for different types of conflicting file clusters, better handling complex situations, and avoiding resource bottlenecks or waste. Therefore, using this updated module, multiple conflicting file clusters can be processed more efficiently, accelerating the reconciliation process and improving overall work efficiency.

[0071] In summary, the embodiments of this application have at least the following technical effects:

[0072] This embodiment extracts multiple query data points from a preset verification window, including external verification codes and verification services. These data are then used to generate transaction files, ensuring the uniqueness of the filename identifier. Subsequently, the transaction files undergo template verification to generate corresponding fund files. Next, a purchase-settlement mapping authentication is performed, generating a file to be verified by mapping the transaction file and fund file one-to-one. In this step, it is determined whether the purchase amount is less than or equal to the settlement amount to determine whether the authentication passes.

[0073] Next, conflict identification is performed on the files to be reconciled. By analyzing the request timestamps, the files to be reconciled are identified as conflicting file clusters for subsequent processing. Then, the resource configuration of the parallel processing module is updated based on the conflicting file clusters and conflict times to improve processing efficiency. Finally, resource configuration is optimized through a resource configuration network layer, improving the efficiency and accuracy of batch reconciliation tasks. These technical effects collectively solve the technical problems of data conflicts, unreasonable resource allocation, and low processing efficiency caused by improper parallel processing in batch reconciliation tasks, achieving real-time extraction, verification, parsing, and authentication of reconciliation data, thereby ensuring data accuracy and consistency. Example

[0074] Based on the same inventive concept as the parallel processing method for batch reconciliation tasks in the foregoing embodiments, as shown in Figure 2, this application provides a parallel processing system for batch reconciliation tasks, the system comprising:

[0075] Data extraction component 1: The data extraction component 1 is used to extract multiple query data from a preset verification window, wherein the multiple query data includes multiple external verification codes and multiple verification services;

[0076] File import component 2: The file import component 2 is used to import files based on the multiple external verification codes and the multiple verification services to generate multiple transaction files, wherein the multiple transaction files include multiple file name identifiers, and the file name identifiers are unique;

[0077] Field parsing component 3: The field parsing component 3 is used to perform template verification on the multiple file name identifiers. If the verification passes, the field parsing is performed on the multiple external verification codes to generate multiple fund files.

[0078] Mapping authentication component 4: The mapping authentication component 4 is used to perform procurement-settlement mapping authentication based on the multiple transaction files and the multiple fund files. If the authentication is successful, multiple files to be reconciled are generated, wherein the multiple files to be reconciled have multiple request time identifiers.

[0079] Conflict identification component 5: The conflict identification component 5 is used to identify reconciliation conflicts of the multiple files to be reconciled based on the multiple request time identifiers, and generate multiple conflict files to be reconciled clusters, wherein the multiple conflict files to be reconciled clusters have multiple conflict times;

[0080] Configuration update component 6: The configuration update component 6 is used to update the resource configuration of the parallel processing module based on the multiple conflict-pending file clusters and the multiple conflict times, so as to obtain an updated parallel processing module;

[0081] Parallel processing component 7: The parallel processing module 7 is used to perform parallel processing on the multiple conflicting file clusters to be revoked using the update parallel processing module.

[0082] Furthermore, the field parsing component 3 is also used to perform the following methods:

[0083] Multiple preset template indicator matrices are extracted based on the multiple file name identifiers, wherein the multiple preset template indicator matrices include store name, product name and settlement price;

[0084] Using the multiple preset template indicator matrices as indexes, indicators are extracted from the multiple transaction files to generate multiple transaction indicator matrices;

[0085] Template verification is performed based on multiple preset template indicator matrices and the multiple transaction indicator matrices. If the store name, product name, and settlement price are all successfully verified, the verification is successful.

[0086] Furthermore, the mapping authentication component 4 is also used to perform the following methods:

[0087] Perform a one-to-one mapping between the multiple transaction files and the multiple fund files to generate multiple transaction file-fund file mapping relationships;

[0088] Based on the multiple transaction file-fund file mapping relationships, the purchase amount and settlement amount are extracted to generate multiple purchase-settlement mapping relationships;

[0089] Determine whether the purchase amount is less than or equal to the settlement amount in the multiple purchase-settlement mapping relationships. If so, the authentication is successful, and the multiple purchase-settlement mapping relationships are treated as multiple pending verification documents.

[0090] If not, the authentication will fail.

[0091] Furthermore, the conflict identification component 5 is also used to perform the following method:

[0092] A first conflict identification unit is constructed by randomly extracting one request time identifier from the plurality of request time identifiers. The first conflict identification unit is used to identify the request time identifiers that meet the conflict window among the plurality of request time identifiers and assign them to the internal nodes of the first conflict identification unit. The remaining plurality of request time identifiers are assigned to the first set to be assigned.

[0093] Another request time identifier is randomly extracted from the first set to be divided to construct a second conflict identification unit. The second conflict identification unit is used to identify the request time identifiers in the first set to be divided that meet the conflict window, and assign them to the internal nodes of the second conflict identification unit. The remaining multiple request time identifiers are then assigned to the second set to be divided.

[0094] After multiple divisions, a request time identifier is randomly extracted from the (N-1)th set to be divided to construct the Nth conflict identification unit. The Nth conflict identification unit is used to identify the request time identifiers in the (N-1)th set to be divided that meet the conflict window, and assign them to the Nth conflict identification internal node. The remaining multiple request time identifiers are assigned to non-conflicting internal nodes.

[0095] A reconciliation conflict identification module is constructed based on the first conflict identification unit, the second conflict identification unit, and the Nth conflict identification unit. The reconciliation conflict identification module is used to identify the multiple files to be reconciled, and to obtain the first conflict identification internal node, the second conflict identification internal node, the Nth conflict identification internal node, and the non-conflicting internal node.

[0096] Furthermore, the conflict identification component 5 is also used to perform the following method:

[0097] The multiple request time identifiers stored in the first conflict identification internal node, the second conflict identification internal node, and the Nth conflict identification internal node are respectively regarded as N conflict-pending file clusters;

[0098] The M request time identifiers in the non-conflicting internal nodes are respectively regarded as M conflicting file clusters to be revoked;

[0099] The N conflicting file clusters and the M conflicting file clusters are referred to as the plurality of conflicting file clusters.

[0100] Furthermore, the configuration update component 6 is also used to perform the following methods:

[0101] Based on the multiple conflict times, the multiple conflict-to-be-verified file clusters are serialized in ascending order to generate a sequence of conflict-to-be-verified file clusters.

[0102] The information entropy is calculated by traversing the sequence of conflicting files to be reconciled, and an information entropy data sequence is generated.

[0103] The information entropy data in the information entropy data sequence is sequentially input into the resource configuration network layer for resource consumption identification, thereby generating a resource consumption data sequence.

[0104] Based on the resource consumption data sequence, the resource configuration of the parallel processing module is updated to generate the updated parallel processing module.

[0105] Furthermore, the configuration update component 6 is also used to perform the following methods:

[0106] Acquire multiple sample information entropy data and multiple sample resource consumption data as training datasets;

[0107] The training dataset is divided equally to obtain n sets of training datasets;

[0108] The framework built on the feedforward neural network is trained in a supervised manner using the n sets of training datasets in sequence, and the model network parameters are updated according to the previous set of training output results until the output results meet the preset output accuracy, thereby generating the resource configuration network layer that has been trained.

[0109] It should be noted that the order of the embodiments described above is merely for descriptive purposes and does not represent the superiority or inferiority of the embodiments. Furthermore, the above description focuses on specific embodiments of this specification. The processes depicted in the accompanying drawings do not necessarily require a specific or sequential order to achieve the desired results. In some implementations, multitasking and parallel processing are possible or may be advantageous.

[0110] The above description is only a preferred embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

[0111] This specification and accompanying drawings are merely illustrative examples of this application and are intended to cover any and all modifications, variations, combinations, or equivalents within the scope of this application. Clearly, those skilled in the art can make various alterations and modifications to this application without departing from its scope. Therefore, if such modifications and modifications fall within the scope of this application and its equivalents, this application intends to include such modifications and modifications.

Claims

1. A parallel processing method for batch reconciliation tasks, characterized in that, The method includes: Extract multiple query data from a preset verification window, wherein the multiple query data includes multiple external verification codes and multiple verification services; Based on the multiple external verification codes and the multiple verification services, files are imported to generate multiple transaction files, wherein the multiple transaction files include multiple file name identifiers, and the file name identifiers are unique; Template validation is performed on the multiple file name identifiers. If the validation passes, the fields of the multiple external verification codes are parsed to generate multiple fund files. Based on the multiple transaction files and the multiple fund files, a procurement-settlement mapping authentication is performed. If the authentication is successful, multiple files to be reconciled are generated, wherein the multiple files to be reconciled have multiple request time identifiers. Based on the multiple request time identifiers, the multiple files to be rewritten are identified for rewriting conflicts, and multiple conflicting file clusters to be rewritten are generated, wherein the multiple conflicting file clusters to be rewritten have multiple conflict times; Based on the multiple conflict-pending file clusters and the multiple conflict times, the resource configuration of the parallel processing module is updated to obtain the updated parallel processing module. The update parallel processing module is used to process the multiple conflicting file clusters in parallel.

2. The method as described in claim 1, characterized in that, The method includes: Multiple preset template indicator matrices are extracted based on the multiple file name identifiers, wherein the multiple preset template indicator matrices include store name, product name and settlement price; Using the multiple preset template indicator matrices as indexes, indicators are extracted from the multiple transaction files to generate multiple transaction indicator matrices; Template verification is performed based on multiple preset template indicator matrices and the multiple transaction indicator matrices. If the store name, product name, and settlement price are all successfully verified, the verification is successful.

3. The method as described in claim 1, characterized in that, The method for performing procurement-settlement mapping authentication based on the multiple transaction files and the multiple fund files includes: Perform a one-to-one mapping between the multiple transaction files and the multiple fund files to generate multiple transaction file-fund file mapping relationships; Based on the multiple transaction file-fund file mapping relationships, the purchase amount and settlement amount are extracted to generate multiple purchase-settlement mapping relationships; Determine whether the purchase amount is less than or equal to the settlement amount in the multiple purchase-settlement mapping relationships. If so, the authentication is successful, and the multiple purchase-settlement mapping relationships are treated as multiple pending verification documents. If not, the authentication will fail.

4. The method as described in claim 1, characterized in that, The method involves identifying rescission conflicts among the multiple request time identifiers and generating multiple conflicting rescission file clusters. A first conflict identification unit is constructed by randomly extracting one request time identifier from the plurality of request time identifiers. The first conflict identification unit is used to identify the request time identifiers that meet the conflict window among the plurality of request time identifiers and assign them to the internal nodes of the first conflict identification unit. The remaining plurality of request time identifiers are assigned to the first set to be assigned. Another request time identifier is randomly extracted from the first set to be divided to construct a second conflict identification unit. The second conflict identification unit is used to identify the request time identifiers in the first set to be divided that meet the conflict window, and assign them to the internal nodes of the second conflict identification unit. The remaining multiple request time identifiers are then assigned to the second set to be divided. After multiple divisions, a request time identifier is randomly extracted from the (N-1)th set to be divided to construct the Nth conflict identification unit. The Nth conflict identification unit is used to identify the request time identifiers in the (N-1)th set to be divided that meet the conflict window, and assign them to the Nth conflict identification internal node. The remaining multiple request time identifiers are assigned to non-conflicting internal nodes. A reconciliation conflict identification module is constructed based on the first conflict identification unit, the second conflict identification unit, and the Nth conflict identification unit. The reconciliation conflict identification module is used to identify the multiple files to be reconciled, and to obtain the first conflict identification internal node, the second conflict identification internal node, the Nth conflict identification internal node, and the non-conflicting internal node.

5. The method as described in claim 4, characterized in that, The multiple request time identifiers stored in the first conflict identification internal node, the second conflict identification internal node, and the Nth conflict identification internal node are respectively regarded as N conflict-pending file clusters; The M request time identifiers in the non-conflicting internal nodes are respectively regarded as M conflicting file clusters to be revoked; The N conflicting file clusters and the M conflicting file clusters are referred to as the plurality of conflicting file clusters.

6. The method as described in claim 1, characterized in that, Based on the multiple conflict-pending file clusters and the multiple conflict times, the resource configuration of the parallel processing module is updated to obtain the updated parallel processing module. The method includes: Based on the multiple conflict times, the multiple conflict-to-be-verified file clusters are serialized in ascending order to generate a sequence of conflict-to-be-verified file clusters. The information entropy is calculated by traversing the sequence of conflicting files to be reconciled, and an information entropy data sequence is generated. The information entropy data in the information entropy data sequence is sequentially input into the resource configuration network layer for resource consumption identification, thereby generating a resource consumption data sequence. Based on the resource consumption data sequence, the resource configuration of the parallel processing module is updated to generate the updated parallel processing module.

7. The method as described in claim 6, characterized in that, The method includes: Acquire multiple sample information entropy data and multiple sample resource consumption data as training datasets; The training dataset is divided equally to obtain n sets of training datasets; The framework built on the feedforward neural network is trained in a supervised manner using the n sets of training datasets in sequence, and the model network parameters are updated according to the previous set of training output results until the output results meet the preset output accuracy, thereby generating the resource configuration network layer that has been trained.

8. A parallel processing system for batch reconciliation tasks, characterized in that, The system is used for implementing any one of the parallel processing methods 1-7 for the batch reconciliation task, and includes: Data extraction component: Extracts multiple query data from a preset verification window, wherein the multiple query data includes multiple external verification codes and multiple verification services; File import component: Based on the multiple external verification codes and the multiple verification services, files are imported to generate multiple transaction files, wherein the multiple transaction files include multiple file name identifiers, and the file name identifiers are unique; Field parsing component: Performs template validation on the multiple file name identifiers. If the validation passes, it performs field parsing on the multiple external verification codes to generate multiple fund files. Mapping authentication component: Performs procurement-settlement mapping authentication based on the multiple transaction files and the multiple fund files. If the authentication is successful, multiple files to be reconciled are generated, wherein the multiple files to be reconciled have multiple request time identifiers. Conflict identification component: Based on the multiple request time identifiers, identify the multiple files to be reconciled for reconciliation conflicts, and generate multiple conflicting file clusters, wherein the multiple conflicting file clusters have multiple conflict times; Configuration update component: Based on the multiple conflict-pending file clusters and the multiple conflict times, update the resource configuration of the parallel processing module to obtain the updated parallel processing module; Parallel processing component: The update parallel processing module is used to process the multiple conflicting file clusters in parallel.

Images