Data processing method and device, electronic equipment, storage medium and program product

Abstract

This application provides a data processing method, apparatus, electronic device, storage medium, and program product. The method includes: obtaining processor instances based on a data processing flow configuration file and a predefined processor abstract base class. The configuration file includes a processor list, processor order, and processor parameters. The processor parameters are used to configure the processor's processing logic. The processor abstract base class defines the type of the request object, the type of the response object, the processor's template method, and chained relationship attributes. Based on the processor order, a chained construction mechanism is used to connect the processor instances to form a data processing chain. Through the request object and the response object, the data to be processed is controlled to be passed along the data processing chain to obtain the data processing result. This method aims to improve the scalability and flexibility of the data processing flow.

Description

Technical Field

[0001] This application relates to the field of databases, and more particularly to a data processing method, apparatus, electronic device, storage medium, and program product. Background Technology

[0002] In complex business scenarios such as database migration assessment and Structured Query Language (SQL) compatibility analysis, there is a need to construct data processing workflows. These workflows may include file decompression, data parsing, SQL transformation, result saving, and report generation. The processing steps in these workflows are dependent on each other and must be executed in a specific order. However, existing data processing methods suffer from poor scalability and flexibility, making it difficult to meet the ever-changing data processing needs of real-world applications.

[0003] Therefore, improving the scalability and flexibility of data processing workflows is an urgent problem to be solved. Summary of the Invention

[0004] This application provides data processing methods, apparatus, electronic devices, storage media, and program products to improve the scalability and flexibility of data processing workflows.

[0005] In a first aspect, embodiments of this application provide a data processing method, including:

[0006] Based on the configuration file of the data processing flow and the predefined abstract base class of the processor, a processor instance is obtained. The configuration file includes a list of processors, a processor order, and processor parameters. The processor parameters are used to configure the processing logic of the processor. The abstract base class of the processor defines the type of the request object, the type of the response object, the template method of the processor, and the chain relationship attribute.

[0007] Based on the processor order, a chain-like construction mechanism is used to connect the processor instances to form a data processing chain;

[0008] The request object and the response object are used to control the transmission of the data to be processed along the data processing chain to obtain the data processing result.

[0009] Optionally, obtaining a processor instance based on the data processing flow configuration file and a predefined processor abstract base class includes:

[0010] Based on the type constraints of the processor abstract base class, the processor classes corresponding to the processor list are loaded through reflection.

[0011] Based on the processor class, create initial processor instances for each processor in the processor list;

[0012] According to the processor order, the chain relationship attributes of each initial processor instance are configured by calling the preset interface of the processor abstract base class;

[0013] The business operation parameters of each initial processor instance are initialized according to the processor parameters, wherein the business operation parameters are the parameters called during the execution of the template method of the processor;

[0014] Generate the processor instance.

[0015] Optionally, the step of configuring the chained relationship attributes of each initial processor instance by calling the preset interface of the processor abstract base class according to the processor order includes:

[0016] Based on the processor sequence, determine the previous and next processor referenced by each processor;

[0017] Configure the previous processor parameters in the chain relationship attribute based on the previous processor referenced by the processor;

[0018] Configure the subsequent processor parameters in the chain relationship attribute according to the next processor referenced by the processor;

[0019] The chain relationship attribute of the initial processor instance is obtained based on the preprocessor parameters and the postprocessor parameters.

[0020] Optional, also includes:

[0021] In the case where the processor includes a target processor with a parent processor, the parent processor parameter in the chain relationship attribute is configured according to the parent processor of the target processor;

[0022] Based on the preprocessor parameters, the postprocessor parameters, and the parent processor parameters, the chain relationship attribute of the initial processor instance corresponding to the processor is obtained.

[0023] Optionally, controlling the transmission of the data to be processed along the data processing chain through the request object and the response object to obtain the data processing result includes:

[0024] During the execution of the data processing chain, data sharing and modification between processors are achieved through the request object, and status transfer and result sharing between processors are achieved through the response object;

[0025] The modified request object and the updated response object are passed along the data processing chain until the tail processor completes the processing and obtains the data processing result.

[0026] Optionally, the processor abstract base class maintains a cache, which includes the response object types corresponding to each processor class. The method further includes:

[0027] When a processor in the data processing chain initializes a response object, it queries the cache to see if there is a response object type corresponding to the processor class of the processor.

[0028] If it exists, a response object instance is created directly based on the response object type in the cache using a reflection mechanism;

[0029] If it does not exist, the response object type corresponding to the processor class is extracted from the generic parameter of the processor abstract base class through reflection, and an instance of the response object is created based on the extracted response object type. At the same time, the mapping relationship between the processor class and the response object type is stored in the cache.

[0030] Optional, also includes:

[0031] If an error or execution exception is detected during the execution of a processor in the data processing chain, the pre-defined early exit handling method in the processor abstract base class is invoked to terminate the execution of the data processing chain. The early exit handling method includes: stopping the execution of the processor's data processing operation and suspending the passing of the request object to the next processor.

[0032] Output the exception message corresponding to the execution exception.

[0033] Secondly, embodiments of this application provide a data processing apparatus, including:

[0034] The first processing module is used to obtain processor instances based on the configuration file of the data processing flow and the predefined processor abstract base class. The configuration file includes a processor list, processor order, and processor parameters. The processor parameters are used to configure the processor's processing logic. The processor abstract base class defines the type of the request object, the type of the response object, the processor's template method, and the chain relationship attribute.

[0035] The second processing module is used to connect the processor instances to form a data processing chain according to the processor order using a chain-building mechanism;

[0036] The control module is used to control the transmission of the data to be processed along the data processing chain through the request object and the response object to obtain the data processing result.

[0037] Thirdly, embodiments of this application provide an electronic device, including: a memory and a processor;

[0038] The memory stores computer-executed instructions;

[0039] The processor executes computer execution instructions stored in the memory, causing the processor to perform the first aspect and / or various possible implementations of the first aspect as described above.

[0040] Fourthly, embodiments of this application provide a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, are used to implement the first aspect and / or various possible implementations of the first aspect.

[0041] Fifthly, embodiments of this application provide a computer program product, including a computer program that, when executed by a processor, implements the first aspect and / or various possible implementations of the first aspect.

[0042] The data processing method, apparatus, electronic device, storage medium, and program product provided in this application generate processor instances based on the configuration file of the data processing flow and a predefined processor abstract base class. Then, a chain-like construction mechanism is used to connect the instances in the order of the processors to form a data processing chain. Request objects and response objects are used to control the transmission of data to be processed in the chain to obtain the processing result. Compared with the prior art, this solution can flexibly configure the processor order according to business changes, avoiding the shortcomings of the traditional sequential processing mode and template method mode in fixing the process, thereby improving the scalability and flexibility of the data processing flow. Attached Figure Description

[0043] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0044] Figure 1 A flowchart illustrating a data processing method provided in an embodiment of this application;

[0045] Figure 2 A flowchart illustrating another data processing method provided in an embodiment of this application;

[0046] Figure 3 A flowchart illustrating another data processing method provided in an embodiment of this application;

[0047] Figure 4 A flowchart illustrating another data processing method provided in an embodiment of this application;

[0048] Figure 5A flowchart illustrating another data processing method provided in an embodiment of this application;

[0049] Figure 6 A flowchart illustrating another data processing method provided in an embodiment of this application;

[0050] Figure 7 This is a schematic diagram of the structure of a data processing device provided in an embodiment of this application;

[0051] Figure 8 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application.

[0052] The accompanying drawings illustrate specific embodiments of this application, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the concept in any way, but rather to illustrate the concept of this application to those skilled in the art through reference to particular embodiments. Detailed Implementation

[0053] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0054] Currently, data flow processing is mainly carried out in the following two ways:

[0055] Traditional sequential processing patterns hard-code business logic into a single method, using sequential if-else or switch-case statements to control the flow. During execution, processing functions are called sequentially according to the business logic, and the return value or exception is checked after each function completes to determine the next step. For simple business logic, its code structure is intuitive and easy to understand. However, for complex business logic, this processing pattern suffers from centralized processing logic, redundant and difficult-to-maintain code, and modifications can easily trigger a chain reaction, resulting in high development costs. Furthermore, this processing pattern has poor scalability and flexibility, making it difficult to adapt to changes in business requirements.

[0056] The template method pattern-based processing flow first defines an abstract base class to plan the process skeleton, and then sets each processing step as an abstract method to be implemented by subclasses. At runtime, the base class template method calls the abstract methods in a fixed order, and the subclasses complete the specific logic. Although this approach separates the process part from the implementation part, improving maintainability, the process order is still hard-coded in the template method. When business requirements change, it is still difficult to dynamically adjust the data processing flow order, and the scalability problem remains.

[0057] In view of this, this application provides a data processing method that generates processor instances based on a configuration file of the data processing flow and a predefined processor abstract base class. Then, it connects the instances in the order of the processors using a chained construction mechanism to form a data processing chain. The method uses request objects and response objects to control the transmission of the data to be processed in the chain to obtain the processing result. Compared with the prior art, this solution can flexibly configure the processor order according to business changes, avoiding the shortcomings of the traditional sequential processing mode and template method mode in fixing the process, thereby improving the scalability and flexibility of the data processing flow.

[0058] The technical solution of this application and how the technical solution of this application solves the above-mentioned technical problems are described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of this application will now be described with reference to the accompanying drawings.

[0059] Figure 1 This is a flowchart illustrating a data processing method provided in an embodiment of this application. Figure 1 As shown, the method includes:

[0060] S101. Based on the configuration file of the data processing flow and the predefined abstract base class of the processor, obtain the processor instance.

[0061] The configuration file includes a list of processors, their order, and their parameters. These parameters configure the processor's processing logic. The abstract base class for processors defines the type of the request object, the type of the response object, the processor's template methods, and chained relationship attributes. The configuration file can be a structured text file describing the rules of the entire data processing flow, such as in JSON or YAML format.

[0062] The processor list is a set of identifiers for all functional processors that will participate in data processing. Each identifier corresponds to a processor class with specific processing logic, such as format validation processor identifiers, content conversion processor identifiers, rule matching processor identifiers, result output processor identifiers, etc.

[0063] The processor order is the execution order of each processor in the processor list. It can be presented in various forms such as linear arrangement and branching arrangement. It also supports if-else type conditional branch configuration based on the condition value in the request object, which is used to clarify the flow path of data between processors.

[0064] Processor parameters are personalized configuration items for each processor, used to limit the specific processing thresholds, rule templates, output formats, and other details of the processor. They may include processing rule sets, processing thresholds, output format templates, and anomaly detection conditions.

[0065] Optionally, the configuration file also supports chain version management and template inheritance. The chain version number can be specified through configuration identifiers. New chain configurations can inherit existing chain templates and override specific processors, enabling chain reuse and rapid adjustment.

[0066] The abstract base class for processors is the generic parent class of all concrete processor classes; for example, it can be defined as ChainHandler.<T extends ChainRequest,R extends ChainResponse> Where T represents the generic request object type and R represents the generic response object type, used to define a uniform specification that all processors must follow.

[0067] The type of the request object is the ChainRequest base class, which refers to the structured type of the data to be processed received by the processor. It can contain multiple dimensions such as the core attributes of the data to be processed, context information, processing requirements, and branch decision conditions. Each dimension corresponds to a data field, forming a structured type with multiple fields.

[0068] The response object is of type ChainResponse, which refers to the structured type of the processing result output by the processor. It can contain multiple dimensions such as processing status identifier, core processing result, exception information record, intermediate process data, and processing progress value. Each dimension corresponds to a data field, forming a structured type with multiple fields.

[0069] The template method of a processor can be, for example, handler(Trequest), which refers to a unified process framework executed by all processors. For example, it can include fixed stages such as response object initialization, data initialization, pre-processing, core logic execution, and result passing. Each stage corresponds to a preset execution logic.

[0070] Chained relational attributes refer to a set of attributes used to store information about the upstream and downstream relationships of processors in a data processing chain. For example, it can include a reference to the previous processor in the chain (i.e., the previous processor previousHandler) and a reference to the next processor in the chain (i.e., the next processor nextHandler). Each reference corresponds to the memory address identifier of a processor instance. The base class can also contain attributes such as response (the response object of the current processor, used to store the processing result) and exitHandler (a boolean flag used to control whether to exit the execution of the chain early).

[0071] Optionally, the chained relationship attribute can also include parent association information in the data processing chain, such as a reference to the parent processor (i.e., the parent processor parentHandler).

[0072] In this step, the configuration file of the data processing flow can be read first. The file content can be structured and parsed by the configuration file parser to extract the specific content of the processor list, processor order, and processor parameters. At the same time, additional configuration information such as version identifier, template inheritance relationship, and conditional branch rules in the configuration can be identified.

[0073] Next, the pre-defined instantiation entry interface in the processor abstract base class is called, and all the parsed configuration information is passed to the interface as input parameters. The interface first performs a validity check on each processor identifier in the processor list based on the type constraints of the processor abstract base class. After the check passes, the corresponding processor class is loaded. Then, an initial processor instance is created based on the loaded processor class. Subsequently, the chain relationship attributes of the initial processor instance are configured in combination with the processor order. At the same time, the business operation parameters of the initial processor instance are initialized using the processor parameters. Finally, a processor instance with complete processing capabilities is generated.

[0074] One possible implementation is to generate processor instances through a chain template inheritance mechanism. This involves first loading an existing chain template, parsing the processor list, order, and parameters in the template, then replacing the processor parameters that need to be covered in the current configuration with those in the template configuration, and generating processor instances based on the merged configuration. This approach is suitable for scenarios with multiple similar processing chains.

[0075] Another possible implementation is to generate processor instances through a version rollback mechanism, which involves reading the historical configuration file of a specified version and generating a corresponding version of the processor instance based on the historical configuration. This is suitable for scenarios that require restoring to historical processing logic.

[0076] S102. Based on the processor order, a chain-like construction mechanism is used to connect processor instances to form a data processing chain.

[0077] Chained construction mechanisms can be implemented, for example, through the predefined setNextHandler(ChainHandler nextHandler) method in the processor abstract base class. This method can set the next processor of the current processor and return the next processor object. It supports chained calls and is used to connect multiple independent processor instances into an organic whole according to a preset execution order. It can include core components such as processor traversal, relationship mapping, and attribute binding.

[0078] Among them, processor traversal refers to accessing each processor instance in the order of processors, relation mapping refers to establishing the association between each processor instance and the preceding and following processor instances, and attribute binding refers to writing the association information into the chained relation attributes of the processor instance.

[0079] When performing this step, you can first sort all processor instances in the processor list according to the processor order to obtain a sorted list of processor instances. At the same time, identify the conditional branching rules in the configuration and mark the corresponding branch path processor list for processor instances with branches.

[0080] Next, iterate through each sorted processor instance in sequence. For each processor instance, find its corresponding previous and next processor instances in the sorted list, and then call the setNextHandler(ChainHandler nextHandler) method, passing the next processor instance as a parameter to the method to complete the configuration of the nextHandler property of the current processor instance, and at the same time assign the reference of the current processor instance to the previousHandler property of the next processor instance.

[0081] For processor instances with conditional branches, the next set of processor instances corresponding to different branch paths can be configured according to the branching rules. After the chain relationship attribute of all processor instances is configured, multiple independent processor instances form a complete data processing chain that supports linear execution and conditional branches, ensuring that the data to be processed can flow through each processor instance sequentially according to the preset path to complete the processing.

[0082] One possible implementation is to use a linear chain construction mechanism, which connects processor instances in processor order into a single linear data processing chain, suitable for branchless linear data processing scenarios.

[0083] Another possible implementation is to use a branch chain construction mechanism, which selects the corresponding branch processing chain for the request object that meets the condition according to the conditional branching rules in the processor sequence. This is suitable for scenarios that need to dynamically adjust the processing path according to data attributes. For example, when the branch judgment condition in the request object is A, the processor instance of the A branch path is executed, and when it is B, the processor instance of the B branch path is executed.

[0084] S103. By using the request object and response object, control the data to be processed to be passed along the data processing chain to obtain the data processing result.

[0085] A request object can be, for example, an instance of the ChainRequest base class, and is a structured data carrier used to carry the data to be processed and its context information. A response object can be, for example, an instance of the ChainResponse base class, and is a structured data carrier used to carry the processor's processing results and its status information.

[0086] When performing this step, you can first create an initial request object and fill the corresponding fields of the initial request object with the core attributes, context information, processing requirements, branch judgment conditions, etc. of the data to be processed. At the same time, create an initial response object, initialize the processing status identifier to the unprocessed state, initialize the processing result field set to an empty set, initialize the exception information record to an empty record, and initialize the processing progress value to 0.

[0087] Next, the template method of the starting processor instance in the data processing chain is called, passing the initial request object as an input parameter to the method. The starting processor instance then executes the processing according to the template method's workflow framework.

[0088] First, the response object is initialized (either through reflection or obtained from the previous handler). Then, the `initData()` method is called to complete data initialization. Next, the `preHandler()` method is called for pre-processing. Then, the `doHandler()` abstract method is called to execute the core logic. During the execution of the core logic, the data content in the request object can be modified, and the processing result and progress value are written to the response object. After processing is complete, the `exitHandler` flag is used to determine whether to continue passing the request. If the flag is false, the modified request object and the updated response object are passed to the next handler instance according to the `nextHandler` reference in the chain relationship attribute or the conditional branching rules. The next handler instance repeats the above processing flow until the end of the data processing chain is completed. At this point, the response object contains the complete processing result, and this response object is output as the final data processing result.

[0089] One possible implementation is to use a mechanism that synchronously transmits request and response objects. That is, after each processor instance completes its processing, it simultaneously passes the modified request object and the updated response object to the next processor instance. This is suitable for scenarios where it is necessary to share the context of the data to be processed and the status of the processing results among processors.

[0090] Another possible implementation is to use a mechanism where the request object is passed separately and the response object is shared globally. That is, after each processor instance completes its processing, it only passes the modified request object to the next processor instance, while the response object is shared by all processor instances through the response property of the processor abstract base class. This is suitable for scenarios that require real-time sharing of processing result status.

[0091] The method provided in this application generates processor instances based on the configuration file of the data processing flow and a predefined abstract base class of processors. Then, it connects the instances in the order of processors using a chained construction mechanism to form a data processing chain. The method uses request objects and response objects to control the transmission of data to be processed in the chain to obtain the processing results. Compared with the prior art, this solution can flexibly configure the processor order according to business changes, avoiding the shortcomings of the traditional sequential processing mode and template method mode that fix the process. This improves the scalability, flexibility and maintainability of the data processing flow, and reduces the adjustment cost when the data processing scenario changes.

[0092] The following section provides a detailed explanation of how the processor instance is obtained in step S101 based on the configuration file of the data processing flow and the predefined processor abstract base class. Figure 2 This is a flowchart illustrating another data processing method provided in an embodiment of this application. Figure 2 As shown, step S101 may specifically include:

[0093] S201. Based on the type constraints of the processor abstract base class, load the processor class corresponding to the processor list through reflection mechanism.

[0094] The type constraint of the processor abstract base class refers to the type restriction of the processor abstract base class on all its subclasses, which may include the type constraint of the request object, the type constraint of the response object, the template method signature constraint, etc.

[0095] Among them, the request object type constraint limits the structured type of the ChainRequest base class or its subclasses received by the subclass processor, the response object type constraint limits the structured type of the ChainResponse base class or its subclasses output by the subclass processor, and the template method signature constraint limits the parameter list and return type of the handler(Trequest) template method that the subclass processor must implement.

[0096] Reflection refers to the implementation logic of dynamically loading classes, creating instances, and calling methods through type information at runtime. It may include core steps such as class path lookup, class loading, and type verification.

[0097] When performing this step, the identifier information of each processor can be extracted from the processor list obtained by parsing the configuration file. Then, based on the type constraints of the processor abstract base class, type verification rules for legal processor classes are constructed. Subsequently, through reflection, the corresponding class path is found according to the identifier information of each processor, and the corresponding processor class is loaded through the class path. After loading, the legality of the loaded processor class is verified using the preset type verification rules. The verification content includes whether the processor class inherits from the generic base class, whether it receives request objects of the ChainRequest base class or its subclasses, whether it outputs response objects of the ChainResponse base class or its subclasses, and whether it implements the handler(Trequest) template method that meets the signature requirements. The processor class that passes the verification is a legal processor class that can be used to create instances.

[0098] One possible implementation is to use a pre-loading reflection mechanism, in which all processor classes that meet the type constraints are loaded into memory during the system initialization phase. Subsequently, it is only necessary to look up and retrieve the processor class corresponding to the processor list from the pre-loaded collection. This is suitable for scenarios where the number of processor classes is fixed and no dynamic addition is required.

[0099] Another possible implementation is to use a reflection mechanism that loads dynamically in real time. The corresponding processor class is searched and loaded in real time based on the identification information in the processor list when this step is executed. This is suitable for scenarios that require dynamic addition or modification of processor classes.

[0100] S202. Based on the processor class, create initial processor instances for each processor in the processor list.

[0101] An initial processor instance refers to a processor entity that has basic processing capabilities but has not yet been configured with chained relationship attributes and business operation parameters. It may contain the core logic code of the processor class, default chained relationship attribute values, default business operation parameter values, etc.

[0102] When performing this step, you can first iterate through the processor classes that have passed the validity check. For each processor class, call the preset initial instance creation interface in the processor abstract base class, and pass the processor class as the input parameter to the interface. The interface can create an instance object of the processor class through reflection. After creation, initialize the chain relationship attributes of the instance object (such as previousHandler, nextHandler, parentHandler, etc.) to null references, initialize the response attribute to a null response object instance of the corresponding type, initialize the exitHandler attribute to false, and initialize the business operation parameters to a set of default values. For example, initialize the processing rule set to default rules, the processing threshold to default threshold, and the output format template to default template. Finally, you get an initial processor instance with basic processing capabilities.

[0103] One possible implementation is to use the singleton pattern to create the initial processor instance, that is, each processor class creates only one initial processor instance, which is suitable for stateless processor scenarios, where multiple requests can share the same processor instance.

[0104] Another possible implementation is to use a multi-instance pattern to create the initial processor instance, that is, a new initial processor instance is created for each request. This is suitable for stateful processor scenarios, where each request's processor instance has independent state information.

[0105] S203. Based on the processor order, call the pre-defined interface of the processor abstract base class to configure the chain relationship attributes of each initial processor instance.

[0106] The predefined interfaces of the abstract base class of the processor can be, for example, the setNextHandler(ChainHandlernextHandler) method, and auxiliary interfaces for configuring the previousHandler and parentHandler properties. These are standardized interfaces for configuring the chain relationship properties of processor instances, and can include functions such as setting parameters for the previous processor, setting parameters for the next processor, and setting parameters for the parent processor. Each interface corresponds to a specific chain relationship parameter configuration operation.

[0107] When performing this step, you can first sort all the initial processor instances in the processor list according to the processor order to obtain a sorted list of initial processor instances.

[0108] Iterate through each sorted initial processor instance in sequence. For each initial processor instance, find its corresponding previous and next processor instances in the sorted list, and then call the setNextHandler(ChainHandler nextHandler) method, passing the next processor instance as a parameter to the method to complete the configuration of the nextHandler property of the current initial processor instance. At the same time, call the auxiliary interface to assign the reference of the current initial processor instance to the previousHandler property of the next processor instance.

[0109] Once the chain relationship attributes of all initial processor instances are configured, the initial processor instances will have clear upstream and downstream relationships.

[0110] One possible implementation is to use an automatic configuration traversal approach. This involves using a pre-defined batch configuration interface in the processor abstract base class, passing the sorted list of initial processor instances as input parameters to the interface. The interface automatically traverses each initial processor instance and calls the setNextHandler method to configure the chained relationship attributes, eliminating the need for manual intervention in the configuration process of each instance.

[0111] Another possible implementation is to manually specify the configuration. For each initial processor instance, the corresponding pre-processor instance and post-processor instance are manually specified, and then the corresponding interface is called to complete the configuration of the chain relationship attributes. This is suitable for complex nested chain scenarios that require special configuration.

[0112] Specifically, this implementation can be achieved through the following sub-steps:

[0113] S2031. Determine the previous and next processors referenced by each processor according to the processor order.

[0114] When performing this step, the processor order can be converted into a structured sequential list, where each element in the list corresponds to a processor identifier. At the same time, the conditional branching rules in the configuration are identified, and the corresponding branch processor identifier set is marked for the processor identifiers that have branches.

[0115] Next, each initial processor instance in the sorted initial processor instance list is traversed sequentially to obtain the processor identifier corresponding to that instance, and the position of that processor identifier is found in the structured sequential list.

[0116] If the position is not the first position in the list, the initial processor instance corresponding to the processor identifier of the previous position is determined as the previous processor of the current initial processor instance. If the position is not the last position in the list, the initial processor instance corresponding to the processor identifier of the next position is determined as the next processor of the current initial processor instance.

[0117] If a branch rule exists at this location, then the initial processor instance corresponding to all branch processor identifiers in the branch rule will be determined as the next set of processors in the branch path of the current initial processor instance.

[0118] If the position is the first position in the list, the previous processor of the current initial processor instance is null. If the position is the last position in the list, the next processor of the current initial processor instance is null.

[0119] S2032. Configure the previous processor parameters in the chain relationship attribute based on the previous processor referenced by the processor.

[0120] When performing this step, for each initial processor instance, if its corresponding previous processor exists, the auxiliary interface preset by the processor abstract base class can be called, and the reference information of the previous processor can be passed to the interface as an input parameter. The interface internally writes the reference information into the previousHandler parameter field of the chain relationship attribute of the current initial processor instance.

[0121] If the corresponding previous processor does not exist, the previousHandler parameter field in the chain relationship property of the current initial processor instance is set to a null reference.

[0122] S2033. Configure the next processor parameter in the chain relationship attribute according to the next processor referenced by the processor.

[0123] When performing this step, for each initial processor instance, if its corresponding next processor exists, the setNextHandler(ChainHandler nextHandler) method is called, passing the reference information of the next processor as an input parameter to the method. Internally, the interface writes this reference information into the nextHandler parameter field of the chain relationship attribute of the current initial processor instance.

[0124] If the corresponding next processor does not exist, the nextHandler parameter field in the chain relationship property of the current initial processor instance is set to a null reference.

[0125] If the next processor corresponding to it is a set of branch paths, then the corresponding triggering conditions are configured for each next processor instance corresponding to each branch path and stored in the branch set of the nextHandler property.

[0126] S2034. Based on the preprocessor parameters and postprocessor parameters, obtain the chain relationship attribute of the initial processor instance.

[0127] When performing this step, for each initial processor instance, the configured previousHandler and nextHandler parameters can be combined into a structured attribute set. This attribute set is then assigned to the chain relationship attribute field of the initial processor instance to complete the configuration of the chain relationship attribute of the initial processor instance. At this point, the chain relationship attribute of the initial processor instance contains complete upstream and downstream relationship information.

[0128] S204. Initialize the service operation parameters of each initial processor instance according to the processor parameters.

[0129] Among them, the business operation parameters are the parameters called during the execution of the processor's template method. They may include processing rule sets, processing thresholds, output format templates, exception judgment conditions, etc. Each item corresponds to a specific parameter value, which is used to limit the specific behavior of the core logic execution phase in the processor's template method.

[0130] When performing this step, you can first extract the personalized parameter content corresponding to each processor identifier from the processor parameters obtained by parsing the configuration file, and then traverse each initial processor instance in turn to obtain the processor identifier corresponding to that instance and find the personalized parameter content corresponding to that processor identifier.

[0131] Then, the pre-defined business parameter initialization interface of the processor abstract base class is called, and the personalized parameter content is passed into the interface as input parameters. Inside the interface, each parameter value in the personalized parameter content is assigned to the corresponding business operation parameter field of the initial processor instance. For example, the processing rule set is assigned to the processing rule field in the business operation parameter, the processing threshold is assigned to the processing threshold field in the business operation parameter, the output format template is assigned to the output format template field in the business operation parameter, and the exception judgment condition is assigned to the exception judgment condition field in the business operation parameter. This completes the initialization of the business operation parameters of the initial processor instance. At this time, the initial processor instance has the processing capability that conforms to the personalized configuration.

[0132] One possible implementation is to use global parameter overriding. First, initialize the business operation parameters of all initial processor instances to default values, and then use the processor parameters in the configuration file to override the corresponding default values. This is suitable for scenarios where some default parameters are shared.

[0133] Another possible implementation is to directly assign personalized parameters, directly assigning the processor parameters in the configuration file to the business operation parameter field of the corresponding initial processor instance, without the need for default value transition, which is suitable for scenarios where the business operation parameters of each processor are completely independent.

[0134] S205, Generate processor instance.

[0135] A processor instance refers to a processor entity with complete processing capabilities. It may include a loaded processor class, configured chained relationship attributes, initialized business operation parameters, etc., and can directly participate in the execution of the data processing chain.

[0136] When performing this step, for each initial processor instance that has completed the configuration of chained relation attributes and the initialization of business operation parameters, the instance confirmation interface preset by the processor abstract base class can be called. This interface internally verifies the configuration completeness of the initial processor instance. The verification content includes whether the chained relation attributes are configured, whether the business operation parameters are initialized, whether the core logic code can be executed normally, whether the exitHandler attribute is initialized to false, and whether the response attribute is initialized to a valid instance. After the verification is passed, the initial processor instance is marked as a processor instance with complete processing capabilities, and the generation of the processor instance is completed.

[0137] One possible implementation is to use a batch generation method, which verifies and marks all initial processor instances that have completed configuration and initialization at once, generating multiple processor instances.

[0138] Another possible implementation is to generate them one by one. For each initial processor instance that has been configured and initialized, a verification and marking process is performed immediately to generate a single processor instance. This approach is suitable for scenarios that require the real-time generation and use of processor instances.

[0139] The method provided in this application embodiment achieves dynamic loading of the processor class through reflection mechanism, ensuring that the processor class conforms to the type constraints of the processor abstract base class. By completing the initial instance creation, chain relationship configuration, and business parameter initialization operations step by step, a processor instance with complete processing capabilities is generated, which can effectively improve the standardization and flexibility of the processor instance generation process, while ensuring that the processing capability of the processor instance meets the preset configuration requirements.

[0140] Figure 3 This is a flowchart illustrating another data processing method provided in an embodiment of this application. Figure 3 As shown, the method may further include:

[0141] S301. If the processor includes a target processor with a parent processor, configure the parent processor parameter in the chain relationship attribute according to the parent processor of the target processor.

[0142] A parent processor is a processor that acts as the superior control node of a certain processor in the data processing flow. It is responsible for the control logic of the target processor, such as startup triggering, result verification, and exception callback. A target processor is a processor that has a parent processor, and its execution process is controlled by the parent processor.

[0143] The parent processor parameter refers to the parentHandler field in the chain relationship attribute of the processor, which is used to store the parent processor reference information and establish the association between the target processor and the parent processor.

[0144] When performing this step, you can first extract the parent processor identifier information corresponding to each processor identifier from the processor parameters obtained by parsing the configuration file, and then filter out the target processor instances that have a parent processor.

[0145] Next, for each target processor instance, find the processor instance corresponding to its parent processor identifier, and then call the pre-defined auxiliary interface of the processor abstract base class, passing the reference information of the parent processor instance as an input parameter to the interface. Inside the interface, the reference information is written into the parentHandler parameter field of the chain relationship attribute of the target processor instance, thus completing the configuration of the parent processor parameter.

[0146] One possible implementation is to use a single parent configuration, where each target processor is configured with only one parent processor, which is suitable for scenarios where the parent processor and the target processor have a one-to-one control relationship.

[0147] Another possible implementation is to use a multi-parent configuration approach, where each target processor can be configured with multiple parent processors. This is suitable for scenarios where the parent processor and the target processor have a many-to-one control relationship, such as when the execution of the target processor needs to meet the triggering conditions of multiple parent processors simultaneously.

[0148] S302. Based on the parameters of the preprocessor, the parameters of the postprocessor, and the parameters of the parent processor, obtain the chain relationship attribute of the initial processor instance corresponding to the processor.

[0149] When performing this step, for each target processor instance, the configured previousHandler parameters, nextHandler parameters, and parentHandler parameters can be combined into a structured attribute set. This attribute set is then assigned to the chain relationship attribute field of the target processor instance. At this point, the chain relationship attribute of the target processor instance contains complete upstream and downstream association information and parent control association information, which can simultaneously meet the requirements of sequential flow, branch processing, and parent control.

[0150] One possible implementation is to directly combine the attributes, concatenating the three parameters into a structured set of attributes, which is suitable for scenarios where there is no correlation or conflict between the three parameters.

[0151] Another possible implementation is to use attribute conflict verification. First, the relationship between the three parameters is checked for conflict. For example, check whether the parent processor has a logical conflict with the previous processor. After the verification is passed, they are combined into a structured attribute set, which is suitable for scenarios with complex relationships.

[0152] The method provided in this application expands the connotation of the chain relationship attribute of the processor by configuring the parent processor parameters of the target processor. It can support the control logic of the parent processor on the target processor, improve the complexity adaptability of the data processing flow, and meet the requirements of data processing scenarios with hierarchical control needs.

[0153] The following section provides a detailed explanation of how, in step S103, the data to be processed is controlled to be passed along the data processing chain through the request object and the response object to obtain the data processing result. Figure 4 This is a flowchart illustrating another data processing method provided in an embodiment of this application. Figure 4 As shown, step S103 may specifically include:

[0154] S401. During the execution of the data processing chain, data sharing and modification between processors are realized through request objects, and status transfer and result sharing between processors are realized through response objects.

[0155] Inter-processor data sharing and modification refers to the process where, as data flows through various processors, each processor can read the data content of the requested object and modify it according to its own processing logic. The modified content can then be read and used by the next processor.

[0156] Inter-processor state transfer and result sharing means that each processor can write its own processing status, processing result, and processing progress value into a response object. The next processor can read the processing status, processing result, and processing progress value from the response object, adjust its own processing logic according to the status information, and share the previous processing result. It can also obtain the specific response result details of the previous processor through the previousHandler.response mechanism.

[0157] When performing this step, the core content of the data to be processed and the branch decision conditions can be read from the request object when the starting processor instance of the data processing chain is executing the processing. The core content is processed according to its own processing logic. After the processing is completed, the modified core content is written to the corresponding field of the request object. At the same time, the processing status identifier, processing result and processing progress value are written to the corresponding field of the response object.

[0158] Next, the initial processor instance determines whether to continue passing the request based on the exitHandler flag. If the flag is false, the modified request object and the updated response object are passed to the next processor instance according to the nextHandler reference in the chain relationship attribute or the conditional branching rules. The next processor instance reads the modified core content from the request object, reads the previous processing status, processing result, and processing progress value from the response object, and can also obtain the response result details of the previous processor through the previousHandler.response mechanism to adjust its own processing logic according to the processing status information.

[0159] For example, if the previous processing status was abnormal, the `exitHandler` flag is set to `true` and processing is terminated. If it was normal, the core logic continues to execute. After execution, the data content in the request object is modified again, and the processing status, processing result, and processing progress value in the response object are updated. Then, it is determined whether to continue passing the request until all processor instances in the data processing chain have completed processing. During this process, the request object completes data sharing and modification among the processors, and the response object completes status passing and result sharing among the processors.

[0160] One possible implementation is to generate a new request object for each processor instance that modifies the request object, while keeping the original request object unchanged. This is suitable for scenarios that require tracing the history of data modifications.

[0161] Another possible implementation is to directly modify the content of the original request object for each processor instance without generating a new request object, which is suitable for scenarios with high memory requirements.

[0162] S402. The modified request object and the updated response object are passed along the data processing chain until the tail processor completes the processing and obtains the data processing result.

[0163] The tail processor is the last processor instance in the data processing chain, responsible for completing the final processing result encapsulation and output.

[0164] Data processing results refer to the structured processing results contained in the response object after the tail processor has completed processing. These results may include complete processing status, multi-dimensional processing result details, summary of exception information, overall processing progress, and other content.

[0165] When performing this step, after each processor instance completes processing, check if the exitHandler flag is true. If it is true, immediately terminate the execution of the data processing chain and output the current response object as the exception handling result.

[0166] If false, the nextHandler reference or conditional branching rule in its chain relationship attribute is used to find the next handler instance. If the next handler instance exists, the modified request object and the updated response object are passed to the next handler instance, triggering the execution of the handler(Trequest) template method of the handler instance.

[0167] If the next processor instance does not exist, the current processor instance becomes the tail processor. At this time, the response object contains the processing status, processing result, and processing progress value of all processor instances. This response object is output as the final data processing result, completing the execution of the data processing flow.

[0168] One possible implementation is to use a synchronous delivery mechanism, where each processor instance immediately delivers the modified request object and the updated response object after completing its processing, and waits for the next processor instance to complete its processing before continuing with subsequent operations. This approach is suitable for scenarios where the order of data processing is critical.

[0169] Another possible implementation is to use an asynchronous delivery mechanism, where each processor instance puts the modified request object and the updated response object into a message queue after completing the processing. The next processor instance retrieves the data from the message queue and performs the processing without waiting for the synchronous notification from the previous processor instance. This is suitable for scenarios with high processing efficiency requirements.

[0170] The method provided in this application, through the combined use of request and response objects, enables the orderly transfer and state sharing of data to be processed between processors. Combined with the exitHandler flag, it achieves flexible process termination control, ensuring the continuity and traceability of the data processing flow, while improving the flexibility and controllability of the data processing process.

[0171] In one possible implementation, the processor abstract base class maintains a cache, which includes the response object types corresponding to each processor class.

[0172] The cache maintained by the processor abstract base class refers to a collection of key-value pairs stored in memory, where the key is the identifier information of the processor class, and the value is the response object type corresponding to that processor class. The response object type refers to the structured type of the processing result output by the processor, which can contain multi-dimensional content such as processing status identifiers, processing result details, and exception information records.

[0173] Specifically, Figure 5 This is a flowchart illustrating another data processing method provided in an embodiment of this application. Figure 5 As shown, the method may further include:

[0174] S501. When a processor in the data processing chain initializes a response object, it checks whether the response object type corresponding to the processor class of the processor exists in the cache.

[0175] When performing this step, the response object initialization phase of the handler(Trequest) template method can be started at a certain processor instance in the data processing chain to obtain the identification information of the processor class corresponding to that processor instance.

[0176] Next, the cache lookup interface preset by the processor abstract base class is called, and the identifier information of the processor class is passed to the interface as an input parameter. The interface searches in the cache for a key-value pair with the identifier information as the key. If it exists, it is determined that the corresponding response object type exists in the cache. If it does not exist, it is determined that the corresponding response object type does not exist in the cache.

[0177] S502. If it exists, then create a response object instance directly based on the response object type in the cache through reflection.

[0178] When performing this step, after finding that the corresponding response object type exists in the cache, the pre-defined response object creation interface of the processor abstract base class can be called. The response object type stored in the cache is passed to the interface as an input parameter. The interface uses reflection to create the corresponding response object instance according to the response object type. After creation, the various fields of the response object instance are initialized to default values. For example, the processing status identifier is initialized to unprocessed status, the processing result details are initialized to an empty set, the exception information record is initialized to an empty record, and the processing progress value is initialized to 0.

[0179] S503. If it does not exist, the response object type corresponding to the processor class is extracted from the generic parameter of the processor abstract base class through reflection, and a response object instance is created based on the extracted response object type. At the same time, the mapping relationship between the processor class and the response object type is stored in the cache.

[0180] When performing this step, after finding that the corresponding response object type does not exist in the cache, the generic parameter extraction interface preset by the processor abstract base class can be called. This interface internally uses reflection to extract the parameter from the ChainHandler.<T extendsChainRequest,R extends ChainResponse> Extract the response object type corresponding to the current processor class from the R parameter of the generic base class.

[0181] After extraction, the response object creation interface is called to create a response object instance based on the extracted response object type. Then, the cache update interface preset by the processor abstract base class is called, and the identification information of the current processor class and the extracted response object type are passed to the interface as key-value pairs. The interface stores the key-value pairs in the cache to complete the cache update. When other instances of the same processor class initialize response objects in the future, they can directly obtain the corresponding response object type from the cache.

[0182] The method provided in this application embodiment stores the mapping relationship between processor classes and response object types through a cache maintained by the processor abstract base class. This avoids repeatedly extracting the response object type from generic parameters, effectively improving the creation efficiency of response object instances, while ensuring the consistency and accuracy of response object types.

[0183] Figure 6 This is a flowchart illustrating another data processing method provided in an embodiment of this application. Figure 6 As shown, the method may further include:

[0184] S601. If an error or execution exception is detected during the execution of the processor in the data processing chain, the pre-defined early exit handling method in the processor abstract base class is called to terminate the execution of the data processing chain.

[0185] One example of an early exit handling method is the `exitHandler()` method, which can accept an error message or exception object as a parameter, set the `exitHandler` flag to `true`, and log error information, including stopping the data processing operations of the processor and aborting the passing of the request object to the next processor.

[0186] Errors or execution exceptions refer to unexpected situations that occur during the core logic execution phase of a template method, and can include data format errors, processing logic exceptions, resource access failures, and results that do not conform to validation rules.

[0187] Early exit handling methods refer to the unified methods pre-defined in the processor abstract base class for handling execution exceptions, which may include core components such as operation stoppage, process abort, status recording, and error information storage.

[0188] When performing this step, the core logic execution phase of the template method can be executed in each processor instance in the data processing chain. A preset exception detection mechanism can be used to detect whether errors or execution exceptions occur in real time. The exception detection mechanism may include data validity verification, logic execution result verification, resource access status verification, etc.

[0189] If an error or execution exception is detected, the `exitHandler()` method is immediately called, passing the error message or exception object as a parameter. Inside the method, the `exitHandler` flag is set to `true`, the error information is written to the exception information record field of the response object, the processing status is updated to an exception status, the data processing operation being executed by the current processor instance is stopped, the currently occupied system resources are released, the operation of passing the modified request object and the updated response object to the next processor is aborted, and then the execution of the entire data processing chain is terminated, and no further processing operations of subsequent processor instances are triggered.

[0190] One possible implementation is to use a partial exit mechanism, which terminates only the execution of the current processor instance and its subsequent processor instances, while the results of processor instances that have already completed processing remain unchanged. This is suitable for scenarios where some processing results are still valuable.

[0191] Another possible implementation is to use a global rollback mechanism, which rolls back the operations of all processor instances that have completed processing after terminating the execution of the data processing chain, restoring the state before the data processing started. This is suitable for scenarios with high requirements for data consistency.

[0192] S602. Output the exception message corresponding to the execution exception.

[0193] Exception message information refers to structured information used to describe the specific circumstances of execution exceptions. It may include the processor identifier where the exception occurred, the exception type, the time of the exception, a detailed description of the exception, the current processing progress value, etc. Each piece of information corresponds to an information field, forming a structured message information composed of multiple fields.

[0194] When performing this step, after terminating the data processing chain by calling the early exit handling method, the processor identifier, exception type, exception occurrence time, detailed exception description, and current processing progress value of the exception can be extracted from the exception information record field and processing progress value field of the response object. These contents are then combined into a structured exception message. Next, the preset information output interface is called to output the exception message to the specified output channel. The output channel can include log files, monitoring platforms, exception alarm systems, etc., to ensure that relevant personnel can obtain exception information and handle it in a timely manner.

[0195] The method provided in this application embodiment, through a preset early exit processing method and anomaly prompt information output operation, can terminate the process in a timely manner and notify relevant personnel when an anomaly occurs during data processing, avoiding invalid subsequent processing operations, reducing the waste of system resources, and improving the reliability and maintainability of the data processing process.

[0196] Optionally, this application may also include a fine-grained progress tracking method. This means that during the core logic execution phase of the template method in each processor instance, the processor updates the processing progress value in the response object in real time based on its own processing progress. The processing progress value can be expressed as a percentage, ranging from 0 to 100. Specifically, the overall processing progress of the data processing chain can be calculated by summing the progress values ​​of all completed processor instances and the real-time progress value of the currently executing processor instance. For example, the overall progress value can be obtained by combining the proportion of completed processors to the total number of processors, the average progress value of each completed processor, and the proportion of the current processor's real-time progress value to the total number of processors.

[0197] Fine-grained progress tracking allows developers and users to gain a more precise understanding of the execution status of the data processing chain at each stage, thereby promptly identifying potential performance bottlenecks or anomalies. This facilitates quick problem location and optimization, improving data processing efficiency and reliability, while providing users with a more transparent and predictable service experience.

[0198] Optionally, this application may also include chain execution state persistence. That is, after each processor instance in the data processing chain completes its processing, the chain's execution state can be persisted. The execution state may include the currently executing processor identifier, a list of completed processor instances, the processing result of each processor, the current state of the request object, the current state of the response object, and processing progress values. The persistent storage medium may include relational databases, non-relational databases, distributed file systems, etc. During storage, a unique execution identifier can be assigned to each execution request, and the execution state can be associated with and stored along with the execution identifier.

[0199] By persisting the execution state of the chain, the execution state of the data processing chain can be quickly restored when the system fails, restarts, or needs to trace historical processing. This ensures the continuity and integrity of data processing, avoids data loss or duplicate processing due to unexpected situations, improves the system's fault tolerance and maintainability, and also provides strong support for subsequent data analysis and auditing.

[0200] Optionally, this application may also include a breakpoint resumption mechanism, which can check whether there are any incomplete execution records associated with the current execution request before the data processing chain starts execution. If there are, information such as the failure processor identifier, the list of completed processor instances, intermediate results, the historical state of the request object, and the historical state of the response object are read from the record, the state of the request object and the response object is restored, and the data processing chain continues to be executed from the failure processor instance without re-executing the completed processor instance. If there are no incomplete execution records, execution starts from the starting processor instance according to the normal process.

[0201] By using the breakpoint resume mechanism, when data processing is interrupted due to various reasons (such as network failure, system crash, etc.), it is possible to avoid re-executing processor instances that have already been successfully completed, thereby reducing unnecessary computing resources and time consumption, improving the overall efficiency of data processing, especially for processing complex and time-consuming data processing chains, which can effectively improve user experience and system stability.

[0202] Optionally, this application may also include real-time progress push, which means that information such as the overall processing progress of the data processing chain, the identifier of the currently executing processor, and the processing progress value of the current processor can be pushed to the front-end system in real time through technologies such as WebSocket or message queues. The front-end system can then display this information to the user, allowing the user to see the real-time processing progress and the currently executing steps. Real-time progress push allows users to obtain data processing status information in real time without actively refreshing the page or frequently querying, thereby enhancing the interactivity and real-time nature between the user and the system, increasing user trust and satisfaction with the system, and also helping developers to monitor the system's operating status in a timely manner and quickly respond to and handle potential problems based on user feedback.

[0203] Figure 7 A schematic diagram of the structure of a data processing apparatus provided in an embodiment of this application is shown below. Figure 7 As shown, the data processing device provided in this embodiment includes: a first processing module 11, a second processing module 12, and a control module 13.

[0204] The first processing module 11 is used to obtain processor instances based on the configuration file of the data processing flow and the predefined processor abstract base class. The configuration file includes a list of processors, processor order, and processor parameters. The processor parameters are used to configure the processor's processing logic. The processor abstract base class defines the type of the request object, the type of the response object, the processor's template method, and the chain relationship attributes.

[0205] The second processing module 12 is used to connect processor instances to form a data processing chain according to the processor order using a chain-like construction mechanism.

[0206] The control module 13 is used to control the transmission of the data to be processed along the data processing chain through the request object and the response object to obtain the data processing result.

[0207] Optionally, the first processing module 11 is specifically used to load the processor classes corresponding to the processor list through reflection, based on the type constraints of the processor abstract base class. According to the processor class, it creates initial processor instances for each processor in the processor list. According to the processor order, it calls the preset interface of the processor abstract base class to configure the chained relationship attributes of each initial processor instance. It initializes the business operation parameters of each initial processor instance according to the processor parameters. Finally, it generates processor instances. The business operation parameters are the parameters called during the execution of the processor's template method.

[0208] Optionally, the first processing module 11 is specifically used to determine the previous and next processors referenced by each processor according to the processor order. Based on the previous processor referenced by the processor, it configures the previous processor parameters in the chain relationship attribute. Based on the next processor referenced by the processor, it configures the subsequent processor parameters in the chain relationship attribute. Based on the previous and subsequent processor parameters, it obtains the chain relationship attribute of the initial processor instance.

[0209] Optionally, the first processing module 11 is further configured to, when the processor includes a target processor with a parent processor, configure the parent processor parameter in the chain relationship attribute according to the parent processor of the target processor. Based on the preceding processor parameter, the following processor parameter, and the parent processor parameter, the chain relationship attribute of the initial processor instance corresponding to the processor is obtained.

[0210] Optionally, the control module 13 is specifically used to achieve data sharing and modification between processors through request objects and to achieve status transmission and result sharing between processors through response objects during the execution of the data processing chain. The modified request object and the updated response object are passed along the data processing chain until the processor at the end of the chain completes processing and obtains the data processing result.

[0211] Optionally, if the processor abstract base class maintains a cache containing response object types corresponding to each processor class, the first processing module 11 is further configured to query the cache for the existence of a response object type corresponding to the processor class when the processor initializes a response object in the data processing chain. If it exists, a response object instance is created directly based on the response object type in the cache using reflection. If it does not exist, the response object type corresponding to the processor class is extracted from the generic parameters of the processor abstract base class using reflection, and a response object instance is created based on the extracted response object type. Simultaneously, the mapping relationship between the processor class and the response object type is stored in the cache.

[0212] Optionally, the first processing module 11 is further configured to, during the execution of a processor in the data processing chain, if an error or execution exception is detected, call a pre-defined early exit handling method in the processor's abstract base class to terminate the execution of the data processing chain. It also outputs the exception message corresponding to the execution exception. The early exit handling method includes: stopping the processor's data processing operations and aborting the passing of the request object to the next processor.

[0213] The data processing device provided in this embodiment can execute the method provided in the above method embodiment. Its implementation principle and technical effect are similar, and will not be described in detail here.

[0214] Figure 8This is a schematic diagram of an electronic device provided in an embodiment of this application. The electronic device is used to execute the aforementioned data processing method. Figure 8 As shown, the electronic device 800 may include at least one processor 801, a memory 802, and a communication interface 803.

[0215] The memory 802 is used to store programs. Specifically, the program may include program code, which includes computer operation instructions.

[0216] The memory 802 may include high-speed RAM memory, and may also include non-volatile memory, such as at least one disk storage device.

[0217] The processor 801 is used to execute computer execution instructions stored in the memory 802 to implement the method described in the foregoing method embodiments. The processor 801 may be a CPU, an Application Specific Integrated Circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of this application.

[0218] The processor 801 can communicate and interact with external devices through the communication interface 803. In specific implementations, if the communication interface 803, memory 802, and processor 801 are implemented independently, they can be interconnected via a bus to complete communication. The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc., but this does not imply that there is only one bus or one type of bus.

[0219] Optionally, in a specific implementation, if the communication interface 803, memory 802, and processor 801 are integrated on a single chip, then the communication interface 803, memory 802, and processor 801 can communicate through an internal interface.

[0220] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the above-described method.

[0221] This application also provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, implement the above-described method.

[0222] The aforementioned readable storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. The readable storage medium can be any available medium accessible to a general-purpose or special-purpose computer.

[0223] An exemplary readable storage medium is coupled to a processor, enabling the processor to read information from and write information to the readable storage medium. Of course, the readable storage medium can also be a component of the processor. The processor and the readable storage medium can reside in an Application Specific Integrated Circuit (ASIC). Alternatively, the processor and the readable storage medium can exist as discrete components in the device.

[0224] The division of units is merely a logical functional division; in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be indirect coupling or communication connection through some interfaces, devices, or units, and may be electrical, mechanical, or other forms.

[0225] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0226] In addition, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0227] If a function is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0228] Those skilled in the art will understand that all or part of the steps of the above-described method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When executed, the program performs the steps of the above-described method embodiments; and the aforementioned storage medium includes various media capable of storing program code, such as ROM, RAM, magnetic disks, or optical disks.

[0229] Finally, it should be noted that other embodiments of the invention will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This invention is intended to cover any variations, uses, or adaptations of the invention that follow the general principles of the invention and include common knowledge or customary techniques in the art not disclosed herein, and is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of the invention is limited only by the appended claims.

Claims

1. A data processing method, characterized in that, include: Based on the configuration file of the data processing flow and the predefined abstract base class of the processor, a processor instance is obtained. The configuration file includes a list of processors, a processor order, and processor parameters. The processor parameters are used to configure the processing logic of the processor. The abstract base class of the processor defines the type of the request object, the type of the response object, the template method of the processor, and the chain relationship attribute. Based on the processor order, a chain-like construction mechanism is used to connect the processor instances to form a data processing chain; The request object and the response object are used to control the transmission of the data to be processed along the data processing chain to obtain the data processing result.

2. The method according to claim 1, characterized in that, The process of obtaining a processor instance based on the configuration file of the data processing flow and the predefined processor abstract base class includes: Based on the type constraints of the processor abstract base class, the processor classes corresponding to the processor list are loaded through reflection. Based on the processor class, create initial processor instances for each processor in the processor list; According to the processor order, the chain relationship attributes of each initial processor instance are configured by calling the preset interface of the processor abstract base class; The business operation parameters of each initial processor instance are initialized according to the processor parameters, wherein the business operation parameters are the parameters called during the execution of the template method of the processor; Generate the processor instance.

3. The method according to claim 2, characterized in that, The step of configuring the chained relationship attributes of each initial processor instance by calling the preset interface of the processor abstract base class according to the processor order includes: Based on the processor order, determine the previous and next processor referenced by each processor; Configure the previous processor parameters in the chain relationship attribute based on the previous processor referenced by the processor; Configure the subsequent processor parameters in the chain relationship attribute according to the next processor referenced by the processor; The chain relationship attribute of the initial processor instance is obtained based on the preprocessor parameters and the postprocessor parameters.

4. The method according to claim 3, characterized in that, Also includes: In the case where the processor includes a target processor with a parent processor, the parent processor parameter in the chain relationship attribute is configured according to the parent processor of the target processor; Based on the preprocessor parameters, the postprocessor parameters, and the parent processor parameters, the chain relationship attribute of the initial processor instance corresponding to the processor is obtained.

5. The method according to claim 1, characterized in that, The process of controlling the transmission of data to be processed along the data processing chain through the request object and the response object to obtain the data processing result includes: During the execution of the data processing chain, data sharing and modification between processors are achieved through the request object, and status transfer and result sharing between processors are achieved through the response object; The modified request object and the updated response object are passed along the data processing chain until the tail processor completes the processing and obtains the data processing result.

6. The method according to claim 1, characterized in that, The processor abstract base class maintains a cache, which includes the response object types corresponding to each processor class. The method further includes: When a processor in the data processing chain initializes a response object, it queries the cache to see if there is a response object type corresponding to the processor class of the processor. If it exists, a response object instance is created directly based on the response object type in the cache using a reflection mechanism; If it does not exist, the response object type corresponding to the processor class is extracted from the generic parameter of the processor abstract base class through reflection, and an instance of the response object is created based on the extracted response object type. At the same time, the mapping relationship between the processor class and the response object type is stored in the cache.

7. The method according to claim 1, characterized in that, Also includes: If an error or execution exception is detected during the execution of a processor in the data processing chain, the pre-defined early exit handling method in the processor abstract base class is invoked to terminate the execution of the data processing chain. The early exit handling method includes: stopping the execution of the processor's data processing operation and suspending the passing of the request object to the next processor. Output the exception message corresponding to the execution exception.

8. A data processing apparatus, characterized in that, include: The first processing module is used to obtain processor instances based on the configuration file of the data processing flow and the predefined processor abstract base class. The configuration file includes a processor list, processor order, and processor parameters. The processor parameters are used to configure the processor's processing logic. The processor abstract base class defines the type of the request object, the type of the response object, the processor's template method, and the chain relationship attribute. The second processing module is used to connect the processor instances to form a data processing chain according to the processor order using a chain-building mechanism; The control module is used to control the transmission of the data to be processed along the data processing chain through the request object and the response object to obtain the data processing result.

9. An electronic device, characterized in that, include: Memory, processor; The memory stores computer-executed instructions; The processor executes computer execution instructions stored in the memory, causing the processor to perform the method as described in any one of claims 1-7.

10. A computer program product, characterized in that, Includes a computer program that, when executed by a processor, implements the method described in any one of claims 1-7.

Images