A data processing method, device, equipment and storage medium

Abstract

The embodiment of the present invention discloses a data processing method, device, equipment and storage medium, the method comprising: obtaining target data to be processed and a target data processing identifier corresponding to the target data; determining the target link configuration information corresponding to the target link for processing the target data based on the pre-configured link configuration information corresponding to each data processing identifier and the target data processing identifier, the target link configuration information comprising: a data processing script corresponding to each target node in the target link and a target execution order corresponding to each target node; loading and compiling the data processing script corresponding to each target node to generate a target instance object corresponding to each target node; processing the target data based on the target execution order and each target instance object. Through the technical solution of the embodiment of the present invention, the dynamic configuration generation of the data processing link can be realized, thereby improving the development efficiency and the flexibility of data processing.

Description

Technical Field

[0001] The embodiments of the present invention relate to Internet technology, and in particular to a data processing method, apparatus, device and storage medium. Background Art

[0002] With the rapid development of Internet technology, software service as an innovative software application model has received more and more attention. Usually, a business process often needs to interact with many data interfaces to realize the corresponding business functions. For the data interface of collaborative services or the flow data interface of business data, the received data is often split, packaged, heterogeneous and reorganized based on business needs, and the processed data is returned.

[0003] Currently, developers can directly write the code corresponding to the processing logic inside the data interface according to business needs, so as to process data by directly executing the code. It is also possible to define a configuration file described by JSON (JavaScript Object Notation) or XML (Extensible Markup Language), so that the data interfaces that call each other can perform data processing operations according to the configuration file.

[0004] However, in the process of implementing the present invention, the inventors found that there are at least the following problems in the prior art:

[0005] When business requirements change, the code needs to be rewritten or the configuration files need to be reconfigured, and the rewritten code or configuration files need to be republished online to complete the change in business requirements, which greatly reduces development efficiency. Moreover, once the two existing data processing methods are implemented, it is impossible to dynamically change the data processing link inside the data interface based on business requirements, that is, it is impossible to dynamically adjust the execution order of each node in the data processing link, or recombine nodes to generate a new data processing link, thereby reducing the flexibility of data processing and failing to meet user personalized needs. Summary of the invention

[0006] The embodiments of the present invention provide a data processing method, apparatus, device and storage medium to realize dynamic configuration generation of data processing links, and do not need to be re-published when business requirements change, thereby improving development efficiency and data processing flexibility, and further meeting user personalized needs.

[0007] In a first aspect, an embodiment of the present invention provides a data processing method, including:

[0008] Acquire target data to be processed and a target data processing identifier corresponding to the target data;

[0009] Based on the pre-configured link configuration information corresponding to each data processing identifier and the target data processing identifier, determining the target link configuration information corresponding to the target link for processing the target data, the target link configuration information including: a data processing script corresponding to each target node in the target link and a target execution order corresponding to each target node;

[0010] Load and compile the data processing script corresponding to each target node to generate a target instance object corresponding to each target node;

[0011] The target data is processed based on the target execution order and each of the target instance objects.

[0012] In a second aspect, an embodiment of the present invention further provides a data processing device, including:

[0013] A target data acquisition module, used to acquire target data to be processed and a target data processing identifier corresponding to the target data;

[0014] A target link configuration information determination module is used to determine the target link configuration information corresponding to the target link for processing the target data based on the pre-configured link configuration information corresponding to each data processing identifier and the target data processing identifier, wherein the target link configuration information includes: a data processing script corresponding to each target node in the target link and a target execution order corresponding to each target node;

[0015] A target instance object generation module, used to load and compile the data processing script corresponding to each of the target nodes, and generate a target instance object corresponding to each of the target nodes;

[0016] The target data processing module is used to process the target data based on the target execution order and each of the target instance objects.

[0017] In a third aspect, an embodiment of the present invention further provides an electronic device, the electronic device comprising:

[0018] one or more processors;

[0019] A memory for storing one or more programs;

[0020] When the one or more programs are executed by the one or more processors, the one or more processors implement the data processing method provided by any embodiment of the present invention.

[0021] In a fourth aspect, an embodiment of the present invention further provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements a data processing method as provided in any embodiment of the present invention.

[0022] The embodiments of the above invention have the following advantages or beneficial effects:

[0023] By pre-configuring the link configuration information corresponding to each data processing identifier, that is, dynamically configuring the data processing script corresponding to each node in the link used to process each type of business data and the execution order corresponding to each node, when processing the target data corresponding to the target data processing identifier, the data processing script corresponding to each target node in the target link configuration information can be loaded and compiled to generate a target instance object corresponding to each target node, and based on the target execution order of each target node and each target instance object, the target data is processed, so that the target link for processing the target data can be dynamically generated based on the target link configuration information, realizing dynamic configuration generation of the data processing link, greatly improving the flexibility of data processing, and thus meeting the personalized needs of users, and each target node in the target link processes data based on the data processing script, so that when business needs change, only the data processing script in the configuration information needs to be adjusted to take effect, without the need to rewrite the logic code and re-publish it online, thereby improving development efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 is a flow chart of a data processing method provided by Embodiment 1 of the present invention;

[0025] Figure 2 is a schematic diagram of a data processing framework involved in Embodiment 1 of the present invention;

[0026] Figure 3 This is an example of a data processing flow involved in the first embodiment of the present invention;

[0027] Figure 4 is a flow chart of a data processing method provided by Embodiment 2 of the present invention;

[0028] Figure 5 This is an example of a target node creation process involved in the second embodiment of the present invention;

[0029] Figure 6 This is an example of a target link creation process involved in the second embodiment of the present invention;

[0030] Figure 7 is a structural schematic diagram of a data processing device provided in Embodiment 3 of the present invention;

[0031] Figure 8It is a structural schematic diagram of an electronic device provided in Embodiment 4 of the present invention. DETAILED DESCRIPTION

[0032] The present invention will be further described in detail below in conjunction with the accompanying drawings and embodiments. It is to be understood that the specific embodiments described herein are only used to explain the present invention, rather than to limit the present invention. It should also be noted that, for ease of description, only parts related to the present invention, rather than all structures, are shown in the accompanying drawings.

[0033] Embodiment 1

[0034] Figure 1 This is a flowchart of a data processing method provided in the first embodiment of the present invention. This embodiment can be applied to the case where data obtained from a data interface is processed, especially in the scenario where data needs to be split, packaged, heterogeneous and reorganized. The method can be executed by a data processing device, which can be implemented by software and / or hardware and integrated into an electronic device with data processing function. Figure 1 As shown, the method specifically comprises the following steps:

[0035] S110, obtaining target data to be processed and a target data processing identifier corresponding to the target data.

[0036] Among them, the target data can be business data that is obtained by calling a data interface and needs to undergo data processing operations, such as business data that needs to undergo processing operations such as splitting, packaging, heterogeneity and / or reorganization. The data processing identifier can be used to identify the processing requirements of the business data. When the data processing requirements are different, that is, when the data processing logic is different, the corresponding data processing identifiers are also different, so that different data processing requirements can be distinguished by using different data processing identifiers to meet different business needs. For example, the business type identifier corresponding to the data to be processed can be used as the data processing identifier. The target data processing identifier can refer to the data processing identifier used to process the target data.

[0037] Specifically, by calling the data interface of the collaborative service or the flow data interface of business data, the data interface of the collaborative service or the flow data interface of business data can obtain the target data to be processed and the target data processing identifier corresponding to the target data, so as to perform processing operations on the target data based on the data processing logic corresponding to the target data processing identifier.

[0038] S120. Based on the pre-configured link configuration information corresponding to each data processing identifier and the target data processing identifier, determine the target link configuration information corresponding to the target link used to process the target data. The target link configuration information includes: a data processing script corresponding to each target node in the target link and a target execution order corresponding to each target node.

[0039] Among them, the link configuration information corresponding to the data processing identifier may be the data processing logic information corresponding to the link for processing the data to be processed corresponding to the data processing identifier. Each link configuration information includes: a data processing script corresponding to each node in the link and a target execution order corresponding to each node. Each link may correspond to a data processing logic. Each node in the link may refer to each step in the data processing logic. The execution order corresponding to each node may refer to the execution order of each step. Each data processing may go through one or more steps, that is, each link may pass through one or more nodes. The specific nodes passed through and the execution order of each node may be dynamically determined based on the link configuration information, so that the link can be dynamically and flexibly configured, and the data processing logic can be customized and modified at any time, without the need to invasively modify the caller system, thereby improving development efficiency.

[0040] Specifically, the link configuration information corresponding to each data processing identifier can be pre-configured based on business requirements, and the configured link configuration information can be stored, so that when processing target data, the target link configuration information corresponding to the target data processing identifier can be determined based on the target data processing identifier. By parsing the target link configuration information, the data processing script corresponding to each target node in the target link for processing the target data and the target execution order corresponding to each target node can be obtained.

[0041] S130. Load and compile the data processing script corresponding to each target node to generate a target instance object corresponding to each target node.

[0042] The target instance object may be an instantiated object constructed based on a data processing script and used to execute script processing logic. Each target node corresponds to a target instance object, so that the data processing script corresponding to the corresponding target node is executed through the target instance object to realize the corresponding data processing logic function.

[0043] Specifically, for each target node, the data processing script corresponding to the target node is read, and the data processing script is loaded and compiled, and the data processing script in string format is compiled into an executable binary script, thereby constructing a script processing class, and the script processing class is instantiated to obtain a target instance object corresponding to each target node. In this embodiment, Java can be used to componentize and encapsulate the data processing script corresponding to each target node, so as to dynamically and flexibly assemble the nodes in the actual link, thereby meeting different data processing requirements.

[0044] S140: Process the target data based on the target execution order and each target instance object.

[0045] Specifically, after generating the target instance object corresponding to each target node in the target link, the calling order of each target instance object can be obtained based on the target execution order corresponding to each target node, thereby dynamically generating the target link, realizing the dynamic configuration generation of the data processing link, and the dynamically generated target link can be used to process the target data. This embodiment can dynamically adjust the data processing link inside the data interface by adjusting the link configuration information, such as dynamically adjusting the execution order of each node in the data processing link, or recombining the nodes to dynamically generate a new data processing link, thereby greatly improving the flexibility of data processing, and thus meeting the personalized needs of users. Moreover, each target node in the target link performs data processing based on the data processing script, so that when the data processing logic of each node needs to be adjusted, the data processing script in the configuration information can be adjusted through the visual view to take effect, without rewriting the logic code and re-publishing it online, thereby improving development efficiency and further improving the scalability of data processing.

[0046] Exemplarily, S140 may include: based on the target execution order, calling the data processing function in the target instance object corresponding to each target node in sequence, and processing the target data in sequence.

[0047] The data processing function may refer to a function within the target instance object that is used to execute the corresponding data processing logic. Specifically, based on the target execution order in the target link, the data processing function in the target instance object corresponding to each target node may be called in sequence, so that the target data is sequentially processed at each target node in sequence, so that the final data processing result can be obtained at the last target node, and the final data processing result can be returned, completing the entire data processing process.

[0048] For example, Figure 2 A schematic diagram of a data processing framework is given. Figure 2As shown, the data processing framework may include: an input interface, a node assembler component, a link generator component and a data processing output. Among them, the input interface may refer to an interface for receiving data to be processed in a data interface of a collaborative service or a flow data interface of business data. The node assembler component may include a dynamic script processor. The dynamic script processor may be used to process the data processing script and assemble each node in the generated link for processing data. The link generator component may include: a configuration management module, a configuration parsing module and a link generation module. The configuration management module may be used to query and manage link configuration information. The configuration parsing module may be a module for parsing link configuration information. The link generation module may be a module for dynamically generating executable links. Figure 3 An example of a data processing flow is given. Figure 3 As shown, this embodiment can obtain the target data to be processed and the target data processing identifier corresponding to the target data through the input interface. The link configuration information corresponding to each data processing identifier pre-stored in the database can be stored in the cache unit, so that the configuration management module can query and determine the target link configuration information based on the target data processing identifier in the cache unit, so as to improve the query efficiency and further improve the data processing efficiency. The dynamic script processor module can load and compile the data processing script corresponding to each target node in the target link configuration information, and generate a target instance object corresponding to each target node, that is, an assembler component corresponding to each target node. The configuration parsing module can parse the link configuration information to obtain the target execution order corresponding to each target node. The link generation module can generate a target link for processing the target data based on the assembler component corresponding to each target node and the target execution order, such as Figure 3 The target link in the example includes three target nodes, and the execution order is target node 1, target node 2, and target node 3, so that the target data can flow through target node 1, target node 2, and target node 3 in turn for data processing, and the final data processing result is returned to the caller, completing the entire data processing flow.

[0049] The technical solution of this embodiment pre-configures the link configuration information corresponding to each data processing identifier, that is, dynamically configures the data processing script corresponding to each node in the link for processing each type of business data and the execution order corresponding to each node, so that when processing the target data corresponding to the target data processing identifier, the data processing script corresponding to each target node in the target link configuration information can be loaded and compiled to generate a target instance object corresponding to each target node, and based on the target execution order of each target node and each target instance object, the target data is processed, so that the target link for processing the target data can be dynamically generated based on the target link configuration information, realizing dynamic configuration generation of the data processing link, greatly improving the flexibility of data processing, and thus meeting the personalized needs of users, and each target node in the target link performs data processing based on the data processing script, so that when business needs change, only the data processing script in the configuration information needs to be adjusted to take effect, without the need to rewrite the logic code and re-publish it online, thereby improving development efficiency.

[0050] Based on the above technical solution, S110 may include: obtaining the target data to be processed and the target data processing identifier corresponding to the target data sent by the caller through the call input interface. Figure 2 As shown, the caller may include but is not limited to: an external WEB network interface, a remote service interface or an internal call interface.

[0051] Specifically, the data processing process in this embodiment can be applied to complex multi-dimensional data processing in scenarios such as between WEB systems, between service systems, or between modules within the system, greatly improving the scope of application of the scenario. Moreover, the data format conversion can be completed based on the data processing script, avoiding the need to rewrite the logic code and repeat the release and online operation when the interface data structure changes. At this time, only the data processing script needs to be adjusted to take effect, reducing labor costs and improving development efficiency.

[0052] On the basis of the above technical solution, the method also includes: storing the target instance object corresponding to each target node in the memory, and canceling the release operation of the memory when the target data processing is completed.

[0053] Specifically, after generating the target instance object corresponding to each target node, each target instance object can be stored in the memory, and the memory is not released, so as to cache the generated target instance objects. Therefore, the next time the data is processed, the stored target instance object corresponding to the same node can be directly used, without the need to generate the instance object in real time each time the same node is assembled, thereby further improving the data processing efficiency.

[0054] Exemplarily, S130 may include: based on the pre-stored instance objects corresponding to each node, detecting whether there is a generated target instance object corresponding to each target node in the target link; if so, directly obtaining the generated target instance object corresponding to the target node; if not, loading and compiling the data processing script corresponding to the target node to generate the target instance object corresponding to the target node.

[0055] Specifically, for each target node in the target link, a match can be made among the instance objects corresponding to the pre-stored nodes to detect whether an instance object corresponding to the same node is stored. If so, it indicates that the data processing script corresponding to the target node has been loaded and compiled to generate the corresponding instance object. At this time, the instance object can be used as the target instance object corresponding to the target node. Therefore, when the same node is included in different links, the generated target instance object corresponding to the node can be directly used to perform the data processing logic of the node without having to reconstruct the target instance object, thereby further improving the data processing efficiency.

[0056] Embodiment 2

[0057] Figure 4 A flowchart of a data processing method provided for Embodiment 2 of the present invention. Based on the above embodiments, this embodiment further optimizes "processing target data based on target execution order and each target processing object", and the explanations of terms that are the same or corresponding to the above embodiments are not repeated here.

[0058] See also Figure 4 The data processing method provided in this embodiment specifically includes the following steps:

[0059] S410: Obtain target data to be processed and a target data processing identifier corresponding to the target data.

[0060] S420. Based on the pre-configured link configuration information corresponding to each data processing identifier and the target data processing identifier, determine the target link configuration information corresponding to the target link used to process the target data. The target link configuration information includes: a data processing script corresponding to each target node in the target link and a target execution order corresponding to each target node.

[0061] S430: Load and compile the data processing script corresponding to each target node to generate a target instance object corresponding to each target node.

[0062] S440: Encapsulate each target instance object and generate an assembleable target proxy object corresponding to each target instance object.

[0063] The target proxy object may refer to a proxy provided for the target instance object to control access to the target instance object, facilitate expansion of the logical functions implemented by the target instance object, and further improve scalability. Each target instance object corresponds to a target proxy object.

[0064] Specifically, dynamic proxy technology and aspect-oriented programming can be used to encapsulate each target instance object into an assembleable target proxy object, and the function can be aspected. Aspect-oriented programming can be a technology that dynamically and uniformly adds functions to functions without modifying the source code through pre-compilation and runtime dynamic proxy.

[0065] For example, Figure 2 In addition to the dynamic script processor, the assembler component in the data processing architecture can also include a dynamic assembly proxy constructor, so that each target instance object can be encapsulated by using the dynamic assembly proxy constructor to generate a corresponding target proxy object. For example, Figure 5 An example of the target node creation process is given. Figure 5 As shown, when each target node needs to be constructed, the assembler component first calls the dynamic assembly agent constructor, and the dynamic assembly agent constructor then calls the dynamic script processor. The dynamic script processor calls the configuration management module based on the target data processing identifier. The configuration management module returns the target link configuration information corresponding to the target data processing identifier, so that the dynamic script processor generates the corresponding target instance object based on the data processing script corresponding to the target node, and returns the target instance object to the dynamic assembly agent constructor. The dynamic assembly agent constructor encapsulates the target instance object and generates the corresponding target agent object, thereby completing the assembly construction process of a target node.

[0066] Exemplarily, after S440, it may also include: obtaining logic code for implementing non-data processing logic functions, and determining a first proxy object to be expanded from each target proxy object; weaving the logic code into the first proxy object to implement the data processing logic function and non-data processing logic function corresponding to the first proxy object by calling the expanded first proxy object.

[0067] Among them, non-data processing logic functions may refer to operations other than data processing operations, for example, operations other than data splitting, packaging, heterogeneity and reorganization, etc. For example, non-data processing logic functions may be functions for implementing monitoring and reporting, etc.

[0068] Specifically, when a third party needs to implement non-data processing logical functions during the data processing process, the logical code for implementing the non-data processing logical functions and the first proxy object for implementing the non-data processing logical functions in each target proxy object can be obtained based on the third-party needs. Therefore, without modifying the script, the logical code can be directly woven into the first proxy object to achieve functional expansion of the first proxy object, so that when the extended first proxy object is subsequently called, the data processing logical functions and non-data processing logical functions corresponding to the first proxy object can be simultaneously implemented, thereby improving the efficiency of functional expansion and further meeting the personalized needs of users.

[0069] S450. Based on the target execution order, call the data processing function in the target proxy object corresponding to each target node in turn, and process the target data in turn.

[0070] The data processing function may refer to a function within the target instance object corresponding to the target proxy object, which is used to execute the corresponding data processing logic. Specifically, based on the target execution order in the target link, the data processing function in the target proxy object corresponding to each target node may be called in sequence, so that the target data is sequentially processed at each target node in sequence, so that the final data processing result can be obtained at the last target node, and the final data processing result can be returned, completing the entire data processing process.

[0071] For example, Figure 6 An example of the target link creation process is given. Figure 6 As shown, in the construction phase, the link generation module can call the configuration parsing module, and the configuration parsing module can call the configuration management module, so that the configuration management model returns the target link configuration information corresponding to the acquired target data processing identifier, and the configuration parsing module can parse the target link configuration, obtain the target execution order of each target node, and return the target execution order to the link generation module, so that the link generation module can construct the specific data processing logic of the target link based on the target instance object and target execution order corresponding to each target node, thereby realizing the dynamic generation of the data processing link. In the data processing phase, the target data flows through each target node in the target link, and after processing the target nodes layer by layer, the final data processing result can be obtained.

[0072] The technical solution of this embodiment encapsulates each target instance object, generates an assembleable target proxy object corresponding to each target instance object, and uses each target proxy object to process the target data, so that the target proxy object can be used to more conveniently control access to the target instance object, facilitate expansion of the logical functions implemented by the target instance object, further improve scalability, and thus meet the personalized needs of users.

[0073] On the basis of the above technical solution, the method may further include: storing the target proxy object corresponding to each target node, so as to directly process the target data subsequently using the stored target proxy object.

[0074] Specifically, after generating the target proxy object corresponding to each target node, each target proxy object can be stored in the memory, and when the target data processing is completed, the memory release operation can be canceled, so that the generated target proxy objects can be cached, and then the next time the data is processed, the stored target proxy object corresponding to the same node can be directly used, without the need to generate instance objects and proxy objects in real time each time the same node is assembled, thereby further improving the data processing efficiency.

[0075] Exemplarily, after S420, it may also include: based on the pre-stored proxy objects corresponding to each node, detecting whether there are generated target proxy objects corresponding to each target node in the target link; if so, directly obtaining the generated target proxy object corresponding to the target node; if not, loading and compiling the data processing script corresponding to the target node to generate a target processing object corresponding to the target node, and encapsulating the target processing object to generate an assembleable target proxy object corresponding to the target processing object.

[0076] Specifically, for each target node in the target link, a match can be made among the pre-stored proxy objects corresponding to each node to detect whether a proxy object corresponding to the same node is stored. If so, it indicates that the data processing script corresponding to the target node has been loaded, compiled and packaged to generate a corresponding proxy object. At this time, the proxy object can be used as the target proxy object corresponding to the target node. Therefore, when the same node is included in different links, the generated target proxy object corresponding to the node can be directly used to perform the data processing logic of the node without having to reconstruct the target proxy object, thereby further improving the data processing efficiency.

[0077] The following is an embodiment of a data processing device provided by an embodiment of the present invention. The device and the data processing methods of the above embodiments belong to the same inventive concept. For details not described in detail in the embodiment of the data processing device, reference can be made to the embodiments of the above data processing methods.

[0078] Embodiment 3

[0079] Figure 7 This is a schematic diagram of the structure of a data processing device provided in Embodiment 3 of the present invention. This embodiment is applicable to the case where data acquired by a data interface is processed. The device may specifically include: a target data acquisition module 710, a target link configuration information determination module 720, a target instance object generation module 730 and a target data processing module 740.

[0080] The target data acquisition module 710 is used to obtain the target data to be processed and the target data processing identifier corresponding to the target data; the target link configuration information determination module 720 is used to determine the target link configuration information corresponding to the target link used to process the target data based on the pre-configured link configuration information corresponding to each data processing identifier and the target data processing identifier, the target link configuration information includes: the data processing script corresponding to each target node in the target link and the target execution order corresponding to each target node; the target instance object generation module 730 is used to load and compile the data processing script corresponding to each target node to generate a target instance object corresponding to each target node; the target data processing module 740 is used to process the target data based on the target execution order and each target instance object.

[0081] Optionally, the target data processing module 740 is specifically used to: based on the target execution order, sequentially call the data processing function in the target instance object corresponding to each target node to process the target data in sequence.

[0082] Optionally, the device further comprises:

[0083] The instance object storage module is used to store the target instance object corresponding to each target node in the memory, and cancel the memory release operation when the target data processing is completed.

[0084] Optionally, the target instance object generation module 730 is specifically configured to:

[0085] According to the pre-stored instance objects corresponding to each node, detect whether there is a generated target instance object corresponding to each target node in the target link; if so, directly obtain the generated target instance object corresponding to the target node; if not, load and compile the data processing script corresponding to the target node to generate the target instance object corresponding to the target node.

[0086] Optionally, the target data processing module 740 is further specifically configured to:

[0087] Each target instance object is encapsulated to generate an assembleable target proxy object corresponding to each target instance object; based on the target execution order, the data processing function in the target proxy object corresponding to each target node is called in sequence to process the target data in sequence.

[0088] Optionally, the device further comprises:

[0089] A logic code acquisition module, used to acquire logic codes for implementing non-data processing logic functions after generating an assembleable target proxy object corresponding to each target instance object, and determine a first proxy object to be expanded from each target proxy object;

[0090] The logic code weaving module is used to weave the logic code into the first proxy object, so as to implement the data processing logic function and non-data processing logic function corresponding to the first proxy object by calling the expanded first proxy object.

[0091] Optionally, the device further comprises:

[0092] The proxy object storage module is used to store the target proxy object corresponding to each target node, so that the stored target proxy object can be used to directly process the target data later.

[0093] Optionally, the target data acquisition module 710 is specifically configured to:

[0094] Obtain the target data to be processed and the target data processing identifier corresponding to the target data sent by the caller through calling the input end interface; wherein the caller includes: an external network interface, a remote service interface or an internal calling interface.

[0095] The data processing device provided in the embodiment of the present invention can execute the data processing method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects for executing the data processing method.

[0096] It is worth noting that in the embodiment of the above-mentioned data processing device, the various units and modules included are only divided according to functional logic, but are not limited to the above-mentioned division, as long as the corresponding functions can be achieved; in addition, the specific names of the functional units are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of the present invention.

[0097] Embodiment 4

[0098] Figure 8 A schematic diagram of the structure of an electronic device provided in Embodiment 4 of the present invention. Figure 8 A block diagram of an exemplary electronic device 12 suitable for use in implementing embodiments of the present invention is shown. Figure 8The electronic device 12 shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.

[0099] like Figure 8 As shown, the electronic device 12 is in the form of a general purpose computing device. The components of the electronic device 12 may include, but are not limited to: one or more processors or processing units 16, a system memory 28, and a bus 18 that connects various system components (including the system memory 28 and the processing unit 16).

[0100] Bus 18 represents one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, an accelerated graphics port, a processor or a local bus using any of a variety of bus architectures. By way of example, these architectures include, but are not limited to, an Industry Standard Architecture (ISA) bus, a Micro Channel Architecture (MAC) bus, an Enhanced ISA bus, a Video Electronics Standards Association (VESA) local bus, and a Peripheral Component Interconnect (PCI) bus.

[0101] The electronic device 12 typically includes a variety of computer system readable media. These media can be any available media that can be accessed by the electronic device 12, including volatile and non-volatile media, removable and non-removable media.

[0102] The system memory 28 may include computer system readable media in the form of volatile memory, such as random access memory (RAM) 30 and / or cache memory 32. The electronic device 12 may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, the storage system 34 may be used to read and write non-removable, non-volatile magnetic media ( Figure 8 not shown, usually called a "hard drive"). Although Figure 8 Not shown in the figure, a disk drive for reading and writing to a removable non-volatile disk (e.g., a "floppy disk"), and an optical disk drive for reading and writing to a removable non-volatile optical disk (e.g., a CD-ROM, a DVD-ROM, or other optical media) may be provided. In these cases, each drive may be connected to the bus 18 via one or more data medium interfaces. The system memory 28 may include at least one program product having a set (e.g., at least one) of program modules that are configured to perform the functions of the various embodiments of the present invention.

[0103] A program / utility 40 having a set (at least one) of program modules 42 may be stored, for example, in system memory 28, such program modules 42 including, but not limited to, an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment. Program modules 42 generally perform the functions and / or methods of the embodiments described herein.

[0104] The electronic device 12 may also communicate with one or more external devices 14 (e.g., keyboards, pointing devices, displays 24, etc.), may communicate with one or more devices that enable a user to interact with the electronic device 12, and / or may communicate with any device that enables the electronic device 12 to communicate with one or more other computing devices (e.g., network cards, modems, etc.). Such communication may be performed via an input / output (I / O) interface 22. Furthermore, the electronic device 12 may also communicate with one or more networks (e.g., local area networks (LANs), wide area networks (WANs), and / or public networks, such as the Internet) via a network adapter 20. As shown, the network adapter 20 communicates with other modules of the electronic device 12 via a bus 18. It should be understood that, although not shown in the figure, other hardware and / or software modules may be used in conjunction with the electronic device 12, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0105] The processing unit 16 executes various functional applications and data processing by running the program stored in the system memory 28, for example, implementing a data processing method provided by the embodiment of the present invention, the method comprising:

[0106] Obtaining target data to be processed and a target data processing identifier corresponding to the target data;

[0107] Based on the pre-configured link configuration information corresponding to each data processing identifier and the target data processing identifier, determining the target link configuration information corresponding to the target link for processing the target data, the target link configuration information including: a data processing script corresponding to each target node in the target link and a target execution order corresponding to each target node;

[0108] Load and compile the data processing script corresponding to each target node to generate the target instance object corresponding to each target node;

[0109] The target data is processed based on the target execution order and each target instance object.

[0110] Of course, those skilled in the art can understand that the processor can also implement the technical solution of the data processing method provided by any embodiment of the present invention.

[0111] Embodiment 5

[0112] This embodiment provides a computer-readable storage medium, on which a computer program is stored. When the program is executed by a processor, the steps of the data processing method provided in any embodiment of the present invention are implemented. The method includes:

[0113] Obtaining target data to be processed and a target data processing identifier corresponding to the target data;

[0114] Based on the pre-configured link configuration information corresponding to each data processing identifier and the target data processing identifier, determining the target link configuration information corresponding to the target link for processing the target data, the target link configuration information including: a data processing script corresponding to each target node in the target link and a target execution order corresponding to each target node;

[0115] Load and compile the data processing script corresponding to each target node to generate the target instance object corresponding to each target node;

[0116] The target data is processed based on the target execution order and each target instance object.

[0117] The computer storage medium of the embodiment of the present invention can adopt any combination of one or more computer-readable media. The computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. The computer-readable storage medium can be, for example, but not limited to: an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples (non-exhaustive list) of computer-readable storage media include: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this document, a computer-readable storage medium can be any tangible medium containing or storing a program, which can be used by an instruction execution system, device or device or used in combination with it.

[0118] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, which carry computer-readable program code. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than a computer-readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0119] The program code embodied on the computer readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

[0120] Computer program code for performing the operations of the present invention may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0121] It should be understood by those skilled in the art that the modules or steps of the present invention described above can be implemented by a general-purpose computing device, they can be concentrated on a single computing device, or distributed on a network composed of multiple computing devices, optionally, they can be implemented by a program code executable by a computer device, so that they can be stored in a storage device and executed by the computing device, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. Thus, the present invention is not limited to any specific combination of hardware and software.

[0122] Note that the above are only preferred embodiments of the present invention and the technical principles used. Those skilled in the art will understand that the present invention is not limited to the specific embodiments described herein, and that various obvious changes, readjustments and substitutions can be made by those skilled in the art without departing from the scope of protection of the present invention. Therefore, although the present invention has been described in more detail through the above embodiments, the present invention is not limited to the above embodiments, and may include more other equivalent embodiments without departing from the concept of the present invention, and the scope of the present invention is determined by the scope of the appended claims.

Claims

1. A data processing method, characterized in that: include: Acquire target data to be processed and a target data processing identifier corresponding to the target data; Based on the pre-configured link configuration information corresponding to each data processing identifier and the target data processing identifier, determining the target link configuration information corresponding to the target link for processing the target data, the target link configuration information including: a data processing script corresponding to each target node in the target link and a target execution order corresponding to each target node; Load and compile the data processing script corresponding to each target node to generate a target instance object corresponding to each target node; Based on the target execution order and each of the target instance objects, processing the target data; The target data is processed based on the target execution order and each target instance object, including: Encapsulating each of the target instance objects to generate an assembleable target proxy object corresponding to each of the target instance objects; Based on the target execution order, the data processing function in the target proxy object corresponding to each target node is called in sequence to process the target data in sequence; After generating the assembleable target proxy object corresponding to each target instance object, the method further includes: Acquire logic code for implementing non-data processing logic functions, and determine a first proxy object to be expanded from each of the target proxy objects; Weaving the logic code into the first proxy object, so as to implement the data processing logic function and the non-data processing logic function corresponding to the first proxy object by calling the expanded first proxy object; The non-data processing logic functions are operations other than data splitting, packaging, heterogeneity and reorganization operations.

2. The method according to claim 1, characterized in that Based on the target execution order and each of the target instance objects, the target data is processed, including: Based on the target execution order, the data processing function in the target instance object corresponding to each target node is called in sequence to process the target data in sequence.

3. The method according to claim 1, characterized in that The method further comprises: The target instance object corresponding to each target node is stored in the memory, and when the target data processing is completed, the release operation of the memory is canceled.

4. The method according to claim 1, characterized in that Loading and compiling the data processing script corresponding to each target node to generate a target instance object corresponding to each target node, including: According to the pre-stored instance object corresponding to each node, detecting whether there is a generated target instance object corresponding to each target node in the target link; If yes, directly obtain the generated target instance object corresponding to the target node; If not, the data processing script corresponding to the target node is loaded and compiled to generate a target instance object corresponding to the target node.

5. The method according to claim 1, characterized in that The method further comprises: The target proxy object corresponding to each target node is stored so that the target data can be directly processed subsequently using the stored target proxy object.

6. The method according to claim 1, characterized in that Acquiring target data to be processed and a target data processing identifier corresponding to the target data, including: Acquire the target data to be processed and the target data processing identifier corresponding to the target data sent by the caller through calling the input end interface; The caller includes: an external network interface, a remote service interface or an internal call interface.

7. A data processing device, characterized in that: include: A target data acquisition module, used to acquire target data to be processed and a target data processing identifier corresponding to the target data; A target link configuration information determination module is used to determine the target link configuration information corresponding to the target link for processing the target data based on the pre-configured link configuration information corresponding to each data processing identifier and the target data processing identifier, wherein the target link configuration information includes: a data processing script corresponding to each target node in the target link and a target execution order corresponding to each target node; A target instance object generation module, used to load and compile the data processing script corresponding to each of the target nodes, and generate a target instance object corresponding to each of the target nodes; A target data processing module, used for processing the target data based on the target execution order and each of the target instance objects; The target data processing module is further specifically used for: Encapsulate each target instance object and generate an assembleable target proxy object corresponding to each target instance object; Based on the target execution order, the data processing function in the target proxy object corresponding to each target node is called in sequence to process the target data in sequence; The device also includes: A logic code acquisition module, used to acquire logic codes for implementing non-data processing logic functions after generating an assembleable target proxy object corresponding to each target instance object, and determine a first proxy object to be expanded from each target proxy object; A logic code weaving module, used for weaving the logic code into the first proxy object, so as to implement the data processing logic function and the non-data processing logic function corresponding to the first proxy object by calling the expanded first proxy object; The non-data processing logic functions are operations other than data splitting, packaging, heterogeneity and reorganization operations.

8. An electronic device, characterized in that: The electronic device comprises: one or more processors; A memory for storing one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the data processing method according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the data processing method as described in any one of claims 1 to 6 is implemented.

Images