Configurable process data processing method, device and equipment

Abstract

The present invention relates to the field of data processing technology, and its embodiments provide a configurable process data processing method, apparatus, and device. The configurable process data processing method includes: determining associated fields based on the industry to which the data belongs and the corresponding business; extracting messages from the data stream that responds to the business link operation based on the associated fields; mapping the messages to a first message format with the associated fields as the content or a second message format with the associated fields as the row key based on the type of the extracted messages; completing the interactive operation between the business link operation and the database based on the mapped messages to achieve a response to the business link operation. The embodiments provided by the present invention can solve the timeliness of multi-stream association and improve the efficiency of multi-table association.

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and in particular to a configurable process data processing method, a configurable process data processing device, a configurable process data processing equipment and corresponding storage medium. Background Art

[0002] Currently, HBase-based massive data queries have found application in other industries. For example, e-commerce platforms use HBase to store transaction data and provide query support. However, these processes are generally simple, with low correlation and coupling between data, making HBase suitable for single-point queries. The complexity of banking services dictates that a transaction must go through multiple stages from start to finish, each involving multiple pieces of related data. This results in a high level of complexity in banking transaction data, often combining massive amounts of detailed data with streamlined data. The HBase approach used in other industries cannot meet the complex data processing requirements of banks.

[0003] Currently, transaction data processing in the banking industry uses different processing architectures tailored to the volume of system data. The traditional approach is to store data in relational databases like Oracle or MySQL and provide highly efficient query services. Furthermore, with the rapid growth of big data services in recent years, traditional relational databases are no longer able to handle the massive amounts of data. Therefore, the mainstream approach to big data processing is to store data in non-relational databases like HBase. However, due to HBase's limitations, it cannot support scenarios with complex query logic, such as those based on multi-table joins and multiple query conditions, resulting in low efficiency. Therefore, the shortcomings of these two mainstream processing approaches are obvious: relational databases can support complex query logic but cannot handle massive amounts of data; non-relational databases can store massive amounts of data but cannot support flexible query logic. Summary of the Invention

[0004] The purpose of the embodiments of the present invention is to provide a configurable process data processing method, apparatus and device.

[0005] In order to achieve the above-mentioned purpose, the first aspect of the present invention provides a configurable process data processing method, which includes: determining the associated fields according to the industry to which the data belongs and the corresponding business; extracting messages from the data stream in response to the business link operation according to the associated fields; mapping the messages into a first message format with the associated fields as content or a second message format with the associated fields as row keys according to the type of the extracted messages; completing the interaction between the business link operation and the database according to the mapped messages to achieve a response to the business link operation.

[0006] Preferably, before determining the associated fields based on the industry to which the data belongs and the corresponding business, the method also includes: dividing the business into multiple business processes and defining a business process definition table for the business process; dividing each business process into multiple process links and defining a process link definition table for the process link.

[0007] Preferably, extracting messages from the data stream in response to business link operations based on the associated fields includes: storing the data stream in response to business link operations in a hybrid cache composed of Guava and Redis; determining whether the associated fields of the messages extracted from the input data stream exist in the hybrid cache; if not, compressing the data and retaining it in the hybrid cache.

[0008] Preferably, the method further comprises: classifying the result data into a plurality of types, and storing the result data into corresponding databases according to the order of magnitude of the result data of different types.

[0009] Preferably, the first message format with the associated field as content includes a row key, and the row key is determined by the following steps: obtaining predefined row key elements and the order of the row key elements, and combining the row key elements extracted from the data in the order of the row key elements to obtain the row key.

[0010] Preferably, the types of messages include request messages, response messages and front-end messages; the interactive operations between the business link operations and the database are completed according to the mapped messages to achieve the response to the business link operations, including: obtaining the request message according to the business query conditions; extracting the associated fields in the request message, and querying the response message and the front-end message; and achieving the response to the business link operations based on the obtained request message, response message and front-end message.

[0011] In a second aspect of the present invention, a configurable process data processing device is provided, the device comprising:

[0012] An associated field determination module is used to determine associated fields based on the industry and corresponding business to which the data belongs; a message extraction module is used to extract messages from the data stream that responds to business link operations based on the associated fields; a format mapping module is used to map the messages into a first message format with the associated fields as content or a second message format with the associated fields as row keys based on the type of the extracted messages; and a message execution module is used to complete the interactive operations between the business link operations and the database based on the mapped messages to achieve a response to the business link operations.

[0013] Preferably, the associated fields are determined according to the industry to which the data belongs and the corresponding business, including: dividing the business into multiple business processes and defining a business process definition table for the business processes; dividing each business process into multiple process links and defining a process link definition table for the process links.

[0014] Preferably, extracting messages from the data stream in response to business link operations based on the associated fields includes: storing the data stream in response to business link operations in a hybrid cache composed of Guava and Redis; determining whether the associated fields of the messages extracted from the input data stream exist in the hybrid cache; if not, compressing the data and retaining it in the hybrid cache.

[0015] Preferably, the device further comprises: a classification storage module for classifying the result data into a plurality of types, and storing the result data into a corresponding database according to the order of magnitude of the result data of different types.

[0016] Preferably, the first message format with the associated field as content includes a row key, and the row key is determined by the following steps: obtaining predefined row key elements and the order of the row key elements, and combining the row key elements extracted from the data in the order of the row key elements to obtain the row key.

[0017] Preferably, the types of messages include request messages, response messages and front-end messages; the interactive operations between the business link operations and the database are completed according to the mapped messages to achieve the response to the business link operations, including: obtaining the request message according to the business query conditions; extracting the associated fields in the request message, and querying the response message and the front-end message; and achieving the response to the business link operations based on the obtained request message, response message and front-end message.

[0018] In a third aspect of the present invention, a configurable process data processing device is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the aforementioned configurable process data processing method when executing the computer program.

[0019] In a fourth aspect of the present invention, a computer-readable storage medium is provided, wherein instructions are stored in the storage medium. When the instructions are executed on a computer, the computer is caused to execute the aforementioned configurable process data processing method.

[0020] In a fifth aspect of the present invention, a computer program product is provided, comprising a computer program, which implements the aforementioned configurable process data processing method when executed by a processor.

[0021] The above technical solution has the following beneficial effects:

[0022] A set of bank process model definition methods enables flexible definition of different processes for diverse business scenarios. Based on the characteristics of bank transaction data, a universal definition of business elements for process links is proposed. This addresses the timeliness issues of multi-flow associations in complex scenarios, as well as the excessive space usage associated with Redis alone. HBase query design addresses HBase's inability to efficiently join multiple tables.

[0023] Other features and advantages of the embodiments of the present invention will be described in detail in the subsequent detailed description. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] The accompanying drawings are used to provide a further understanding of the embodiments of the present invention and constitute a part of the specification. Together with the following detailed description, they are used to explain the embodiments of the present invention, but do not constitute a limitation of the embodiments of the present invention. In the accompanying drawings:

[0025] Figure 1 The following schematically illustrates the steps of a configurable process data processing method according to an embodiment of the present application;

[0026] Figure 2 The following schematically shows the steps involved in the door-to-door payment collection process according to an embodiment of the present application;

[0027] Figure 3 The following schematically shows the implementation steps of the cache processing process according to an embodiment of the present application;

[0028] Figure 4 Schematically shows a schematic diagram of the step-by-step implementation of the configurable process data processing method according to an embodiment of the present application;

[0029] Figure 5 The schematic diagram shows the structure of a configurable process data processing device according to an embodiment of the present application. DETAILED DESCRIPTION

[0030] The following describes the specific implementation of the embodiment of the present invention in detail with reference to the accompanying drawings. It should be understood that the specific implementation described herein is only used to illustrate and explain the embodiment of the present invention and is not used to limit the embodiment of the present invention.

[0031] The acquisition, storage, use, and processing of data in this application's technical solution comply with relevant national laws and regulations.

[0032] Figure 1 The following schematically shows the steps of a configurable process data processing method according to an embodiment of the present application. Figure 1As shown, in one embodiment of the present application, a configurable process data processing method includes:

[0033] S01. Determine associated fields based on the data's industry and corresponding business. Data is generated in different business scenarios and therefore possesses characteristics specific to those scenarios. To distinguish data, each piece of data should have a distinguishing identifier. This distinguishing identifier can be used as the associated field in this embodiment, or it can be processed and used as the associated field in this embodiment. For example, using bank data as an example, the associated fields determined could be: transaction code + transaction type + tracking number.

[0034] S02. extracting a message from a data stream in response to a business link operation according to the associated field;

[0035] The data streams here include, but are not limited to, Flink data streams. As a mature open-source stream processing framework, Flink can execute both batch and stream processing programs. The following implementation will still use Flink data streams as an example. Flink extracts the associated fields defined in the previous step from the incoming data stream: transaction code, transaction type, and tracking number. Based on these associated fields, it extracts messages and performs further processing on the extracted messages.

[0036] S03. Mapping the message into a first message format with the associated field as content or a second message format with the associated field as a row key according to the type of the extracted message;

[0037] Different types of messages are stored in three separate HBase database tables. Directly querying these three tables using a join method results in very low query efficiency. To improve query efficiency, the message data format needs to be redesigned to provide a more optimized index. Two message formats are provided: one uses previously defined join fields as row keys and related information as content, which can be used in response messages and front-end messages; the other uses a custom row key, with related information and join fields as row keys, which can be used in request messages. Row keys are indexes in HBase.

[0038] S04. Complete the interactive operation between the business link operation and the database according to the mapped message to achieve a response to the business link operation.

[0039] The message mapped in the previous step has better query efficiency. The interactive operation between the business link operation and the database mainly depends on the aforementioned response message, front-end message and request message. The business link is restored or completed through the above messages.

[0040] The above implementation approach, based on the characteristics of banking services, has designed a correlation field consisting of "transaction code + transaction type + tracking number." Based on this correlation field, a set of universal standards has been developed to support flexible definition of processes and steps in different scenarios, addressing the timeliness of multi-flow correlation in complex scenarios. HBase query design also addresses HBase's inability to efficiently perform multi-table correlations.

[0041] In some embodiments provided by the present invention, before determining the associated fields based on the industry to which the data belongs and the corresponding business, the method further includes: dividing the business into multiple business processes and defining a business process definition table for the business process; dividing each business process into multiple process links and defining a process link definition table for the process link. This embodiment provides a set of configurable business model definition methods for quickly and flexibly defining business processes and information about each link in the process as the basis for Flink data processing. The business model definition mainly includes two parts: business process definition and process link definition. The main information is as follows:

[0042] Business process definition table

[0043] Process definition ID Process Name Association ID Creation time Founder

[0044] Process step definition table

[0045] Session ID Link Name Unique identifier Transaction Code Menu Number Association Process

[0046] Figure 2 The schematic diagram shows the steps of the door-to-door payment collection business process according to the embodiment of the present application. Figure 2 For example, the "Door-to-Door Collection" business process includes five steps, including "Generate Door-to-Door Collection Task," as shown in the figure. Each step has a defined "Process Step Definition Table," where "Unique Identifiers" are used to connect the steps together to form a complete process. This model definition approach, based on a common set of standards, allows businesses to flexibly define processes and steps for different scenarios.

[0047] Figure 3 The following schematically shows the implementation steps of the cache processing process according to the embodiment of the present application. Figure 3As shown, in this embodiment, extracting messages from a data stream responding to a business process operation based on the associated fields includes: storing the data stream responding to the business process operation in a hybrid cache composed of Guava and Redis; determining whether the associated fields of the message extracted from the input data stream exist in the hybrid cache; and if not, compressing the data and retaining it in the hybrid cache. This embodiment specifically designs the cache structure based on the characteristics of the data structure. Currently, most banking systems are separated in terms of front-end and back-end. Therefore, a business process operation often corresponds to three types of data: front-end messages containing operation interface information such as menus, request messages received by the server, and response messages after the server completes processing logic. These three types of message data correspond to the three data streams in Flink. Due to the large amount of data and the out-of-order arrival of data, for example, three data streams may only partially arrive and need to wait for other data streams. This requires caching the already arrived data. A reasonable caching method determines the level of system throughput.

[0048] This implementation builds on existing technologies by adding Guava as a secondary cache. The parameters in Redis are dynamically loaded into Guava at regular intervals and stored in memory, further reducing the frequency of interaction between Flink and Redis and improving processing efficiency. The hybrid cache uses Redis as the primary cache and Guava as the secondary cache. The data flow processing process in the hybrid cache is as follows:

[0049] This implementation uses Guava and Redis as a data cache. Flink extracts the associated fields defined in the data streams in the aforementioned process steps from the incoming data stream: transaction code, transaction type, and tracking number. These fields are combined as query keys. If a get operation fails to retrieve the data from Guava or Redis, it indicates that the data has arrived for the first time and needs to be compressed and written to the cache. This allows subsequent incoming data streams to access the previously arrived data, and compression also reduces cache space usage. This step provides cache support for efficient Flink processing in the next step.

[0050] In one embodiment of the present invention, the method further includes: classifying the data in the data stream of the business link operation into multiple types based on the business scenario, wherein the data volumes corresponding to different types vary in magnitude, and storing the data in corresponding databases according to the types. This embodiment uses the Flink engine, combined with business model definitions, to perform real-time processing of transaction message information. Different business scenarios generate result data of varying magnitudes. For example, data can be classified into business process categories and process indicator categories. Business process data is detailed data with a large volume, while process indicator data is aggregated results with a relatively small volume. These two data types are written to different databases for storage, preparing for subsequent differentiated query scenarios. For example, data can also be classified based on security requirements into sensitive data, important data, and general data, and different types of data can be written to different databases for storage, thereby improving data security. For example, data can be classified into hot data, warm data, and cold data based on frequency of use to improve data reading efficiency. The Flink data processing steps provided in this embodiment implement categorized data storage.

[0051] In one embodiment provided by the present invention, the first message format with the associated field as content includes a row key, and the row key is determined using the following steps: obtaining predefined row key elements and the order of the row key elements, and combining the row key elements extracted in the order of the row key elements to obtain the row key.

[0052] Although HBase can provide efficient row key queries, different business data elements vary, and query scenarios and query conditions also vary, so the composition of row keys is not fixed. This implementation defines row key rules based on the characteristics of banking business data. For example, each request message data has a transaction code, and configurable parameters are set based on this to define row key rules. Based on the set rules, the required row key element information is extracted from the business data and used to match the row key of the request message table structure. For example, the row key elements and the order of the row key elements are defined in the following table.

[0053] Transaction Code Row key elements order A0821TX01 Card_no 1 A0821TX01 App_id 2 A0821TX02 Country_code 1 A0821TX02 City_code 2

[0054] From the table above, we can see that the row key elements for transaction code A0821TX01 include: Card_no, sequence 1; App_id, sequence 2. The assembled row key is: the value corresponding to Card_no + App_id. This data is used to match the row key field in the request message table.

[0055] In one embodiment provided by the present invention, the types of messages include request messages, response messages and front-end messages; the interactive operations between the business link operations and the database are completed according to the mapped messages to achieve the response to the business link operations, including: obtaining the request message according to the business query conditions; extracting the associated fields in the request message, and querying the response message and the front-end message; and achieving the response to the business link operations based on the obtained request message, response message and front-end message.

[0056] As previously described, the message format includes either the first message format or the second message format. The associated fields (transaction code + transaction type + tracking number) in the first message format serve as the content, and their row keys are determined as described in the previous embodiment. The associated fields (transaction code + transaction type + tracking number) in the second message format serve as the row key, which contains relevant information. The formats of the request message, response message, and front-end message obtained through the above methods are shown below, respectively.

[0057] Request message

[0058] Row Key Interface information 1 Interface information 2 Transaction Code Transaction Type Tracking number

[0059] Response message

[0060] Transaction Code Transaction Type Tracking number Business Information Business Information Business Information

[0061] Front-end message

[0062] Transaction Code Transaction Type Tracking number Menu Information 1 Menu Information 2

[0063] Because HBase's multi-table joins are inefficient, after finding the request message based on the business query criteria, the associated fields in the request message's content (transaction code + transaction type + tracking number) are combined into a row key. This is then used to query the response message and the front-end message. This yields three messages, restoring a single business process. This highly flexible implementation allows for dynamic configuration based on different transaction codes, allowing for real-time processing of business data.

[0064] Figure 4 The following schematically shows a step-by-step implementation diagram of a configurable process data processing method according to an embodiment of the present application. Figure 4 As shown, the implementation method includes: 1. business model definition; 2. cache design; 3. FINK data processing; 4. row key and query design; and 5. user-initiated query and other steps.

[0065] Based on the same inventive concept, an embodiment of the present invention further provides a configurable process data processing device. Figure 5 The following schematically shows a structural diagram of a configurable process data processing device according to an embodiment of the present application. Figure 5 As shown, the device includes:

[0066] An associated field determination module is used to determine associated fields based on the industry and corresponding business to which the data belongs; a message extraction module is used to extract messages from the data stream that responds to business link operations based on the associated fields; a format mapping module is used to map the messages into a first message format with the associated fields as content or a second message format with the associated fields as row keys based on the type of the extracted messages; and a message execution module is used to complete the interactive operations between the business link operations and the database based on the mapped messages to achieve a response to the business link operations.

[0067] In an optional implementation provided by the present invention, the associated fields are determined based on the industry to which the data belongs and the corresponding business, including: dividing the business into multiple business processes and defining a business process definition table for the business process; dividing each business process into multiple process links and defining a process link definition table for the process link.

[0068] In an optional embodiment provided by the present invention, a message is extracted from a data stream in response to a business link operation according to the associated field, including: storing the data stream in response to the business link operation in a hybrid cache composed of Guava and Redis; determining whether the associated field of the message extracted from the stored data stream exists in the hybrid cache; if not, compressing the data and retaining it in the hybrid cache.

[0069] In an optional embodiment provided by the present invention, the device also includes: a classification storage module, which is used to divide the data in the data flow of the business link operation into multiple types according to the business scenario, and the data volume corresponding to different types has different orders of magnitude; according to different types, the data is stored in the corresponding database according to the type.

[0070] In an optional embodiment provided by the present invention, the first message format with the associated field as content includes a row key, and the row key is determined by the following steps: obtaining predefined row key elements and the order of the row key elements, and combining the row key elements extracted in the order of the row key elements to obtain the row key.

[0071] In an optional implementation manner provided by the present invention, the types of messages include request messages, response messages and front-end messages; the interactive operations between the business link operations and the database are completed according to the mapped messages to achieve the response to the business link operations, including: obtaining the request message according to the business query conditions; extracting the associated fields in the request message, and querying the response message and the front-end message; and achieving the response to the business link operations based on the obtained request message, response message and front-end message.

[0072] The specific definition of each functional module in the above-mentioned configurable process data processing device can be found in the definition of the configurable process data processing method above, and will not be repeated here. Each module in the above-mentioned device can be implemented in whole or in part by software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory in the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules. The configurable process data processing device includes a processor and a memory, and the above-mentioned associated field determination module, etc. are all stored in the memory as program units, and the processor executes the above-mentioned program modules stored in the memory to implement the corresponding functions.

[0073] Those skilled in the art will understand that Figure 5 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0074] An embodiment of the present application provides a device, which includes a processor, a memory, and a program stored in the memory and executable on the processor. When the processor executes the program, configurable process data processing steps are implemented.

[0075] The present application also provides a computer program product, which, when executed on a data processing device, is suitable for executing a program that initializes configurable process data processing method steps.

[0076] Those skilled in the art will appreciate that the embodiments of the present application can be provided as methods, systems, or computer program products. Therefore, the present application can adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application can adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0077] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the steps in the process. Figure 1a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0078] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0079] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0080] In a typical configuration, a computing device includes one or more processors (CPUs), input / output interfaces, network interfaces, and memory.

[0081] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0082] Computer-readable media includes permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology for information storage. Information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable media does not include transitory computer-readable media such as modulated data signals and carrier waves.

[0083] It should also be noted that the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, commodity, or apparatus that includes a series of elements includes not only those elements but also other elements not explicitly listed, or includes elements inherent to such process, method, commodity, or apparatus. In the absence of further limitations, an element defined by the phrase "comprises a ..." does not exclude the presence of other identical elements in the process, method, commodity, or apparatus that includes the element.

[0084] The above are merely embodiments of the present application and are not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present application should all be included within the scope of the claims of the present application.

Claims

1. A configurable process data processing method, characterized in that: The method comprises: Determine the associated fields based on the industry and corresponding business of the data; Extracting a message from a data stream responsive to a business link operation according to the associated field; Mapping the message into a first message format with the associated field as content or a second message format with the associated field as a row key according to the type of the extracted message; Complete the interactive operation between the business link operation and the database according to the mapped message to achieve the response to the business link operation; The types of messages include request messages, response messages and front-end messages; The interactive operation between the business link operation and the database is completed according to the mapped message to achieve the response to the business link operation, including: Obtain request message according to business query conditions; Extract the associated fields in the request message and query the response message and the front-end message; The response to the business link operation is realized based on the obtained request message, response message and front-end message.

2. The method according to claim 1, characterized in that Before determining the associated fields based on the industry and corresponding business of the data, the method further includes: Divide the business into multiple business processes and define a business process definition table for the business processes; Each business process is divided into multiple process links, and a process link definition table is defined for the process links.

3. The method according to claim 1, characterized in that Extracting a message from a data stream in response to a business link operation according to the associated field includes: Storing the data stream in response to the business link operation in a hybrid cache composed of Guava and Redis; determining whether a correlation field of data extracted from the stored data stream exists in the hybrid cache; If not, the data is compressed and retained in the hybrid cache.

4. The method according to claim 1, wherein The method further comprises: The data in the data stream of the business link operation is divided into multiple types according to the business scenario, and the data volume corresponding to different types is of different magnitudes; According to different types, data is stored in corresponding databases.

5. The method according to claim 1, characterized in that The first message format having the associated field as content includes a row key, and the row key is determined using the following steps: Obtaining predefined row key elements and the order of the row key elements; The row key is obtained by extracting the row key elements in the order of the row key elements and combining them.

6. A configurable process data processing device, characterized in that: The device comprises: The associated field determination module is used to determine the associated fields based on the industry and corresponding business of the data; A message extraction module, configured to extract a message from a data stream responsive to a business link operation according to the associated field; a format mapping module, configured to map the extracted message into a first message format having the associated field as content or a second message format having the associated field as a row key according to the type of the extracted message; and A message execution module, used to complete the interactive operation between the business link operation and the database according to the mapped message, so as to realize the response to the business link operation; The types of the messages include request messages, response messages and front-end messages; The interactive operation between the business link operation and the database is completed according to the mapped message to achieve the response to the business link operation, including: Obtain request message according to business query conditions; Extract the associated fields in the request message and query the response message and the front-end message; The response to the business link operation is realized based on the obtained request message, response message and front-end message.

7. The device according to claim 6, characterized in that The associated field determination module is further configured to: Divide the business into multiple business processes and define a business process definition table for the business processes; Each business process is divided into multiple process links, and a process link definition table is defined for the process links.

8. The device according to claim 6, characterized in that Extracting a message from a data stream in response to a business link operation according to the associated field includes: Storing the data stream in response to the business link operation in a hybrid cache composed of Guava and Redis; determining whether a correlation field of data extracted from the stored data stream exists in the hybrid cache; If not, the data is compressed and retained in the hybrid cache.

9. The device according to claim 6, characterized in that The device further comprises: The classification storage module is used to classify the data in the data flow of the business link operation into multiple types according to the business scenario, and the data volume corresponding to different types has different orders of magnitude; according to different types, the data is stored in the corresponding database according to the type.

10. The device according to claim 6, characterized in that The first message format having the associated field as content includes a row key, and the row key is determined using the following steps: Obtaining predefined row key elements and the order of the row key elements; The row key is obtained by combining the row key elements extracted in the order of the row key elements.

11. A configurable process data processing device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the configurable process data processing method according to any one of claims 1 to 5 is implemented.

12. A computer-readable storage medium, characterized in that The storage medium stores instructions, which, when executed on a computer, enable the computer to execute the configurable process data processing method according to any one of claims 1 to 5. 13 . A computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the computer program implements the configurable process data processing method according to claim 1 .

Images