A service data processing system, method, storage medium and device

Abstract

The application relates to a business data processing system, method, storage medium and equipment. The system comprises a business operation layer and an object aggregation layer which are independent of each other, and the business operation layer and the object aggregation layer transmit data in an asynchronous decoupling mode. The business operation layer performs a persistent operation on original business data to form persistent business data, and stores the persistent business data in a database. The object aggregation layer processes data of multiple business operation scenes according to a user business request to form aggregation data, and sends the aggregation data as a first response result to a user terminal. The data of the multiple business operation scenes comprises the persistent business data. The application can more quickly and effectively respond to business logic, can support efficient business application in complex, changeable and extended scenes, reduce the bloated, uncontrollable and unsustainable of a lightweight architecture system after continuous iteration, and maintain the lightweight architecture and flexibility of the business system.

Description

Technical Field

[0001] This invention relates to the field of computer data processing technology, and in particular to a business data processing system, method, storage medium, and device. Background Technology

[0002] Typically, when a user accesses data, the system retrieves, processes, or combines the business data through different methods before presenting it to the user. This involves real-time data processing. When the access volume reaches the system's maximum processing capacity, it can lead to request blocking or system crashes. In other words, as the software system's data storage and retrieval become increasingly large and complex, maintaining a lightweight architecture and high flexibility in scenarios involving high availability, high concurrency, and data mining and search still faces numerous bottlenecks.

[0003] 1. Under high traffic and high load conditions, the system response time becomes longer, user requests become blocked, and the system may even crash.

[0004] 2. The use of system data requires real-time data combination and processing through different database tables or different services, resulting in repeated processing and performance pressure on the system.

[0005] 3. Over time, the system has accumulated and user needs have evolved, resulting in bloated and highly coupled data and business logic, which can cause a domino effect. Summary of the Invention

[0006] The technical problem to be solved by the present invention is to provide a business data processing system, method, storage medium and device.

[0007] The technical solution adopted by the present invention to solve its technical problem is: a business data processing system, including a business operation layer and an object aggregation layer that are independent of each other, wherein the business operation layer and the object aggregation layer transmit data in an asynchronous decoupling manner;

[0008] The business operation layer is used to perform persistence operations on the original business data to form persistent business data, and store the persistent business data in the database; wherein, the original business data comes from data in user business requests and / or basic application data of the system, and the basic application data includes log data;

[0009] The object aggregation layer is used to process data from multiple business operation scenarios to form aggregated data according to the user's business request, and send the aggregated data as a first response result to the user terminal; wherein, the data from the multiple business operation scenarios includes the persistent business data.

[0010] Furthermore, in the business data processing system described in this invention, the business operation layer is also used to publish the persistent business data to the scene center of the object aggregation layer;

[0011] The scenario center distinguishes the corresponding business operation scenarios based on the attributes of the persistent business data. The object aggregation layer aggregates data from multiple business operation scenarios into aggregated data according to the preset business operation scenario logic to respond to the user's business request.

[0012] Furthermore, in the business data processing system described in this invention, the business operation layer publishes the persistent business data to the scene center of the object aggregation layer via a message queue; and / or, the business operation layer publishes the persistent business data to the scene center of the object aggregation layer via task scheduling.

[0013] Furthermore, in the business data processing system described in this invention, the business operation layer is also used to initiate an aggregated data change request to the object aggregation layer in an asynchronous and decoupled manner; the object aggregation layer updates the corresponding aggregated data according to the received aggregated data change request.

[0014] Furthermore, in the business data processing system described in this invention, the business operation layer is used to receive user requests, which include user business requests and single data access requests.

[0015] In response to the single data access request, the service operation layer sends the data of the service operation layer as a second response result to the user terminal.

[0016] In response to the user service request, after receiving the user service request, the service operation layer will send the corresponding response result back to the user terminal to inform the user that the system has received the request.

[0017] Furthermore, in the business data processing system described in this invention, the data for the various business operation scenarios also includes at least one of search data and cached data. The search data is data that quickly retrieves content by querying search terms in the form of binding search terms with content data. The cached data is hot data.

[0018] In addition, the present invention also provides a business data processing method, applied in an object aggregation layer within a business data processing system as described above, the method comprising:

[0019] In response to a user's business request, data from multiple business operation scenarios are processed according to a preset business operation scenario logic to form aggregated data, and the aggregated data is sent to the user terminal as the first response result.

[0020] The data in the various business operation scenarios includes persistent business data stored in the database after the business operation layer performs persistence operations on the original business data. The original business data comes from the data in the user's business request and / or the system's basic application data.

[0021] Furthermore, in the business data processing method of the present invention, the method further includes:

[0022] Based on the received aggregated data change request issued by the business operation layer in an asynchronous and decoupled manner, update the corresponding aggregated data.

[0023] In addition, the present invention also provides a computer-readable storage medium storing a computer program adapted for loading by a processor to perform the steps of the business data processing method described above.

[0024] In addition, the present invention also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the steps of the business data processing method described above by calling the computer program stored in the memory.

[0025] The business data processing system, method, storage medium, and device implementing the present invention have the following beneficial effects: The present invention constructs a business operation layer and an object aggregation layer that transmit data asynchronously and decoupledly. The object aggregation layer processes data from various business operation scenarios, including persistent business data formed by the business operation layer's persistence operations on the original business data, according to user business requests, forming aggregated data, and sends the aggregated data as the first response result to the user terminal. This aggregation and packaging approach enables faster and more efficient responses to business logic, supports efficient business applications in complex, dynamic, and scalable scenarios, reduces the bloat, uncontrollability, and unsustainability of lightweight architecture systems after continuous iteration, maintains the lightweight architecture and flexibility of the business system, and also reduces the development and maintenance costs of business functions. Attached Figure Description

[0026] The present invention will be further described below with reference to the accompanying drawings and embodiments. In the accompanying drawings:

[0027] Figure 1 This is a schematic diagram of the structure of an embodiment of the business data processing system of the present invention;

[0028] Figure 2 This is a flowchart illustrating an embodiment of the business data processing method of the present invention;

[0029] Figure 3These are schematic flowcharts of some embodiments of the business data processing method of the present invention;

[0030] Figure 4 This is a schematic diagram of the structure of an embodiment of the computer device of the present invention. Detailed Implementation

[0031] To provide a clearer understanding of the technical features, objectives, and effects of the present invention, specific embodiments of the present invention will now be described in detail with reference to the accompanying drawings.

[0032] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of the invention. However, those skilled in the art will understand that the invention can be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods are omitted so as not to obscure the description of the invention with unnecessary detail.

[0033] In a preferred embodiment, reference Figure 1 The business data processing system in this embodiment includes a business operation layer and an object aggregation layer that are independent of each other. The business operation layer and the object aggregation layer transmit data in an asynchronous decoupled manner. That is, the business data flow between the two layers is achieved through asynchronous decoupling. Asynchronous decoupling can be achieved by broadcasting to the object aggregation layer via a message queue, or by calling the object aggregation layer in the form of a task queue. The former sends broadcast information to the object aggregation layer in real time according to each scenario, emphasizing real-time, fast, and efficient processing; the latter is for scenarios with lower real-time requirements, where scheduling or timed tasks are triggered based on scenarios, delaying or batching processing. This emphasizes the rational utilization of resources.

[0034] The business operation layer is used to perform persistence operations on the original business data to form persistent business data. Figure 1 The DB shown here refers to persistent business data, which is stored in the database. The original business data includes data from user business requests and / or basic application data from the system. Specifically, data in user business requests is data submitted by the user, such as changing a phone number or email address, or submitting an order—data directly submitted by the user on the operation page. Basic application data refers to automatically triggered underlying business data, such as log data. This layer primarily stores the original business data, providing a data source for business operations, business access, and modularized standardized processing, enabling rapid and efficient responses to business logic.

[0035] The object aggregation layer processes data from various business operation scenarios based on user business requests, forming aggregated data, which is then sent to the user terminal as the first response result. This data includes persistent business data. The primary function of this layer is to aggregate and reconstruct business data to support efficient business applications in complex, dynamic, and scalable scenarios, reducing the bloat, uncontrollability, and unsustainability that arises from continuous iterations of lightweight architecture systems.

[0036] It's important to note that data across various business operation scenarios includes at least one of the following: search data and cached data. Search data involves integrating and reorganizing business data, binding search terms with content data, and quickly retrieving content through search term queries. Cached data is another form of storage for aggregated or trending data, essentially designed to solve the problem of fast and efficient data retrieval under high concurrency and large datasets. Different business operation scenarios may involve accessing a single data format, or accessing search data, cached data, or database data, depending on the specific business scenario or the processing logic of aggregated data.

[0037] At the business operation layer, business data producers generate basic business application data through real-time business operations and store it in the business database. This data is then written to the object aggregation layer in an asynchronous, decoupled manner. The business operation layer responds to user writes and simple access to raw data in real time, adhering to a fast response principle without complex logic processing or calculations such as business combination. The object aggregation layer, by aggregating business data, can present multi-dimensional and comprehensive data to the user, such as the overall content data of a webpage. Modularization and standardization are determined by the system module division, such as: user center, order center, service center, etc. Assuming a user submits an order, this order submission process will generate multiple business data entries. After the business data is persisted, the object aggregation layer distributes these entries to the user center, order center, service center, etc., for processing and refactoring based on their attributes. Then, it aggregates and reconstructs the aggregated data, presenting it to the user as the response result.

[0038] Alternatively, the business operation layer can also be used to publish persistent business data to the scenario center of the object aggregation layer. For example, persistent business data can be published to the scenario center of the object aggregation layer via message queues. And / or, the business operation layer can publish persistent business data to the scenario center of the object aggregation layer via task scheduling. The scenario center distinguishes the corresponding business operation scenarios based on the attributes of the persistent business data and triggers the object aggregation layer to aggregate and reconstruct data from multiple business operation scenarios according to preset business operation scenario logic to form aggregated data in response to user business requests.

[0039] It should be noted that each business module is further divided into business scenarios. Examples of business operation scenarios are as follows: User information scenarios include: adding users, editing users, deleting users, etc.

[0040] Order information can be used in the following scenarios: order creation, order editing, order payment, order shipment, and order completion.

[0041] Service information can be used in the following scenarios: service creation, service editing, service processing, and service termination.

[0042] Specifically, during the persistence operation of the original business data, the business operation layer determines the specific business operation scenario based on the attributes of the business data and then writes it into the business module. After the business module is written, it asynchronously notifies the scenario center to trigger the data update of the object aggregation layer.

[0043] Preferably, the business operation layer is also used to initiate aggregated data change requests to the object aggregation layer in an asynchronous and decoupled manner. The object aggregation layer updates the corresponding aggregated data based on the received aggregated data change request. Specifically, after a user submits data additions, edits, or modifications to the system backend, the system directly completes the business module logic and data processing, i.e., data persistence in the business operation layer. After the business module logic and data processing, the system distinguishes between business operation scenarios and initiates object aggregation layer data change requests for each scenario using task scheduling or MQ (message queue). At the scenario center of the aggregation layer, upon receiving the business operation scenario change request, the system completes the aggregated data change according to the operation scenario logic. Finally, for different user data request usage scenarios, the system retrieves data from the business operation layer or the object aggregation layer and presents it to the user.

[0044] In this embodiment, the aggregation and packaging approach can respond to business logic more quickly and effectively, support efficient business applications in complex, dynamic, and scalable scenarios, reduce the bloat, uncontrollability, and unsustainability of lightweight architecture systems after continuous iterations, maintain the lightweight architecture and flexibility of business systems, and also reduce the development and maintenance costs of business functions.

[0045] In one specific implementation, a company's official website contains a massive amount of content information. This content information is processed and produced through different stages in different forms before being provided to users in a published state.

[0046] Different content formats and structures share commonalities but also have differences. In other words, the data at the business operation layer is naturally scattered, irregular, and unstandardized. Whether it is the content processing and production stage or the content publishing stage, the business operation layer must directly process the database to complete the data persistence of the business in real time.

[0047] After the data is persisted at the business operation layer, when the scenario changes, such as when content is published manually or automatically, the object aggregation layer is broadcast in the form of MQ (or called in the form of task queue) through the content publishing scenario. The object aggregation layer scenario center combines and processes the published content data encapsulated by the business operation layer, and completes operations such as writing the aggregated data to the cache (Redis) and searching (ES).

[0048] The massive amount of content on a company's official website is presented to users by calling the object aggregation layer through data interfaces.

[0049] In some embodiments of the business data processing system, the business operation layer is also used to receive user requests, including user business requests and single data access requests.

[0050] For a single data access request, the business operation layer responds to the single data access request by sending its data as a second response result to the user terminal.

[0051] In response to user service requests, the service operation layer sends the corresponding response back to the user terminal after receiving the request, informing the user that the system has received the request.

[0052] In this embodiment, business logic can be responded to more quickly and effectively, supporting efficient business applications in complex, variable, and extended scenarios. It reduces the bloat, uncontrollability, and unsustainability of lightweight architecture systems after continuous iteration, while maintaining the lightweight architecture and flexibility of the business system.

[0053] In another preferred embodiment, the business data processing method of this embodiment is applied to the object aggregation layer within the business data processing system described above, and the method includes:

[0054] S1. In response to a user's business request, process data from multiple business operation scenarios according to preset business operation scenario logic to form aggregated data, and send the aggregated data as the first response result to the user terminal. The data from multiple business operation scenarios includes persistent business data stored in the database after the business operation layer performs persistence operations on the original business data; the original business data comes from data in the user's business request and / or the system's basic application data.

[0055] The business operation layer and the object aggregation layer transmit data asynchronously and decoupledly. That is, the business data flow between the two layers is achieved asynchronously and decoupled. Asynchronous decoupling can be achieved by broadcasting to the object aggregation layer via a message queue, or by calling the object aggregation layer via a task queue. The former sends broadcast information to the object aggregation layer in real time according to each scenario, emphasizing real-time, fast, and efficient processing; the latter is for scenarios with lower real-time requirements, where scheduling or timed tasks are triggered based on scenarios, delaying or batching processing, emphasizing efficient resource utilization.

[0056] It's important to note that data across various business operation scenarios includes at least one of the following: search data and cached data. Search data involves integrating and reorganizing business data, binding search terms with content data, and quickly retrieving content through search term queries. Cached data is another form of storage for aggregated or trending data, essentially designed to solve the problem of fast and efficient data retrieval under high concurrency and large datasets. Different business operation scenarios may involve accessing a single data format, or accessing search data, cached data, or database data, depending on the specific business scenario or the processing logic of aggregated data.

[0057] At the business operation layer, business data producers generate basic business application data through real-time business operations and store it in the business database. This data is then written to the object aggregation layer asynchronously and decoupled. The business operation layer responds to user writes and simple access to raw data in real time, adhering to a fast response principle without complex logic processing or operations such as business processing, calculation, or combination. The object aggregation layer, by aggregating business data, can present multi-dimensional and comprehensive data to the user, such as the overall content data of a webpage. Modularization and standardization are determined by the system module division, such as: user center, order center, service center, etc. Assuming a user submits an order, this order submission process will generate multiple business data entries. After the business data is persisted, the object aggregation layer distributes these entries to the user center, order center, service center, etc., for processing and refactoring based on their attributes. Then, it aggregates and reconstructs the aggregated data, presenting it to the user as the response result.

[0058] As an option, refer to Figure 2 The business operation layer is also used to publish persistent business data to the scenario center of the object aggregation layer. For example, it can publish persistent business data to the scenario center of the object aggregation layer via message queues. And / or, the business operation layer can publish persistent business data to the scenario center of the object aggregation layer via task scheduling. The scenario center distinguishes the corresponding business operation scenarios based on the attributes of the persistent business data and triggers the object aggregation layer to aggregate and reconstruct data from multiple business operation scenarios according to preset business operation scenario logic, forming aggregated data to respond to user business requests.

[0059] It should be noted that each business module is further divided into business scenarios. Examples of business operation scenarios are as follows: User information scenarios include: adding users, editing users, deleting users, etc.

[0060] Order information can be used in the following scenarios: order creation, order editing, order payment, order shipment, and order completion.

[0061] Service information can be used in the following scenarios: service creation, service editing, service processing, and service termination.

[0062] During the persistence operation of the original business data, the business operation layer determines the specific business operation scenario based on the attributes of the business data and then writes it into the business module. After the business module is written, it asynchronously notifies the scenario center to trigger the data update of the object aggregation layer.

[0063] In this embodiment, the aggregation and packaging approach can respond to business logic more quickly and effectively, support efficient business applications in complex, dynamic, and scalable scenarios, reduce the bloat, uncontrollability, and unsustainability of lightweight architecture systems after continuous iterations, maintain the lightweight architecture and flexibility of business systems, and also reduce the development and maintenance costs of business functions.

[0064] In some embodiments of the business data processing method, reference is made to Figure 3 The method also includes:

[0065] S2. Based on the received aggregated data change request issued asynchronously and decoupledly by the business operation layer, update the corresponding aggregated data. Specifically, after the user submits the addition, editing, or modification of data to the system backend, the system directly completes the business module logic and data processing, i.e., data persistence in the business operation layer; after the business module logic and data processing, distinguish the business operation scenario and initiate the object aggregated layer data change request for the operation scenario in the form of task scheduling or MQ; at the scenario center of the aggregated layer, after receiving the business operation scenario change request, the system completes the change of aggregated data according to the operation scenario logic; finally, for different user data request usage scenarios, obtain the data from the business operation layer or the data from the object aggregated layer and present it to the user.

[0066] In this embodiment, business logic can be responded to more quickly and effectively, supporting efficient business applications in complex, changing, and extended scenarios. It reduces the bloat, uncontrollability, and unsustainability of lightweight architecture systems after continuous iteration, maintains the lightweight architecture and flexibility of business systems, and also reduces the development and maintenance costs of business functions.

[0067] In another preferred embodiment, the computer-readable storage medium of this embodiment stores a computer program adapted for loading by a processor to perform the steps of the business data processing method described above.

[0068] The computer-readable storage medium of the present invention can be any computer-readable storage medium capable of storing program code, such as a USB flash drive, a portable hard drive, a read-only memory (ROM), a magnetic disk, or an optical disk.

[0069] This embodiment can respond to business logic more quickly and effectively, support efficient business applications in complex, changing, and extended scenarios, reduce the bloat, uncontrollability, and unsustainability of lightweight architecture systems after continuous iteration, maintain the lightweight architecture and flexibility of business systems, and also reduce the development and maintenance costs of business functions.

[0070] In another preferred embodiment, reference Figure 4 The computer device in this embodiment includes a memory and a processor. The memory stores a computer program, and the processor executes the steps of the business data processing method described above by calling the computer program stored in the memory.

[0071] The processor of this invention provides computing and control capabilities to support the operation of the aforementioned entire business data processing system. It should be understood that, in the embodiments of this application, the processor may be a Central Processing Unit (CPU), but it can also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor.

[0072] This embodiment can respond to business logic more quickly and effectively, support efficient business applications in complex, changing, and extended scenarios, reduce the bloat, uncontrollability, and unsustainability of lightweight architecture systems after continuous iteration, maintain the lightweight architecture and flexibility of business systems, and also reduce the development and maintenance costs of business functions.

[0073] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0074] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein can be implemented directly by hardware, a software module executed by a processor, or a combination of both. The software module can be located in random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art.

[0075] It is understood that the above embodiments only illustrate preferred embodiments of the present invention, and their descriptions are relatively specific and detailed, but they should not be construed as limiting the scope of the present invention. It should be noted that those skilled in the art can freely combine the above technical features without departing from the concept of the present invention, and can also make several modifications and improvements, all of which fall within the protection scope of the present invention. Therefore, all equivalent transformations and modifications made with respect to the scope of the claims of the present invention should fall within the scope of the claims of the present invention.

Claims

1. A business data processing system, characterized in that, It includes a business operation layer and an object aggregation layer that are independent of each other, and the business operation layer and the object aggregation layer transmit data in an asynchronous decoupled manner; The business operation layer is used to perform persistence operations on the original business data to form persistent business data, and store the persistent business data in the database; wherein, the original business data comes from data in user business requests and / or basic application data of the system, and the basic application data includes log data; The object aggregation layer is used to process data from multiple business operation scenarios to form aggregated data according to the user's business request, and send the aggregated data as a first response result to the user terminal; wherein, the data from the multiple business operation scenarios includes the persistent business data; The business operation layer is also used to publish the persistent business data to the scene center of the object aggregation layer; The scenario center distinguishes the corresponding business operation scenarios based on the attributes of the persistent business data. The object aggregation layer aggregates data from multiple business operation scenarios into aggregated data according to the preset business operation scenario logic to respond to the user's business request. The business operation layer publishes the persistent business data to the scene center of the object aggregation layer via a message queue; and / or, the business operation layer publishes the persistent business data to the scene center of the object aggregation layer via task scheduling. The data for the various business operation scenarios also includes at least one of search data and cached data. The search data is data that quickly retrieves content by querying search terms in the form of binding search terms with content data. The cached data is hot topic data.

2. The business data processing system according to claim 1, characterized in that, The business operation layer is also used to initiate an aggregated data change request to the object aggregation layer in an asynchronous and decoupled manner; the object aggregation layer updates the corresponding aggregated data according to the received aggregated data change request.

3. The business data processing system according to claim 1, characterized in that, The business operation layer is used to receive user requests, which include user business requests and single data access requests. In response to the single data access request, the service operation layer sends the data of the service operation layer as a second response result to the user terminal. In response to the user service request, after receiving the user service request, the service operation layer will send the corresponding response result back to the user terminal to inform the user that the system has received the request.

4. A business data processing method, characterized in that, Applied to the object aggregation layer within the business data processing system according to any one of claims 1 to 3, the method comprises: In response to a user's business request, data from multiple business operation scenarios are processed according to a preset business operation scenario logic to form aggregated data, and the aggregated data is sent to the user terminal as the first response result. The data in the various business operation scenarios includes persistent business data stored in the database after the business operation layer performs persistence operations on the original business data. The original business data comes from the data in the user's business request and / or the system's basic application data.

5. The business data processing method according to claim 4, characterized in that, The method further includes: Based on the received aggregated data change request issued by the business operation layer in an asynchronous and decoupled manner, update the corresponding aggregated data.

6. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program adapted for loading by a processor to perform the steps of the business data processing method as described in any one of claims 4 to 5.

7. A computer device, characterized in that, It includes a memory and a processor, wherein the memory stores a computer program, and the processor executes the steps of the business data processing method as described in any one of claims 4 to 5 by invoking the computer program stored in the memory.

Images