Data refreshing method, API node, execution node, data refreshing system, computer equipment and storage medium

Abstract

The invention relates to a data refreshing method, an API node, an execution node, a data refreshing system, computer equipment and a storage medium. The method is applied to an API node in a data refreshing system, the data refreshing system comprises the API node, a plurality of execution nodes and a plurality of edge nodes, the method comprises the steps that a task submission request sent by a client terminal is received, the task submission request is used for requesting to execute data refreshing, and the task submission request comprises client terminal information and task data; obtaining a task domain name corresponding to the task data; sending the task data and the task domain name to an execution node corresponding to the client terminal; wherein the client terminal information is used for determining an execution node for executing refresh service logic processing on the task data to obtain a refresh task, and the task domain name is used for determining an edge node for executing refresh operation based on the refresh task. The method can support high concurrent processing requirements of massive tasks.

Description

Technical Field

[0001] The present application relates to the technical field of content delivery networks, and particularly to a data refreshing method, an API node, an execution node, a data refreshing system, a computer device, and a storage medium. Background Art

[0002] Currently, the CDN (Content Delivery Network) refreshing technology generally adopts a single waterfall architecture, and its core processes are centralized task access, serialized data processing, and global resource scheduling. Under this architecture, all refresh tasks submitted by customers are uniformly accessed. After centralized verification, the tasks enter the global shared queue and are processed by a unified computing cluster and distributed to edge nodes for execution. Although this architecture simplifies the initial deployment, with the rapid increase in business volume (such as billions of tasks per day), it cannot support the high-concurrency processing requirements of massive tasks. Summary of the Invention

[0003] Based on this, it is necessary to provide a data refreshing method, an API node, an execution node, a data refreshing system, a computer device, and a storage medium for the above technical problems.

[0004] In a first aspect, the present application provides a data refreshing method, which is applied to an API node in a data refreshing system. The data refreshing system includes: the API node, multiple execution nodes, and multiple edge nodes. The method includes:

[0005] Receiving a task submission request sent by a client terminal, where the task submission request is used to request the execution of data refreshing, and the task submission request includes: client terminal information, task data, and a task domain name corresponding to the task data;

[0006] Sending the task data and the task domain name to an execution node corresponding to the client terminal;

[0007] Wherein, the client terminal information is used to determine the execution node that performs refresh service logic processing on the task data to obtain a refresh task, and the task domain name is used to determine an edge node that performs a refresh operation based on the refresh task.

[0008] In one embodiment, routing data between each client terminal information and each execution node is stored in the API node; the sending the task data and the task domain name to an execution node corresponding to the client terminal includes: determining routing data between the client terminal information and the execution node from the routing data; and based on the routing data, sending the task data and the task domain name to the execution node.

[0009] In one embodiment, the method further includes:

[0010] Determine the routing number corresponding to the routing data;

[0011] Generate a task ID based on the routing number, timestamp, and an incrementing sequence number generated corresponding to the timestamp, where the incrementing sequence number generated corresponding to the timestamp is used to indicate the order of the task ID among the task IDs generated within the time indicated by the timestamp;

[0012] Send the task ID to the client terminal, where the task ID is used for the client terminal to query the execution status of the refresh task.

[0013] In a second aspect, the present application further provides a data refresh method, which is applied to an execution node in a data refresh system. The data refresh system includes: the API node, multiple execution nodes, and multiple edge nodes. The method includes:

[0014] Receive the task data and task domain name sent by the API node, where the execution node is associated with the client terminal information to which the task data belongs;

[0015] Perform refresh service logic processing on the task data to obtain the refresh task;

[0016] Send the refresh task to the edge node corresponding to the task domain name.

[0017] In one embodiment, after performing the refresh service logic processing on the task data to obtain the refresh task, the method further includes:

[0018] Store the refresh task in the refresh task database;

[0019] After sending the refresh task to the edge node corresponding to the task domain name, the method further includes:

[0020] Receive the execution status of the refresh task sent by the edge node corresponding to the task domain name;

[0021] Update the execution status to the refresh task database and store it in association with the refresh task.

[0022] In a third aspect, the present application further provides an API node. The API node is in a data refresh system. The data refresh system includes: the API node, multiple execution nodes, and multiple edge nodes. The API node includes:

[0023] A receiving module, configured to receive a task submission request sent by a client terminal, where the task submission request is used to request data refreshing, and the task submission request includes: client terminal information, task data, and a task domain name corresponding to the task data;

[0024] A sending module, configured to send the task data and the task domain name to an execution node corresponding to the client terminal;

[0025] Wherein, the client terminal information is used to determine the execution node that performs refreshing service logic processing on the task data to obtain a refresh task, and the task domain name is used to determine an edge node that performs a refresh operation based on the refresh task.

[0026] In a fourth aspect, the present application further provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps of the method described in the first aspect or the second aspect are implemented.

[0027] In a fifth aspect, the present application further provides a computer program product, including a computer program, and when the computer program is executed by a processor, the steps of the method described in the first aspect or the second aspect are implemented.

[0028] The above data refreshing method, API node, execution node, data refreshing system, computer device, and storage medium are applied to an API node in a data refreshing system. The data refreshing system includes: an API node, multiple execution nodes, and multiple edge nodes. The method includes: receiving a task submission request sent by a client terminal, where the task submission request is used to request data refreshing, and the task submission request includes: client terminal information, task data, and a task domain name corresponding to the task data; sending the task data and the task domain name to an execution node corresponding to the client terminal; wherein, the client terminal information is used to determine the execution node that performs refreshing service logic processing on the task data to obtain a refresh task, and the task domain name is used to determine an edge node that performs a refresh operation based on the refresh task. Through this solution, the API node receives the task submission request of the client terminal, dynamically routes it to a dedicated execution node for service logic processing based on the client terminal information, and accurately maps to the target edge node to perform the refresh operation in combination with the task domain name. Using the dual routing strategy of client information and domain name, it isolates the resource competition and fault impact between different customer groups and modules, realizes parallel processing of tasks, can effectively support the high-concurrency processing requirements of a large number of tasks, and significantly improves the throughput and response speed of the system. Description of the Drawings

[0029] To more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the following will briefly introduce the accompanying drawings required for the description of the embodiments or related technologies. Obviously, the accompanying drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, without creative efforts, other accompanying drawings can be obtained based on these drawings.

[0030] Figure 1 Schematic diagram of the architecture of a data refresh system provided by an embodiment of the present application;

[0031] Figure 2 Flow chart of a data refresh method;

[0032] Figure 3 Flow chart of another data refresh method;

[0033] Figure 4 Flow chart of still another data refresh method;

[0034] Figure 5 Structural block diagram of an API node;

[0035] Figure 6 Structural block diagram of an execution node;

[0036] Figure 7 Internal structure diagram of a computer device in an embodiment. Detailed implementation manners

[0037] In order to make the objectives, technical solutions and advantages of the present application more clear and understandable, the present application will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0038] The content delivery network (CDN) refresh technology is one of the core functions to ensure the efficient operation of the CDN. In a dynamically changing network environment, website content needs to be updated frequently to reflect the latest information or product changes. The CDN refresh technology can quickly push the latest version of resources to users after the content is updated, preventing users from accessing outdated content, thereby enhancing the user experience and strengthening the professionalism and reliability of the website. In addition, this technology also helps to improve the security of the website and the performance of search engine optimization (SEO), and is an indispensable part of building an efficient, stable and fast CDN architecture.

[0039] With the development of the Internet, the volume of cache invalidation tasks faced by CDN has increased explosively, and billions of cache invalidation tasks may be generated in a single day. In the current CDN industry, most mainstream CDN cache invalidation architectures adopt a single waterfall architecture. The processing capacity of this architecture has an upper limit, and its performance is poor when dealing with the access, processing, and distribution of a large number of tasks, which easily leads to excessive load on the cache invalidation platform and the corresponding middleware.

[0040] In addition, the characteristics of the cache invalidation service determine that most of the traffic is concentrated in a few top customers. Under the single waterfall architecture, the tasks of top customers and ordinary customers are mixed for processing. When sudden traffic occurs to top customers, it is easy to affect the services of all customers on the entire platform. More importantly, this architecture has a single point of failure risk. Once a certain component fails, it may trigger an exception at the entire platform level, and the scope of the failure impact is difficult to control.

[0041] In the embodiments of the present application, a data cache invalidation method and a data cache invalidation system are provided, which can isolate resource competition and failure impacts between different customer groups and modules, implement parallel processing of tasks, effectively support the high-concurrency processing requirements of a large number of tasks, and significantly improve the throughput and response speed of the system.

[0042] Exemplarily, Figure 1 is a schematic architecture diagram of a data cache invalidation system provided by an embodiment of the present application. The data cache invalidation system includes: at least one Application Programming Interface (API) node, multiple execution nodes, and multiple edge nodes. As Figure 1 shown in, it includes 2 API nodes, namely API node 1 and API node 2; the multiple execution nodes are Figure 1 the execution node 1, execution node 2,..., execution node m in, and the multiple edge nodes are Figure 1 the edge node 1, edge node 2,..., edge node n in, where both m and n are integers greater than or equal to 2.

[0043] The above API node (such as Figure 1 API node 1 in) is used to receive the task submission request sent by the client terminal, and the data will be filtered and verified in the API node. In the API node, the routing data between each client terminal information and each of the execution nodes is also stored. Based on these routing data, the API node can forward the task to the execution node corresponding to the route.

[0044] Exemplarily, Figure 1In the example, API node 1 is the main API node, and API node 2 is the standby API node. There are databases in both API nodes. The database data of the standby API node can be synchronized from the database of the main API node. The rest of the resources are independent resources. When the main API node and the standby API node need to be switched, you only need to modify the DNS resolution of the business domain name used by the API node to the standby API node, and then you can introduce the customer request traffic to the standby API node. At the same time, in the master-slave mode of the database in the main API node and the standby API node, the standby API node can be upgraded to the main API node. When the main API node and the standby API node switch, the same refresh task is written to the queue of the same execution node, and the execution node is unaware.

[0045] Above Figure 1 Each execution node in the execution node may include: a data processing module, a data distribution module, a task callback module, a database, and a queue. These execution nodes are independent of each other.

[0046] For example, suppose Figure 1 These modules in Execution Node 1 can achieve the following different functions:

[0047] (1) Queue: used to receive task data, task domain name and other information forwarded by API node 1.

[0048] (2) Data processing module: It can obtain task data from the queue and execute refresh business logic processing to convert it into a refresh task that can be executed by the edge node.

[0049] (3) Data distribution module: Calculates the edge node 1 to which the task needs to be sent based on the task domain name of the refresh task, and pushes the data to edge node 1.

[0050] (4) Database: used to store refresh tasks and information such as the execution status of the corresponding refresh tasks.

[0051] (5) Task callback module: After the refresh task is executed on the edge node, the execution status information of the refresh task will be reported to the task callback module. The task callback module will update the task status in the database based on the execution status of the refresh task.

[0052] It should be noted that the above description is based on execution node 1 as an example, and other execution nodes are similar.

[0053] Above Figure 1 Each edge node in the CDN is an edge machine that is used to perform the actual refresh action and refresh the specified path file on the disk.

[0054] It should be noted that in practical applications, the above API nodes, execution nodes, and edge nodes can all be server clusters.

[0055] During the daily operation of the API node component, the primary API node and the standby API node maintain data synchronization. The specific synchronization method can adopt the master-slave replication technology of the database. The database of the standby API node synchronizes the update operations of the master database to the standby database by listening to the transaction log of the master API node database in real time. When the primary API node fails or needs to be maintained, by modifying the DNS resolution record, the business domain name is resolved to the IP address of the standby API node cluster, so as to direct the customer request traffic to the standby API node. At the same time, the master-standby mode of the database is switched, and the database of the standby API node is upgraded to the master database to continue processing new business requests. During the entire switching process, since the same refresh task is written into the queue in the same execution node, the execution node does not need to perform any additional operations, ensuring the continuity and stability of the business.

[0056] In practical application scenarios, each execution node sets an upper limit for the business Queries Per Second (QPS) threshold. QPS is an important indicator to measure the processing capacity of the system and is often used to evaluate the number of requests that services such as API interfaces and databases can process per unit time. In the scenario mentioned in the text, the QPS threshold is used to trigger load balancing and traffic scheduling strategies to ensure system stability.

[0057] When the business volume of a certain customer suddenly increases and triggers the QPS threshold alarm, the system can dynamically switch some tasks of other customers on the same unit to other execution nodes according to the preset task-switching strategy. For example, the QPS threshold of execution node 1 is 1000, and the sudden increase in the business of customer C causes the unit QPS to reach 900. At this time, the system can transfer some tasks of customer D to execution node 2, thereby reducing the load on execution node 1 and ensuring business stability. When the business volume of a certain customer exceeds the upper limit of the unit QPS, the system can switch all other customers on the corresponding unit to other units, and only keep this customer using all unit resources independently. For example, the business volume of customer E increases sharply and exceeds the bearing capacity of execution node 3. The system will migrate all customers such as customer F and G on execution node 3 to execution node 4 to ensure that the business of customer E is not affected and at the same time avoid interfering with other unit customers.

[0058] For example, if a component of execution node 5 fails, the system will automatically detect the exception and switch all customer routing information on execution node 5 to other normal execution nodes, such as execution node 6, according to the pre-configured routing switching rules. When the customer initiates a request, the API node forwards the task to execution node 6 according to the updated routing information. The whole process is completely transparent to the customer, and the customer will not perceive the interruption of service. When the failure of execution node 5 is repaired, the system administrator can choose to switch some or all of the customer's routing information back to execution node 5 according to the actual business situation. If the default unit is abnormal, the system can direct customer requests for unconfigured units to the new default unit by modifying the default unit identifier in the configuration file. For example, if the original default unit 7 fails, after the default unit is changed to unit 8, subsequent customer requests that do not specify an execution node will be sent to unit 8 for processing.

[0059] As the business volume grows, when the existing execution nodes approach or reach their carrying capacity, the computing modules need to be expanded horizontally. The specific process is as follows: First, the business load of each execution node is monitored in real time through the monitoring system. When the overall business volume is close to the threshold of the current execution node cluster, a new computing unit is deployed. The deployment process of the new computing unit is completely independent of the existing unit, including operations such as setting up the server environment, installing the required software components, and configuring the database. After the new computing unit is deployed, the tasks of some customers are assigned to the new computing unit by modifying the customer routing information stored in the API node. For example, the routing information of customers H and I is assigned from the original execution node to the newly launched execution node 9, so that the new unit begins to undertake business processing tasks. In the early stage of the new unit's launch, the system will continuously monitor its operating status and performance indicators to ensure its stable operation. At the same time, according to actual business needs and load conditions, the allocation ratio of customers between various execution nodes can be flexibly adjusted to achieve optimal configuration of system resources.

[0060] When a customer's routing is switched, for example, customer J's business was originally processed by execution node 10, and its routing is switched to execution node 11 due to business adjustments. Since the task ID contains routing information, when customer J uses the previously generated task ID to query the status, the API node can accurately query the historical status information of the task in the database of execution node 10 according to the routing number in the task ID. Even if the customer's belonging unit changes, it will not affect the query operation of the old task, avoiding the abnormal perception of customer queries caused by internal customer routing switching, ensuring that customers can obtain complete and accurate task execution status, and minimizing the impact on customer business.

[0061] In an embodiment of the present application, a data refreshing method is provided, which is applied to an API node in a data refreshing system. The data refreshing system includes: an API node, multiple execution nodes, and multiple edge nodes. The method includes: receiving a task submission request sent by a client terminal, where the task submission request is used to request the execution of data refreshing, and the task submission request includes: client terminal information, task data, and a task domain name corresponding to the task data; sending the task data and the task domain name to an execution node corresponding to the client terminal; wherein, the client terminal information is used to determine the execution node that performs the refreshing service logic processing on the task data to obtain a refreshing task, and the task domain name is used to determine an edge node that performs a refreshing operation based on the refreshing task.

[0062] In an embodiment of the present application, there is also a data refreshing method, which is applied to an execution node in a data refreshing system. The data refreshing system includes: an API node, multiple execution nodes, and multiple edge nodes. The method includes: receiving the task data and the task domain name sent by the API node, and the execution node is associated with the client terminal information to which the task data belongs; performing refreshing service logic processing on the task data to obtain a refreshing task; sending the refreshing task to an edge node corresponding to the task domain name.

[0063] The above API node is an application programming interface node, which serves as an entry for the data refreshing system to interact with external client terminals. It is responsible for receiving requests sent by client terminals, and after preliminary processing of the request information, forwarding it to other nodes within the system, playing a role in data transfer and protocol adaptation.

[0064] The above execution node is a node specifically used to perform specific refreshing service logic processing on task data in the data refreshing system. It is associated with client terminal information and converts the received task data into an executable refreshing task according to business rules.

[0065] The above edge node: The nodes distributed at the network edge are closer to the user terminal. This edge node is responsible for performing actual data refreshing operations in the local cache according to the received refreshing task, providing the latest data to the user, and improving data access speed and response efficiency.

[0066] In an exemplary embodiment, Figure 2 It is a schematic flowchart of a data refreshing method, and the method includes the following steps:

[0067] 201. The API node receives a task submission request sent by a client terminal.

[0068] Wherein, the task submission request is used to request the execution of data refreshing, and the task submission request includes: client terminal information, task data, and a task domain name corresponding to the task data.

[0069] Among them, the above task submission request is a request initiated by the client terminal to the data refresh system, used to inform the system that a data refresh operation needs to be performed.

[0070] Exemplarily, the above client terminal information is used to identify the client identity and associate with the corresponding execution node. The above task data refers to the data content that needs to be refreshed, and the above task domain name is used to locate the edge node that performs the refresh operation.

[0071] 202. The API node sends the task data and the task domain name to the execution node corresponding to the client terminal.

[0072] Among them, the client terminal information is used to determine the execution node that performs the refresh service logic processing on the task data to obtain the refresh task, and the task domain name is used to determine the edge node that performs the refresh operation based on the refresh task.

[0073] In some embodiments, the API node stores the routing data between each client terminal information and each execution node.

[0074] In the process of the above API node sending the task data and the task domain name to the execution node corresponding to the client terminal, it can first determine the target routing data between the client terminal information and the execution node from the routing data; then based on the target routing data, send the task data and the task domain name to the execution node.

[0075] 203. The execution node performs the refresh service logic processing on the task data to obtain the refresh task.

[0076] Among them, the refresh service logic processing may include a series of operations performed by the execution node on the task data, including but not limited to: data format conversion, data verification, task priority determination, etc. The purpose is to convert the original task data into a refresh task that the edge node can directly execute.

[0077] 204. Send the refresh task to the edge node corresponding to the task domain name.

[0078] 205. The edge node performs the refresh operation based on the refresh task.

[0079] The above data refresh method receives the task submission request of the client terminal through the API node, dynamically routes to the exclusive execution node for service logic processing based on the client terminal information, and combines the task domain name to accurately map the target edge node to perform the refresh operation. Using the dual routing strategy of client information and domain name, it isolates the resource competition and fault impact between different customer groups and modules, realizes the parallel processing of tasks, can effectively support the high-concurrency processing requirements of massive tasks, and significantly improves the throughput and response speed of the system.

[0080] In an exemplary embodiment,Figure 3 It is a flowchart of another data refresh method, and this method includes the following steps:

[0081] 301. The execution node stores the refresh task in the refresh task database.

[0082] Among them, after the execution node executes step 203 above, step 301 can be executed.

[0083] The refresh task database is a database used to store the refresh tasks generated by the execution node and their related information, and can be integrated inside the execution node or used as an independent storage component. The functions of the refresh task database mainly include but are not limited to: 1) Persistently store the refresh tasks to avoid task loss; 2) Record the task execution status for subsequent query and management; 3) Support operations such as task retry and rollback.

[0084] Exemplarily, after the execution node completes the processing such as format conversion and verification of the task data, the generated refresh tasks are serialized into structured data, such as data in the format of Extensible Markup Language (XML). Then, the task data is written into the refresh task database through the database interface, and at the same time, metadata such as the creation time and priority of the task is recorded. Among them, the database interface is such as Structured Query Language (SQL), Not Only SQL (NoSQL) driver.

[0085] 302. The execution node receives the execution status of the refresh task sent by the edge node corresponding to the task domain name.

[0086] Exemplarily, after the edge node completes the refresh task, it can send a status report to the execution node through the network communication protocol, including the task ID and the status value; the execution node can receive the status data by listening on a specific port or message queue to ensure the reliability and asynchrony of communication.

[0087] In the above step 302, the process of the edge node feeding back the status to the execution node after executing the refresh task includes: after the edge node completes the actual refresh operation of the disk file, it extracts the routing number from the received download message, and there is a mapping relationship between this routing number and the callback address of the execution node. The edge node can determine the target callback address through the mapping table stored locally, and send a message containing the task ID and the execution status (such as success / failure) to the task callback module of the corresponding execution node, so as to realize the real-time feedback of the execution status.

[0088] 303. The execution node updates the execution status to the refresh task database and stores it in association with the refresh task.

[0089] After the above step 204 is executed at the execution node, the above steps 302 and 303 can be executed.

[0090] Exemplarily, the result status of the edge node executing the refresh task may include, but is not limited to:

[0091] (1) To be executed: The task has been received but not started to be processed;

[0092] (2) Processing: The edge node is executing data refresh;

[0093] (3) Success: The data refresh operation is completed and correct;

[0094] (4) Failure: An error occurs during the refresh process (such as data verification failure, cache write exception, etc.);

[0095] (5) Timeout: The task has not been completed after exceeding the preset time.

[0096] Exemplarily, the execution node can query the corresponding refresh task record from the database according to the task ID; update the status field of the task, and record the update time, error information, etc.; if task tracking needs to be supported, the historical record of status changes can be additionally stored, such as: content such as timestamps, status change logs, etc.

[0097] The above embodiment shown in FIG. 3 realizes the full life cycle state management of the refresh task. By introducing a refresh task database in the execution node, the system can persistently store and associatively update the generated refresh tasks and their execution statuses. The execution node can track the task execution status of the edge node in real time (such as statuses like success, failure, processing, etc.), forming a complete closed loop of "task generation - sending - execution - feedback". Its advantages are reflected in: on the one hand, the database storage avoids the problem that the status information cannot be queried due to the loss of in-memory data, ensuring that the task status is traceable; on the other hand, when an exception occurs during the edge node's task execution, the system can quickly locate the faulty link based on the database record (such as data verification failure, node communication exception, etc.), significantly improving the fault troubleshooting efficiency and enhancing the stability and maintainability of the system.

[0098] In an exemplary embodiment, Figure 4 For the flowchart of yet another data refresh method, this method includes the following steps:

[0099] 401. The API node determines the routing number corresponding to the target routing data.

[0100] Among them, the above-mentioned target routing data is the routing information between the customer terminal information determined from the routing data and the execution node. The mapping relationship between the customer terminal information stored in the API node and the execution node is used to determine the routing path of the task data. For example: Customer A corresponds to execution node X, and Customer B corresponds to execution node Y.

[0101] 402. The API node generates a task ID based on the routing number, timestamp, and an incrementing sequence number corresponding to the timestamp.

[0102] Among them, the incrementing sequence number corresponding to the timestamp is used to indicate the order of the task ID among the task IDs generated within the time indicated by the timestamp.

[0103] In some embodiments, a total unique task ID can be generated using a generation rule.

[0104] Exemplarily, the generation rule is: {[10 - digit timestamp][routing number][5 - digit incrementing sequence number]}, and the generated task ID is 16 - digit. It should be noted that the number of digits shown for the routing number, timestamp, and the incrementing sequence number corresponding to the timestamp (i.e., this 5 - digit incrementing sequence number) in the above - mentioned generation rule are all exemplary illustrations, and in practice, there can be more or fewer digits.

[0105] 403. The API node sends the task ID to the customer terminal.

[0106] Among them, the above - mentioned task ID is used by the customer terminal to query the execution status of the refresh task.

[0107] 404. The customer terminal sends a task status query request to the API node.

[0108] Among them, the task status query request includes the above - mentioned task ID.

[0109] 405. The API node parses the task ID to obtain the corresponding routing number.

[0110] Among them, the parsing rule of the task ID needs to be consistent with the generation rule.

[0111] 406. The API node obtains the execution status of the refresh task in the corresponding execution node according to the path indicated by the routing number.

[0112] The API node can find the corresponding execution node address according to the routing number, send a status query request to the execution node, obtain the execution status of the task (such as success / failure / processing), and return the status result to the customer terminal.

[0113] Figure 4 constructs a standardized task status query system. The API node generates a globally unique task ID through the routing number, timestamp, and incrementing sequence number, and uses it as the core identifier for the interaction between the client terminal and the system. The client terminal can actively initiate a status query request by virtue of the task ID. The API node accurately locates the corresponding execution node by parsing the routing number in the ID to obtain the status information and feedback. In this solution, the uniqueness of the task ID ensures the accurate differentiation of task statuses in high-concurrency scenarios, avoiding query confusion; moreover, the client terminal can independently monitor the task progress; also, by embedding routing information in the task ID, the routing logic for cross-node queries is simplified, the data interaction cost is reduced, and the status query process becomes more efficient.

[0114] In the data refreshing method provided in the embodiments of the present application, each execution node is independent, so that the execution nodes do not affect each other and platform-level exceptions do not occur. Moreover, the independent execution nodes can support horizontal expansion and can be flexibly deployed according to sudden business modules, breaking through the limitations of a single waterfall model architecture. Theoretically, it can support an unlimited number of refreshed business volumes. And each client terminal belongs to an execution unit. When scenarios such as sudden business growth and cluster failures occur, the belonging unit can be dynamically switched to maximize the guarantee of the smoothness of the client's business.

[0115] It should be understood that although the steps in the flowcharts involved in the above-described embodiments are shown in sequence according to the arrows, these steps do not necessarily have to be executed in the order indicated by the arrows. Unless there is a clear indication in this article, the execution of these steps has no strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-described embodiments may include multiple steps or multiple stages. These steps or stages do not necessarily have to be executed at the same time, but can be executed at different times. The execution order of these steps or stages does not necessarily have to be sequential, but can be executed alternately or alternately with at least a part of other steps or steps or stages in other steps.

[0116] In an exemplary embodiment, as Figure 5 shown, a structural block diagram of an API node is provided. The API node is in a data refreshing system, and the data refreshing system includes: the API node, multiple execution nodes, and multiple edge nodes. The API node includes:

[0117] A receiving module 501, configured to receive a task submission request sent by a client terminal, where the task submission request is used to request data refreshing, and the task submission request includes: client terminal information, task data, and a task domain name corresponding to the task data;

[0118] A sending module 502, configured to send the task data and the task domain name to an execution node corresponding to the client terminal;

[0119] Wherein, the client terminal information is used to determine the execution node that performs the refresh service logic processing on the task data to obtain a refresh task, and the task domain name is used to determine an edge node that performs a refresh operation based on the refresh task.

[0120] In some embodiments, the API node further includes:

[0121] A storage module 503, configured to store routing data between each client terminal information and each execution node:

[0122] A determination module 504, configured to determine target routing data between the client terminal information and the execution node from the routing data;

[0123] The sending module 502 is configured to send the task data and the task domain name to the execution node based on the target routing data.

[0124] In some embodiments, the determination module 504 is further configured to determine a routing number corresponding to the target routing data; generate a task ID based on the routing number, a time stamp, and an incrementing sequence number generated corresponding to the time stamp, where the incrementing sequence number generated corresponding to the time stamp is used to indicate the order of the task ID among the task IDs generated at the time indicated by the time stamp;

[0125] The sending module 502 is configured to send the task ID to the client terminal, and the task ID is used for the client terminal to query the execution status of the refresh task.

[0126] In an exemplary embodiment, as Figure 6 shown, a structural block diagram of an execution node is provided. The execution node is in a data refresh system, and the data refresh system includes: an API node, multiple execution nodes, and multiple edge nodes. The execution node includes:

[0127] A receiving module 601, configured to receive the task data and the task domain name sent by the API node. The execution node is associated with the client terminal information to which the task data belongs;

[0128] A logic processing module 602, configured to perform refresh service logic processing on the task data to obtain the refresh task;

[0129] A sending module 603, configured to send the refresh task to an edge node corresponding to the task domain name.

[0130] In some embodiments, the execution node further includes:

[0131] A storage module 605 for storing the refresh task in the refresh task database;

[0132] After the logic processing module 602 performs refresh service logic processing on the task data to obtain the refresh task, the receiving module 601 is further configured to receive the execution status of the refresh task sent by the edge node corresponding to the task domain name;

[0133] An update module 604 for updating the execution status to the refresh task database and storing it in association with the refresh task.

[0134] Each module in the above API node and execution node can be implemented in whole or in part by software, hardware, and their combination. Each of the above modules can be embedded in the processor of the computer device in hardware form or independent of it, or stored in the memory of the computer device in software form, so that the processor can call and execute the operations corresponding to each of the above modules.

[0135] In an exemplary embodiment, a computer device is provided. The computer device can be any cluster among the above API node, execution node, or edge node, and its internal structure diagram can be as Figure 7 shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, abbreviated as I / O), and a communication interface. Among them, the processor, memory, and input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store data. The input / output interface of the computer device is used to exchange information between the processor and external devices. When the computer program is executed by the processor, it implements the data refresh method involved in any one of the above API node, execution node, or edge node in the above method embodiment.

[0136] Those skilled in the art can understand that Figure 7 the structure shown in is only a block diagram of some structures related to the solution of this application, and does not constitute a limitation on the computer device to which the solution of this application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine some components, or have different component arrangements.

[0137] In an exemplary embodiment, a computer device is provided, including a memory and a processor. A computer program is stored in the memory, and when the processor executes the computer program, a data refreshing method involved in any one of the above API nodes, execution nodes, or edge nodes in the above method embodiment is implemented.

[0138] In an embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, a data refreshing method involved in any one of the above API nodes, execution nodes, or edge nodes in the above method embodiment is implemented.

[0139] In an embodiment, a computer program product is provided, including a computer program. When the computer program is executed by a processor, a data refreshing method involved in any one of the above API nodes, execution nodes, or edge nodes in the above method embodiment is implemented.

[0140] In the embodiments of the present application, a data refreshing system is further provided. The system includes: an API node, a plurality of execution nodes, and a plurality of edge nodes; the API node is used to execute each process involved in the API node in the above method embodiment, and any one of the plurality of execution nodes is used to execute each process involved in the execution node in the above method embodiment; any one of the plurality of edge nodes is used to execute each process involved in the execution node in the above method embodiment. Exemplarily, the data refreshing system may be as shown above. Figure 1 as shown.

[0141] Those of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments can be completed by instructing relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above methods. Among them, any reference to a memory, database, or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memories. Non-volatile memories can include read-only memory (ROM), magnetic tapes, floppy disks, flash memories, optical memories, high-density embedded non-volatile memories, resistive random access memories (ReRAM), magnetoresistive random access memories (MRAM), ferroelectric random access memories (FRAM), phase change memories (PCM), graphene memories, etc. Volatile memories can include random access memory (RAM) or external cache memories, etc. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc. The databases involved in the embodiments provided in the present application can include at least one of relational databases and non-relational databases. Non-relational databases can include distributed databases based on blockchain, etc., without limitation. The processors involved in the embodiments provided in the present application can be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, data processing logics based on quantum computing, etc., without limitation.

[0142] The technical features of the above embodiments can be combined arbitrarily. For the sake of brevity of description, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, it should be considered as the scope described in this specification.

[0143] The above-described embodiments merely represent several implementation manners of the present application. The description thereof is relatively specific and detailed, but it should not be construed as a limitation on the patent scope of the present application. It should be noted that for those of ordinary skill in the art, without departing from the concept of the present application, several modifications and improvements can still be made, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the appended claims.

Claims

1. A data refreshing method, characterized in that, An API node applied to a data refresh system, the data refresh system including: the API node, a plurality of execution nodes, and a plurality of edge nodes, the method including: Receiving a task submission request sent by a client terminal, the task submission request being used to request data refresh execution, the task submission request including: client terminal information, task data, and a task domain name corresponding to the task data; Sending the task data and the task domain name to an execution node corresponding to the client terminal; Wherein, the client terminal information is used to determine the execution node that performs refresh service logic processing on the task data to obtain a refresh task, and the task domain name is used to determine an edge node that performs a refresh operation based on the refresh task.

2. The method according to claim 1, wherein Routing data between each client terminal information and each of the execution nodes is stored in the API node; The sending the task data and the task domain name to an execution node corresponding to the client terminal includes: Determining target routing data between the client terminal information and the execution node from the routing data; Based on the target routing data, sending the task data and the task domain name to the execution node.

3. The method according to claim 2, characterized in that, The method further includes: Determining a routing number corresponding to the target routing data; Generating a task ID based on the routing number, a timestamp, and an incrementing sequence number corresponding to the timestamp, the incrementing sequence number corresponding to the timestamp being used to indicate the order of the task ID among the task IDs generated at the time indicated by the timestamp; Sending the task ID to the client terminal, the task ID being used by the client terminal to query the execution status of the refresh task.

4. A data refreshing method, characterized in that, An execution node applied to a data refresh system, the data refresh system including: an API node, a plurality of the execution nodes, and a plurality of edge nodes, the method including: Receiving the task data and the task domain name sent by the API node, the execution node being associated with the client terminal information to which the task data belongs; Performing refresh service logic processing on the task data to obtain a refresh task; Sending the refresh task to an edge node corresponding to the task domain name.

5. The method according to claim 4, characterized in that, After performing the refresh service logic processing on the task data to obtain the refresh task, the method further includes: Storing the refresh task in a refresh task database; After sending the refresh task to an edge node corresponding to the task domain name, the method further includes: Receiving the execution status of the refresh task sent by the edge node corresponding to the task domain name; Updating the execution status to the refresh task database and storing it in association with the refresh task.

6. An API node, characterized in that, The API node is in a data refresh system, the data refresh system including: the API node, a plurality of execution nodes, and a plurality of edge nodes, the API node including: A receiving module, configured to receive a task submission request sent by a client terminal, the task submission request being used to request data refresh execution, the task submission request including: client terminal information, task data, and a task domain name corresponding to the task data; A sending module, configured to send the task data and the task domain name to an execution node corresponding to the client terminal; Wherein, the client terminal information is used to determine the execution node that performs the refresh service logic processing on the task data to obtain a refresh task, and the task domain name is used to determine an edge node that performs a refresh operation based on the refresh task.

7. An execution node, characterized in that, The execution node is in a data refresh system, and the data refresh system includes: an API node, a plurality of the execution nodes, and a plurality of edge nodes. The execution node includes: A receiving module, which receives the task data and the task domain name sent by the API node, and the execution node is associated with the client terminal information to which the task data belongs; A logic processing module, configured to perform a refresh service logic process on the task data to obtain a refresh task; A sending module, configured to send the refresh task to an edge node corresponding to the task domain name.

8. A data refresh system, characterized in that, The system includes: an API node, a plurality of execution nodes, and a plurality of edge nodes; The API node is configured to execute the method according to any one of claims 1 to 4; Any one of the plurality of execution nodes is configured to execute the method according to claim 5 or 6; Any one of the plurality of edge nodes is configured to receive the refresh task sent by the execution node and perform a refresh operation.

9. A computer device, comprising a memory and a processor, the memory storing a computer program, characterized in that, The processor executes the computer program to implement the steps of the method according to any one of claims 1 to 5.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, the steps of the method according to any one of claims 1 to 5 are implemented.

Images