Data processing method and device, equipment, storage medium and program product

Abstract

The application provides a data processing method and device, equipment, a storage medium and a program product. The method comprises the following steps: in response to the fact that the received first scheduling information is different from the second scheduling information stored locally by a service node, determining a second data source of the service node based on the access addresses of a plurality of data sources included in the first scheduling information; and switching the first data source of the service node to the second data source based on the access addresses of the second data source. Through the application, the data processing efficiency of the service node can be effectively improved.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a data processing method, apparatus, device, storage medium, and program product. Background Technology

[0002] Clustering technology, a technology that has emerged with the development of the internet, connects a group of independent computers through a high-speed network to form a cluster system that can be uniformly managed and scheduled. This system is used to handle various big data scenarios, thereby improving service scalability and reliability. To ensure the normal operation of big data scenarios, cluster systems typically store massive amounts of data to support each type of business within the big data environment.

[0003] In related technologies, for the data source connection of business nodes, the business nodes usually connect to a fixed data source for data processing. However, since the fixed data source cannot meet the data processing requirements during the data processing process, the processing efficiency of the business nodes is low. Summary of the Invention

[0004] This application provides a data processing method, apparatus, electronic device, computer-readable storage medium, and computer program product, which can effectively improve the data processing efficiency of business nodes.

[0005] The technical solution of this application embodiment is implemented as follows:

[0006] This application provides a data processing method, including:

[0007] In response to the fact that the received first scheduling information is different from the second scheduling information stored locally by the service node, the second data source of the service node is determined based on the access addresses of multiple data sources included in the first scheduling information;

[0008] Based on the access address of the second data source, the first data source of the business node is switched to the second data source.

[0009] This application provides a data processing method, including:

[0010] Generate first scheduling information for a business node, and in response to an information acquisition request sent by the business node, send the first scheduling information to the business node. The first scheduling information is used to trigger the business node to determine a second data source and switch the connected first data source to the second data source if the first scheduling information is different from the second scheduling information stored locally by the business node.

[0011] This application provides a data processing apparatus, including:

[0012] The determination module is used to determine the second data source of the business node based on the access addresses of multiple data sources included in the first scheduling information, in response to the difference between the received first scheduling information and the second scheduling information stored locally on the business node.

[0013] The switching module is used to switch the first data source of the business node to the second data source based on the access address of the second data source.

[0014] This application provides a data processing apparatus, including:

[0015] The generation module is used to generate the first scheduling information for the business nodes;

[0016] The sending module is used to send the first scheduling information to the service node in response to the information acquisition request sent by the service node;

[0017] The first scheduling information is used to trigger the service node to determine the second data source and switch the connected first data source to the second data source when the first scheduling information is different from the second scheduling information stored locally by the service node.

[0018] This application provides an electronic device, including:

[0019] Memory is used to store executable instructions or computer programs.

[0020] The processor, when executing computer-executable instructions or computer programs stored in the memory, implements the data processing method provided in the embodiments of this application.

[0021] This application provides a computer-readable storage medium storing computer-executable instructions for inducing a processor to execute and implement the data processing method provided in this application.

[0022] This application provides a computer program product, which includes a computer program or computer-executable instructions stored in a computer-readable storage medium. The processor of an electronic device reads the computer-executable instructions from the computer-readable storage medium and executes the computer-executable instructions, causing the electronic device to perform the data processing method described in this application.

[0023] The embodiments of this application have the following beneficial effects:

[0024] During data processing at a business node connected to the first data source, if the received first scheduling information differs from the second scheduling information stored locally on the business node, and since the first scheduling information carries access addresses for multiple data sources, including the second data source, the second data source that the business node needs to switch to can be determined based on the first scheduling information. Based on the access address of the second data source, the first data source connected to the business node is switched to the second data source. Thus, when the received first scheduling information differs from the second scheduling information stored locally on the business node, the second data source for the business node is determined based on the access addresses of the multiple data sources included in the first scheduling information, and the first data source connected to the business node is switched to the second data source based on the access address of the second data source. This achieves real-time hot switching of data sources during data processing, meeting the data processing needs of different data sources and effectively improving the processing efficiency of the business node. Attached Figure Description

[0025] Figure 1 This is a schematic diagram of the architecture of the data processing system provided in the embodiments of this application;

[0026] Figure 2 This is a schematic diagram of the structure of an electronic device for data processing provided in an embodiment of this application. Figure 1 ;

[0027] Figure 3 This is a schematic diagram of the structure of an electronic device for data processing provided in an embodiment of this application. Figure 2 ;

[0028] Figure 4 This is a flowchart illustrating the data processing method provided in the embodiments of this application. Figure 1 ;

[0029] Figure 5 This is a flowchart illustrating the data processing method provided in the embodiments of this application. Figure 2 ;

[0030] Figure 6 This is a flowchart illustrating the data processing method provided in the embodiments of this application. Figure 3 ;

[0031] Figure 7 This is a schematic diagram illustrating the principle of the data processing method provided in the embodiments of this application. Detailed Implementation

[0032] To make the objectives, technical solutions, and advantages of this application clearer, the application will be further described in detail below with reference to the accompanying drawings. The described embodiments should not be regarded as limitations on this application. All other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0033] In the following description, references are made to “some embodiments,” which describe a subset of all possible embodiments. However, it is understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0034] In the following description, the terms "first, second, third" are used merely to distinguish similar objects and do not represent a specific ordering of objects. It is understood that "first, second, third" may be interchanged in a specific order or sequence where permitted, so that the embodiments of this application described herein can be implemented in an order other than that illustrated or described herein.

[0035] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit this application.

[0036] Before providing a further detailed description of the embodiments of this application, the nouns and terms involved in the embodiments of this application will be explained, and the nouns and terms involved in the embodiments of this application shall be interpreted as follows.

[0037] 1) Distributed System: This refers to a distributed file system designed to run on commodity hardware. It shares many similarities with existing distributed file systems, but its differences from others are also significant. HDFS is a highly fault-tolerant system suitable for deployment on inexpensive machines. HDFS provides high-throughput data access, making it ideal for applications with large datasets. HDFS relaxes some POSIX constraints to enable streaming access to file system data. HDFS is fault-tolerant and designed for deployment on low-cost hardware. It provides high throughput access to application data, making it suitable for applications with very large datasets. HDFS relaxes POSIX requirements to enable streaming access to data in the file system.

[0038] 2) Cloud Technology: Cloud technology refers to a hosting technology that unifies hardware, software, and network resources within a wide area network (WAN) or local area network (LAN) to achieve data computation, storage, processing, and sharing. Cloud technology is a general term encompassing network technology, information technology, integration technology, management platform technology, and application technology applied to the cloud computing business model. It can form resource pools, providing flexible and convenient on-demand access. Cloud computing technology will become a crucial support. Backend services of technical network systems require substantial computing and storage resources, such as video websites, image websites, and many portal websites. With the rapid development and application of the internet industry, every item may have its own identification mark in the future, requiring transmission to backend systems for logical processing. Data at different levels will be processed separately, and each industry's data will require robust system support, which can only be achieved through cloud computing.

[0039] 3) Response: used to indicate the conditions or states on which the operation is performed. When the conditions or states on which the operation is performed are met, one or more operations may be performed in real time or with a set delay. Unless otherwise specified, there is no restriction on the order in which the multiple operations are performed.

[0040] 4) Cluster technology: This technology has emerged recently with the development of the Internet. It connects a group of independent computers through a high-speed network to form a cluster system that can be uniformly managed and scheduled to handle various big data scenarios, thereby improving the scalability and reliability of services. To ensure the normal operation of big data scenarios, cluster systems typically store massive amounts of data to support each type of business within the big data scenario.

[0041] 5) Pod: A Pod is the smallest unit that can be created and managed in Kubernetes. It is a collection of containers that share the same network namespace and storage volumes. A Pod typically contains one or more containers, along with shared resources for inter-container communication and storage. Containers within a Pod share the same network interface and storage volumes, meaning they can communicate with each other via localhost and access the same storage resources. Pods have their own lifecycle, including states such as creation, running, paused, resumed, and terminated. Pods are used to deploy and manage applications; they are the basic unit for Kubernetes to schedule and manage compute tasks.

[0042] 6) Business Nodes: Business nodes typically refer to components or services that perform specific data processing functions. They are responsible for executing corresponding logical processing based on access requests, such as data validation, business logic calculations, and data storage. A business node may be a module within an application or an external service.

[0043] 7) Remote Dictionary Service (RDS): Commonly known as Redis, it is a high-performance key-value store. It can be used as a database, cache, and message broker middleware. Redis is open-source, written in ANSI C, and licensed under the BSD license. Technically, Redis is an in-memory data structure store that supports various data structures, such as strings, lists, sets, sorted sets (ZSet, also known as SortedSet), and hashes. These data structures enable Redis to handle every complex data manipulation task. These features make Redis suitable for scenarios requiring fast access and processing of large amounts of data, such as distributed caching, session caching, message queues, leaderboards, and social networks. Due to its high performance and high availability, Redis has become an indispensable part of many large systems and applications.

[0044] 8) Nacos: An open-source service discovery and configuration management platform developed by Alibaba Group for building applications based on microservice architectures. Nacos's name comes from the abbreviation of "Naming and Configuration Service." It provides a unified system for managing and configuring services, as well as dynamic discovery between services. In a microservice architecture, services typically run on different servers or containers. Nacos's role is to manage the registration and discovery of these services, ensuring that services can communicate correctly with each other. Key features include: Service discovery and registration: Nacos allows services to register themselves at startup and allows other services to query and discover these registered services for communication. Dynamic configuration management: Nacos centrally manages service configuration information, supporting dynamic updates and pushes, so configuration changes take effect in real time without restarting the service. Service management: Nacos provides features such as service grouping, namespace management, service weight adjustment, and routing rule settings, helping developers better control service traffic and load balancing. High availability: Nacos supports cluster mode; multiple Nacos nodes can be deployed together to form a highly available cluster, ensuring efficient and stable service discovery and configuration management. Easy to integrate: Nacos integrates well with mainstream microservice frameworks such as Spring Cloud and Dubbo, simplifying the complexity for developers to introduce Nacos into existing architectures. Multi-environment support: Nacos supports multi-environment configurations, allowing management and pushing of different configuration information based on different environments (e.g., development, testing, production). Lightweight and easy to deploy: As a lightweight platform, Nacos is easily integrated into various cloud-native environments, such as Kubernetes and Docker.

[0045] During the implementation of the embodiments of this application, the applicant discovered the following problems with the related technology:

[0046] In related technologies, for the data source connection of business nodes, the business nodes usually connect to a fixed data source for data processing. However, since the fixed data source cannot meet the data processing requirements during the data processing process, the processing efficiency of the business nodes is low.

[0047] This application provides a data processing method, apparatus, electronic device, computer-readable storage medium, and computer program product, which can effectively improve the data processing efficiency of business nodes. The following describes an exemplary application of the data processing system provided in this application.

[0048] See Figure 1 , Figure 1This is a schematic diagram of the architecture of the data processing system 100 provided in the embodiments of this application. The terminal (terminal 400 is shown as an example) connects to the server 200 through the network 300. The network 300 can be a wide area network or a local area network, or a combination of the two.

[0049] Terminal 400 is used by users to access client 410 and display test scripts on a graphical interface 410-1 (graphical interface 410-1 is shown as an example). Terminal 400 and server 200 are interconnected via wired or wireless network.

[0050] In some embodiments, server 200 can be a standalone physical server, a server cluster or business system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms. Terminal 400 can be a smartphone, tablet, laptop, desktop computer, smart speaker, smart TV, smartwatch, in-vehicle terminal, etc., but is not limited to these. The electronic device provided in this application embodiment can be implemented as a terminal or a server. The terminal and server can be directly or indirectly connected via wired or wireless communication, which is not limited in this application embodiment.

[0051] In some embodiments, in response to the fact that the received first scheduling information is different from the second scheduling information stored locally on the service node, the server 200 determines the second data source of the service node based on the first scheduling information, and switches the first data source connected to the service node to the second data source based on the access address.

[0052] In other embodiments, in response to the fact that the received first scheduling information is different from the second scheduling information stored locally by the service node, the terminal 400 determines the second data source of the service node based on the first scheduling information, and switches the first data source connected to the service node to the second data source based on the access address.

[0053] In other embodiments, the embodiments of this application can be implemented with the aid of cloud technology, which refers to a hosting technology that unifies a series of resources such as hardware, software, and networks within a wide area network or local area network to realize the computation, storage, processing, and sharing of data.

[0054] Cloud technology is a general term encompassing network technology, information technology, integration technology, management platform technology, and application technology based on the cloud computing business model. It can form resource pools, allowing for on-demand use with flexibility and convenience. Cloud computing technology will become a crucial support. The backend services of cloud computing systems require substantial computing and storage resources.

[0055] See Figure 2 , Figure 2 This is a schematic diagram of the structure of the electronic device 500 for data processing provided in the embodiments of this application. Figure 1 ,in, Figure 2 The electronic device 500 shown can be Figure 1 Server 200 or terminal 400 in the middle, Figure 2 The illustrated electronic device 500 includes at least one processor 430, a memory 450, and at least one network interface 420. Each component in the electronic device 500 is coupled together via a bus system 440. It is understood that the bus system 440 is used to implement communication between these components. In addition to a data bus, the bus system 440 also includes a power bus, a control bus, and a status signal bus. However, for clarity, ... Figure 2 The lieutenant general labeled each bus as Bus System 440.

[0056] Processor 430 can be an integrated circuit chip with signal processing capabilities, such as a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Among them, the general-purpose processor can be a microprocessor or any conventional processor, etc.

[0057] The memory 450 may be removable, non-removable, or a combination thereof. Exemplary hardware devices include solid-state storage, hard disk drives, optical disk drives, etc. The memory 450 may optionally include one or more storage devices physically located away from the processor 430.

[0058] The memory 450 may include volatile memory or non-volatile memory, or both. The non-volatile memory may be read-only memory (ROM), and the volatile memory may be random access memory (RAM). The memory 450 described in this application embodiment is intended to include any suitable type of memory.

[0059] In some embodiments, memory 450 is capable of storing data to support each operation, examples of which include programs, modules, and data structures or subsets or supersets thereof, as illustrated below.

[0060] Operating system 451 includes system programs for handling each basic system service and performing hardware-related tasks, such as the framework layer, core library layer, driver layer, etc., for implementing each basic business and handling hardware-based tasks;

[0061] The network communication module 452 is used to reach other electronic devices via one or more (wired or wireless) network interfaces 420, such as Bluetooth, WiFi, and Universal Serial Bus (USB).

[0062] In some embodiments, the data processing apparatus provided in this application can be implemented in software. Figure 2 A data processing device 455 stored in memory 450 is shown. This device can be software in the form of programs and plug-ins, and includes the following software modules: a determination module 4551 and a switching module 4552. These modules are logically linked and can therefore be arbitrarily combined or further split according to the functions they implement. The function of each module will be described below.

[0063] See Figure 3 , Figure 3 This is a schematic diagram of the structure of an electronic device for data processing provided in an embodiment of this application. Figure 2 ,in, Figure 3 The electronic device 600 shown can be Figure 1 Server 200 or terminal 400 in the middle, Figure 3 The illustrated electronic device 600 includes at least one processor 530, a memory 550, and at least one network interface 520. Each component in the electronic device 600 is coupled together via a bus system 540. It is understood that the bus system 540 is used to implement communication between these components. In addition to a data bus, the bus system 540 also includes a power bus, a control bus, and a status signal bus. However, for clarity, ... Figure 3 The Lieutenant General designated each bus as Bus System 540.

[0064] Processor 530 can be an integrated circuit chip with signal processing capabilities, such as a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Among them, the general-purpose processor can be a microprocessor or any conventional processor.

[0065] The memory 550 may be removable, non-removable, or a combination thereof. Exemplary hardware devices include solid-state storage, hard disk drives, optical disk drives, etc. The memory 550 may optionally include one or more storage devices physically located away from the processor 530.

[0066] The memory 550 may include volatile memory or non-volatile memory, or both. The non-volatile memory may be read-only memory (ROM), and the volatile memory may be random access memory (RAM). The memory 550 described in this application embodiment is intended to include any suitable type of memory.

[0067] In some embodiments, memory 550 is capable of storing data to support each operation, examples of which include programs, modules, and data structures or subsets or supersets thereof, as illustrated below.

[0068] Operating system 551 includes system programs for handling each basic system service and performing hardware-related tasks, such as the framework layer, core library layer, driver layer, etc., for implementing each basic business and handling hardware-based tasks;

[0069] The network communication module 552 is used to reach other electronic devices via one or more (wired or wireless) network interfaces 520, such as Bluetooth, WiFi, and Universal Serial Bus (USB).

[0070] In some embodiments, the data processing apparatus provided in this application can be implemented in software. Figure 3 A data processing device 555 stored in memory 550 is shown. This device can be software in the form of programs and plug-ins, and includes the following software modules: a generation module 5551 and a transmission module 5552. These modules are logically linked and can therefore be arbitrarily combined or further divided according to the functions they implement. The function of each module will be described below.

[0071] In other embodiments, the data processing apparatus provided in this application can be implemented in hardware. As an example, the data processing apparatus provided in this application can be a processor in the form of a hardware decoding processor, which is programmed to execute the data processing method provided in this application. For example, the processor in the form of a hardware decoding processor can be one or more application-specific integrated circuits (ASICs), DSPs, programmable logic devices (PLDs), complex programmable logic devices (CPLDs), field-programmable gate arrays (FPGAs), or other electronic components.

[0072] In some embodiments, the terminal or server can implement the data processing method provided in this application by running a computer program or computer-executable instructions. For example, the computer program can be a native program in the operating system (e.g., a dedicated data processing program) or a software module, such as a data processing module that can be embedded in any program (e.g., an instant messaging client, a photo album program, an electronic map client, a navigation client); or it can be a native application (APP), i.e., a program that needs to be installed in the operating system to run. In summary, the above-mentioned computer program can be any form of application, module, or plugin.

[0073] The data processing method provided in this application will be described in conjunction with exemplary applications and implementations of the server or terminal provided in the embodiments of this application.

[0074] See Figure 4 , Figure 4 This is a flowchart illustrating the data processing method provided in the embodiments of this application. Figure 1 , will combine Figure 4 Steps 101 to 102 are described below. The data processing method provided in this application embodiment can be implemented by the server or the terminal alone, or by the server and the terminal working together. The following description will take the implementation by the server alone as an example.

[0075] In step 101, in response to the fact that the received first scheduling information is different from the second scheduling information stored locally by the service node, the second data source of the service node is determined based on the access addresses of multiple data sources included in the first scheduling information.

[0076] In some embodiments, the first scheduling information carries access addresses of multiple data sources, including a second data source. The first scheduling information is generated during the process of a business node performing business processing based on the initial data source connection.

[0077] In some embodiments, the first data source refers to the data source that the business node connects to when it starts processing business. It can be a database, file system, API interface, etc., used to provide the data required for data processing.

[0078] In some embodiments, the first data source refers to the data source that the business node connects to when it starts processing business. It can be a database, file system, API interface, etc., used to provide the data required for data processing.

[0079] In some embodiments, the second scheduling information is a set of scheduling rules or configuration information stored locally by the service node. This information determines how the service node reads and writes data sources when processing services, including the order of data reading and the target of data writing.

[0080] In some embodiments, the first scheduling information refers to scheduling instructions or adjustment information dynamically sent to the business node during the business node's processing. This information may include data source switching rules, priority adjustments, changes in data processing order, etc., used to adjust the data processing logic of the business node in real time.

[0081] In some embodiments, during the process of a business node connecting to the first data source for data processing, if the received first scheduling information is inconsistent with the locally stored second scheduling information, the business node needs to determine a new second data source based on the first scheduling information in order to switch to the new data source to continue performing data processing. This process ensures that the business node can flexibly adjust the data processing logic according to the latest scheduling requirements to meet changing business needs.

[0082] In some embodiments, a service node connects to a first data source for data processing based on second scheduling information stored locally. If the received first scheduling information is different from the second scheduling information stored locally by the service node, it indicates that the service node cannot connect to the first data source for data processing based on the second scheduling information stored locally, and needs to switch to the second data source for data processing based on the received first scheduling information.

[0083] In some embodiments, the service node and the data source have multiple communication modes. The first scheduling information includes the first sub-scheduling information corresponding to each communication mode, and the second scheduling information includes the second sub-scheduling information corresponding to each communication mode.

[0084] In some embodiments, the communication mode of a business node is used to indicate the connection method of the business node to the data source. For example, the communication mode can be a single-node communication mode, which indicates that the business node can only connect to a single data source, or a cluster communication mode, which indicates that the business node can connect to a data source cluster that includes multiple data sources.

[0085] As an example, the service node includes a first communication mode and a second communication mode. The first scheduling information includes a first sub-scheduling information corresponding to the first communication mode and a first sub-scheduling information corresponding to the second communication mode. The second scheduling information includes a second sub-scheduling information corresponding to the first communication mode and a second sub-scheduling information corresponding to the second communication mode.

[0086] In some embodiments, before determining the second data source of a service node based on the access addresses of multiple data sources included in the first scheduling information, it can be determined whether the first scheduling information and the second scheduling information are the same in the following way: for each communication mode, the first sub-scheduling information corresponding to the communication mode and the second sub-scheduling information corresponding to the communication mode are compared. If the first sub-scheduling information and the second sub-scheduling information are different, it is determined that the first scheduling information and the second scheduling information are different.

[0087] In some embodiments, if the first sub-scheduling information and the second sub-scheduling information are the same in each of the communication modes, then it is determined that the first scheduling information and the second scheduling information are the same.

[0088] In some embodiments, the first sub-scheduling information includes at least one first information field, and the second sub-scheduling information includes a second information field that corresponds one-to-one with the first information field.

[0089] In some embodiments, the first sub-scheduling information and the second sub-scheduling information corresponding to the communication mode are compared. If the first sub-scheduling information and the second sub-scheduling information are different, it is determined that the first scheduling information and the second scheduling information are different. This can be achieved by comparing the first information field with the corresponding second information field. If the first information field and the corresponding second information field are different, it is determined that the first scheduling information and the second scheduling information are different.

[0090] In some embodiments, two data structures are defined to represent the first sub-scheduling information and the second sub-scheduling information, respectively. Each sub-scheduling information contains a set of information fields, which should correspond one-to-one in the two sub-scheduling information. A comparison logic is implemented that compares the corresponding information fields in the first and second sub-scheduling information sequentially according to a certain order or rule (e.g., field order). When comparing each corresponding information field, their values ​​are checked for equality. If any pair of field values ​​is found to be unequal, the comparison operation is stopped immediately, as this is sufficient to prove that the first and second sub-scheduling information are different. If all information fields have been compared and are equal, it can be determined that the two sub-scheduling information are the same. If any field is unequal, according to the comparison logic, we can determine that the first and second sub-scheduling information are different. Based on the comparison results of the sub-scheduling information, it can be inferred whether the first and second scheduling information are different. If the sub-scheduling information is different, then it can be concluded that the first and second scheduling information are also different. By comparing each information field in the two sub-scheduling information one by one, the difference between the scheduling information and the second sub-scheduling information can be determined quickly and accurately. This comparison method ensures that as long as differences exist, a conclusion can be reached in the shortest possible time, thus improving efficiency.

[0091] In some embodiments, it is necessary to determine all possible communication modes for the business node. Communication modes may involve direct connection to a single data source or connection to multiple data sources via middleware, proxies, etc. For each communication mode, the current first sub-scheduling information needs to be obtained from the real-time scheduling system. This information may include the current state, priority, and load of the data source. Simultaneously, the corresponding second sub-scheduling information needs to be obtained from the business node's local storage. This information is loaded from the global scheduling system at startup or at some previous time. For each communication mode, the obtained first and second sub-scheduling information are compared. The comparison is based on factors such as scheduling rules, data source state, and priority. An information comparison result is generated. This result is a representation indicating whether the first and second sub-scheduling information are the same. All information comparison results are checked. If any information comparison result indicates that the first and second sub-scheduling information are different, then it can be determined that the first and second scheduling information are different. If all information comparison results indicate that the first and second sub-scheduling information are the same, then it can be determined that the first and second scheduling information are the same. If the first scheduling information differs from the second scheduling information, the business node needs to update its second scheduling information based on the first. This may include updating the data source priority, switching to a new data source, and adjusting data processing logic. If the first and second scheduling information are the same, the business node can continue to use the current data source and scheduling rules for data processing without any changes. Once the business node has updated its second scheduling information based on the first, it needs to update this new scheduling information to local storage so that the correct scheduling information can be loaded the next time it starts or executes.

[0092] In this way, business nodes can ensure that they adjust their data processing logic based on the latest primary scheduling information, thereby guaranteeing the efficient and accurate operation of the business. At the same time, this also allows business nodes to flexibly adapt to changes in the scheduling system, improving the system's reliability and flexibility.

[0093] In some embodiments, during the process of the business node connecting to the first data source to perform the data processing, the following processing may also be performed: in response to the fact that the received first scheduling information is the same as the second scheduling information, data processing continues based on the first data source.

[0094] In some embodiments, during the process of a business node connecting to the first data source for data processing, in response to the first scheduling information being the same as the second scheduling information received, when the first scheduling information received is the same as the second scheduling information stored locally by the business node, it indicates that the business node can continue to connect to the first data source for data processing based on the second scheduling information stored locally.

[0095] In some embodiments, if the comparison result shows that the first scheduling information is the same as the second scheduling information, it indicates that the current scheduling information has not changed. This may be because new scheduling information has not yet been published, or the current business node is already in the latest scheduling state. Since the scheduling information has not changed, the business node can continue to use the first data source for data processing. This means that the business node does not need to switch data sources and can execute tasks according to the original scheduling rules and data processing logic. Although the current scheduling information has not changed, the business node usually needs to continuously monitor the first scheduling information so that it can respond promptly once the scheduling information changes. Even if the first scheduling information is the same as the second scheduling information, the business node should update the second scheduling information periodically or according to specific trigger conditions. This ensures that the business node always has the latest scheduling rules and configurations. The business node can ensure that its processing logic is consistent with the latest scheduling requirements, and can also quickly respond to any changes in scheduling information, thereby improving the flexibility and reliability of the business.

[0096] In some embodiments, before determining the second data source of the service node based on the access addresses of multiple data sources included in the first scheduling information, the following processing may also be performed: based on the second scheduling information, perform permission authentication on the first scheduling information to obtain the permission authentication result.

[0097] In some embodiments, the permission authentication result is used to indicate whether the first scheduling information has data source scheduling permission for the service node.

[0098] In some embodiments, a business node reads the current storage scheduling information from local storage. This information includes access permissions, operation permissions, priorities, etc., of the data source. The business node receives the latest first scheduling information through a scheduling system interface or other mechanisms. This information may include updates to the data source status, priority, processing logic, etc. The business node uses the locally stored scheduling information as a reference to authenticate the first scheduling information. This involves verifying whether the permission parameters in the first scheduling information match the permission configuration of the local storage. The authentication result is an indication that the first scheduling information has the permission to schedule the data source of the current business node. If the permission parameters of the first scheduling information match the permission configuration in the second scheduling information, the authentication result is "passed"; otherwise, it is "failed." If the authentication result is "passed," it means that the first scheduling information has the permission to schedule the data source of the business node. The business node can then perform corresponding scheduling operations, such as switching data sources or adjusting processing logic. If the authentication result is "failed," it means that the first scheduling information does not have the permission to schedule the data source of the business node. The business node should not perform any scheduling operations based on this first scheduling information but should continue to process data according to the locally stored scheduling information.

[0099] In some embodiments, the above-mentioned determination of the second data source of the business node based on the access addresses of the multiple data sources included in the first scheduling information can be implemented in the following way: if the permission authentication result indicates that the first scheduling information has the permission to schedule the data source of the business node, then the second data source of the business node is determined based on the access addresses of the multiple data sources included in the first scheduling information.

[0100] In some embodiments, the business node checks the authorization authentication result to determine whether the first scheduling information has scheduling authority over the current business node's data source. If the authorization authentication result indicates that the first scheduling information has scheduling authority, the business node further analyzes the first scheduling information to determine whether a second data source needs to be switched. This may involve comparing factors such as data source priority, load, and availability in the first scheduling information. If the first scheduling information indicates that a data source switch is needed, the business node determines a new second data source based on the access addresses of multiple data sources included in the first scheduling information, which may be the data source with the highest priority, lowest load, or highest availability. Once the business node determines the new second data source, it needs to update the scheduling information in local storage to ensure that the correct scheduling information can be loaded on the next startup or execution. The business node performs a data source switching operation based on the new second data source. This may include disconnecting from the current data source, establishing a connection to the new data source, reinitializing the data access interface, etc. For tracking and auditing, the business node should record the process and results of the data source switching. This helps in troubleshooting and performance analysis when problems occur. After the data source switching is completed, the business node continues to process data based on the new second data source. Business nodes can ensure that their scheduling operations are performed within their authorized scope and can flexibly switch data sources based on initial scheduling information when needed, thereby improving system reliability and flexibility. Simultaneously, this enables business nodes to quickly respond to changes in the scheduling system, guaranteeing efficient and accurate business operations.

[0101] In some embodiments, the first scheduling information includes first authentication information, and the second scheduling information includes second authentication information.

[0102] In some embodiments, the above-mentioned authentication of the first scheduling information based on the second scheduling information to obtain an authentication result can be achieved in the following manner: comparing the first authentication information and the second authentication information; if the first authentication information and the second authentication information are the same, then the authentication result indicates that the first scheduling information has data source scheduling authority over the business node; if the first authentication information and the second authentication information are different, then the authentication result indicates that the first scheduling information does not have data source scheduling authority over the business node.

[0103] In some embodiments, the business node obtains the information to be authenticated from the real-time scheduling system. This information may include a signature of the scheduling information, encrypted authentication information, or other data used to verify identity and permissions. The business node obtains second authentication information from local storage or an authentication service. This information is predefined and serves as a reference standard for comparison with the first authentication information. The business node compares the first authentication information with the second authentication information. The comparison process may involve verifying the legality of the signature, the integrity of the authentication information, or other security verification steps. If the information comparison result indicates that the first authentication information and the second authentication information are the same, the business node can determine that the first scheduling information has the authority to schedule the business node's data source. In this case, the authorization authentication result is "passed." If the information comparison result indicates that the first authentication information and the second authentication information are different, the business node can determine that the first scheduling information does not have the authority to schedule the business node's data source. In this case, the authorization authentication result is "failed." Based on the authorization authentication result, the business node updates its authorization authentication status. If authentication passes, the business node can continue to perform scheduling operations; if authentication fails, the business node should comply with existing scheduling rules and not perform any new scheduling operations. For tracking and auditing purposes, the business node should record the authentication process and results. This helps with troubleshooting and performance analysis when problems occur.

[0104] In this way, business nodes can ensure that only the first scheduling information with the correct permissions can schedule the data source, thereby protecting data security and business compliance. At the same time, this also allows business nodes to flexibly adapt to changes in the scheduling system, improving system reliability and flexibility.

[0105] In some embodiments, after performing permission authentication on the first scheduling information based on the second scheduling information and obtaining the permission authentication result, the following processing can also be performed: if the permission authentication result indicates that the first scheduling information does not have the data source scheduling permission for the business node, then the data processing continues based on the first data source.

[0106] In some embodiments, the business node reads the current storage scheduling information from local storage. This information includes access permissions, operation permissions, priorities, etc., of the data source. The business node receives the latest first scheduling information through the scheduling system interface or other mechanisms. This information may contain updates to the data source status, priority, processing logic, etc. The business node uses the local storage scheduling information as a reference to authenticate the first scheduling information. This involves verifying whether the permission parameters in the first scheduling information match the permission configuration of the local storage. If the permission parameters of the first scheduling information match the permission configuration in the second scheduling information, the authentication result is "passed"; otherwise, it is "failed". If the authentication result is "failed", it means that the first scheduling information does not have the permission to schedule the data source of the business node. The business node should not perform any scheduling operations based on the first scheduling information. Since the authentication result is "failed", the business node continues to process data based on the first data source. This means that the business node will not switch data sources based on the first scheduling information, but will continue to use the first data source to execute its business logic. Although the current authentication result is "failed", the business node usually needs to continuously monitor changes in the first scheduling information so that it can respond promptly once the scheduling information changes. Business nodes should record the results of authentication and the data processing performed based on those results. This helps with troubleshooting and performance analysis when problems occur.

[0107] In this way, business nodes can ensure that their scheduling operations are performed within their authorized scope, thereby protecting data security and business compliance. Simultaneously, this allows business nodes to flexibly adapt to changes in the scheduling system, improving system reliability and flexibility. By authenticating the initial scheduling information, business nodes can ensure that only information with correct permissions can schedule the data source, thus protecting data security and business compliance. Business nodes can flexibly adjust their processing logic based on the authentication results, responding to updates to the initial scheduling information while preventing unauthorized information from interfering with business processes. Even when the initial scheduling information changes, business nodes can continue to stably execute data processing based on the initial data source according to the authentication results, ensuring business continuity and consistency. Recording the authentication process and processing results facilitates tracking and auditing, enabling rapid troubleshooting and performance analysis in case of problems. Authentication ensures that only authenticated information can schedule business nodes, improving the overall reliability and stability of the system.

[0108] In some embodiments, the service node includes multiple communication modes, and the first scheduling information includes first sub-scheduling information corresponding to each communication mode. The first sub-scheduling information includes switch information for characterizing whether the communication mode is started, and access address of at least one data source in the communication mode.

[0109] In some embodiments, the above-mentioned determination of the second data source of the service node based on the first scheduling information can be implemented in the following way: for each of the first sub-scheduling information, if the switch information in the first sub-scheduling information indicates that the communication mode is started, then at least one data source in the communication mode is determined as the second data source.

[0110] In some embodiments, a business node sets up multiple communication modes upon startup, each corresponding to a different data source access strategy and priority. The business node receives first scheduling information containing first sub-scheduling information for each communication mode through a scheduling system or other mechanism. The business node parses the first sub-scheduling information within the first scheduling information; each communication mode has a corresponding set of first sub-scheduling information. For each communication mode's first sub-scheduling information, the business node checks switch information. If the switch information in the first sub-scheduling information of a certain communication mode indicates that the mode is started, then at least one data source under that communication mode will be identified as a second data source. This means that the business node will connect to this data source for data processing. Once the second data source is identified, the business node connects to it and performs data processing and business logic based on it. The business node records the information about connecting to the second data source and the data processing performed for subsequent auditing and analysis.

[0111] In some embodiments, before performing step 101 above, the first scheduling information can also be obtained by: sending a scheduling information acquisition request at a preset time interval; and receiving the first scheduling information returned in response to the scheduling information acquisition request.

[0112] In some embodiments, a service node first receives first scheduling information, which contains access addresses for multiple data sources. The service node compares the received first scheduling information with its locally stored second scheduling information. Through comparison, the service node detects a difference between the first and second scheduling information. Based on the detected difference, the service node initiates a scheduling information retrieval process. The first step of this process is to send a scheduling information retrieval request at preset time intervals. This time interval can be fixed, such as sending a request every 10 seconds, or it can be a dynamic interval based on certain conditions. The purpose of sending the scheduling information retrieval request is to obtain the latest scheduling information from the relevant scheduling management node or other service nodes to ensure that the local scheduling information is up-to-date, or to resolve inconsistencies between the local scheduling information and the received scheduling information. The service node receives a response to the scheduling information retrieval request, which is the new first scheduling information. This information should contain the latest or correct scheduling information. After receiving the new first scheduling information, the service node processes it, such as updating its local second scheduling information to ensure that the local scheduling information is consistent with the latest scheduling information. After obtaining the new scheduling information, the service node may determine the second data source based on the data source access addresses contained in the new scheduling information.

[0113] In some embodiments, before performing step 102 below, the scheduling information stored locally on the service node can be updated by storing the first scheduling information locally on the service node and deleting the second scheduling information stored locally on the service node.

[0114] In some embodiments, the business node needs to determine why the scheduling information needs to be updated. This could be because the received first scheduling information is inconsistent with the second scheduling information stored locally, or because there are new scheduling instructions to be executed. Before deciding to update, the business node needs to verify the validity of the received first scheduling information. This may include checking the source, format, signature, etc., of the information to ensure that the information is reliable and correct. The business node writes the complete content of the first scheduling information to local storage, ensuring that all data is correctly saved. After the storage operation is completed, the business node may need to perform some confirmation operations, such as verifying whether the stored information is consistent with the original information, to ensure data integrity. The business node needs to locate the second scheduling information in local storage. Once the second scheduling information is found, the business node will perform a deletion operation. This may include deleting records from the database, deleting files from the file system, etc. After performing the deletion operation, the business node needs to confirm that the second scheduling information has been completely deleted without leaving any data. The business node may need to update the local status flag to indicate that the current scheduling information is the latest version of the first scheduling information. This ensures that the business node can safely and effectively update its locally stored scheduling information and maintain the consistency and normal operation of the entire system.

[0115] In step 102, based on the access address of the second data source, the first data source connected to the service node is switched to the second data source.

[0116] In some embodiments, step 102 above can be implemented as follows: if the second data source is a data source cluster and the data source cluster includes the first data source, then a data connection with the third data source other than the first data source in the data source cluster is established based on the access address of the third data source in the data source cluster, while maintaining the connection with the first data source; if the second data source is the data source cluster and the data source cluster does not include the first data source, then the data connection with the first data source is disconnected, and a data connection with the data source cluster is established based on the access address.

[0117] In some embodiments, the business node determines the second data source based on the first scheduling information. If the second data source is a data source cluster and the data source cluster includes the first data source, then the connection with the first data source is maintained. If the second data source is a data source cluster and the data source cluster does not include the first data source, then the connection with the first data source needs to be disconnected. Data connections are established with third data sources other than the first data source in the data source cluster based on their access addresses. After establishing connections with the third data sources in the data source cluster, these data source connections are kept active so that the business node can read or write data from them at any time. The business node records the current data source connection status, including which data source it has established a connection with and the connection status information (such as connection activity, read / write performance, etc.). After completing the data source connection switch, the business node performs corresponding data processing based on the new data source connection. This may include data reading, processing, writing, and other operations. The business node continuously monitors the connection status with each data source in the data source cluster to ensure connection stability and data availability. The business node remains sensitive to changes in the first scheduling information; once new scheduling information is received, it immediately switches the data source connection according to its representation.

[0118] As an example, consider an online shopping platform's order processing system connected to a database cluster. When a database in the cluster experiences performance degradation due to high load, the real-time scheduling system may instruct the business node to switch to another database within the cluster. At this point, the business node will disconnect from the original database and establish a new connection based on the access address of the other database to ensure the continuity and performance of order processing.

[0119] As an example, in a content delivery network (CDN) application scenario, a website's server might be connected to a CDN cluster. When a node in the CDN cluster fails, the real-time scheduling system instructs the service node to switch to another healthy node. The service node will disconnect from the failed node and re-establish a connection based on the access address of the other node to maintain the rapid distribution and availability of website content.

[0120] As an example, in cloud computing applications, an application may be deployed across multiple cloud service instances. When a cloud service instance experiences a problem, the real-time scheduling system may indicate that the application is migrating to another healthy instance. The business node will disconnect from the failed cloud service instance and establish a new connection based on the access address of the other healthy instance to ensure the continuous operation of the application and the availability of services.

[0121] As an example, in financial trading systems, transaction records may be stored across multiple data centers or cloud services to improve system reliability and scalability. When a data center or cloud service experiences a problem, the real-time scheduling system will instruct business nodes to switch to other working data centers or cloud services. The business nodes will disconnect from the failed data center and re-establish a connection based on the access address of the other working data center or cloud service to ensure the accuracy of transaction records and the continuity of transactions.

[0122] In this way, by dynamically switching between different data sources, business nodes can utilize the optimal performance of each data source, thereby improving overall data processing speed and efficiency. When a new data source is added to the data source cluster, the system can automatically switch based on the first scheduling information without manual intervention, thus easily expanding system capacity and processing power. The first scheduling information provides flexibility in data source selection, allowing business nodes to dynamically adjust data source usage strategies based on actual load, performance, or priority. Even if a data source fails, the first scheduling information can guide business nodes to switch to other normal data sources, thereby reducing the impact of system failures and ensuring business continuity. The first scheduling information can guide business nodes to select data sources based on the current load of the data sources, thereby avoiding resource waste and achieving more efficient resource utilization. Through the first scheduling information, business nodes can automatically manage complex data source configurations and switching, reducing manual intervention and management complexity. The first scheduling information can help business nodes avoid accessing insecure or non-compliant data sources, ensuring data security and compliance with regulatory requirements.

[0123] In some embodiments, step 102 above can also be implemented as follows: if the first data source is a data source cluster and the data source cluster includes the second data source, then based on the access address, the data connection with the third data source other than the second data source in the data source cluster is disconnected, while the data connection with the second data source is maintained; if the first data source is the data source cluster and the data source cluster does not include the second data source, then the data connection with the data source cluster is disconnected, and based on the access address, a data connection with the second data source is established.

[0124] In some embodiments, the business node identifies the second data source based on the first scheduling information. The business node checks whether the first data source is part of a data source cluster and whether it contains the second data source. If the first data source is part of the data source cluster and contains the second data source, then the data connection with the second data source is maintained, and data connections with other non-second data sources are disconnected. If the first data source is part of the data source cluster but does not contain the second data source, then the data connection with the entire data source cluster is disconnected, and a data connection with the second data source is established. Based on the access address of the second data source, the corresponding data connection is disconnected or established. This may include closing old connections, starting new connections, configuring connection parameters, etc. After completing the data source connection switch, the business node performs corresponding data processing based on the new data source connection. This may include data reading, processing, writing, etc. The business node continuously monitors the connection status with the second data source to ensure connection stability and data availability. The business node remains sensitive to changes in the first scheduling information; once new scheduling information is received, it immediately switches the data source connection according to its representation.

[0125] As an example, in an online retail platform, the order processing system connects to a database cluster. When a database node experiences performance degradation due to high load, the real-time scheduling system might instruct the business node to switch to a different, higher-performing database node as a secondary data source. At this point, the business node will disconnect from all other database nodes in the cluster except the target database, while maintaining its connection to the target database to optimize order processing performance.

[0126] As an example, in a cloud computing environment, a web application may be deployed across multiple cloud service instances. When one of these instances fails, the real-time scheduling system may instruct the business node to switch to a secondary data source from another healthy cloud service instance. The business node will disconnect from the failed cloud service instance and establish a new connection with the target cloud service instance to ensure the continuity of web application services.

[0127] As an example, in a Content Delivery Network (CDN) scenario, a website's server might connect to a CDN cluster. When a node in the CDN cluster fails, the real-time scheduling system instructs the service node to switch to another healthy node. The service node will disconnect from the failed node and establish a connection with the target healthy node to ensure rapid distribution and availability of website content.

[0128] As an example, in a financial trading system, transaction records may be stored across multiple data centers or cloud services to improve system reliability and scalability. When a data center or cloud service experiences a problem, the real-time scheduling system instructs business nodes to switch to other working data centers or cloud services. The business node will disconnect from the failed data center and establish a connection with the target working data center or cloud service to ensure the accuracy of transaction records and the continuity of transactions.

[0129] Thus, by disconnecting or establishing connections based on the access address of the second data source, business nodes can more effectively utilize resources in the data source cluster, avoiding resource waste. By maintaining connections to the second data source and disconnecting connections to other non-second data sources, business nodes can optimize data processing performance and improve system response speed. In cases where the second data source is not present in the data source cluster, business nodes can quickly disconnect from faulty or degraded data source clusters and establish connections to the second data source, thereby improving system fault tolerance. Whether maintaining or establishing connections to the second data source, business nodes can ensure business continuity and reduce business interruptions caused by data source issues. Data source connection management guided by the first scheduling information reduces the need for manual intervention and simplifies system management complexity. By selecting the second data source through the first scheduling information, business nodes can ensure that data access and processing are conducted within a secure scope, protecting data from unauthorized access by data sources.

[0130] In some embodiments, during the process of the business node connecting to the first data source to perform the data processing, the following processing may also be performed: sending an information acquisition request to the scheduling node at a preset time interval, and receiving the first scheduling information returned by the scheduling node in response to the information acquisition request.

[0131] In some embodiments, service nodes send information retrieval requests to the scheduling node via a network or other communication mechanism at preset time intervals. This may be an HTTP request, a custom communication protocol message, or other forms of network data packets. Upon receiving the information retrieval request from the service node, the scheduling node processes the request and then returns corresponding first scheduling information. The scheduling node's response may be sent back to the service node via the network in the form of data packets. The service node receives the first scheduling information returned by the scheduling node and performs reception and parsing. This may include steps such as reading network data packets, parsing the returned data format, and extracting the first scheduling information. The service node processes the received first scheduling information. This may include updating its internal state, triggering corresponding logical processing, and logging. After completing one information retrieval request and response cycle, the service node will periodically repeat the information retrieval request at preset time intervals to continuously receive updates to the first scheduling information.

[0132] As an example, suppose there is an online video streaming platform whose backend system consists of multiple data processing nodes. These nodes need to adjust the video content distribution strategy based on real-time user viewing data. To achieve this, the platform maintains a scheduling node, which is responsible for collecting and analyzing data from user behavior and generating initial scheduling information, including video content distribution priorities and data source selection for hotspot areas. Each data processing node sends an HTTP GET request to the scheduling node at preset time intervals (e.g., every minute). The request may contain the node's own ID or other identifying information. Upon receiving the information retrieval request, the scheduling node extracts the latest user viewing data and video content distribution strategy from its database or real-time analysis system and generates the corresponding initial scheduling information. Then, the scheduling node returns the initial scheduling information as a response to the data processing nodes. This response is typically a JSON or XML data packet. After receiving the initial scheduling information from the scheduling node, the data processing nodes parse the data packet and extract the necessary scheduling information, such as the video content distribution priority. Based on the received initial scheduling information, the data processing nodes adjust their own working strategies. For example, if scheduling information shows that a certain video content is very popular in a specific region, the data processing node will prioritize distributing that video content to improve user experience. By periodically sending repeated requests, it is ensured that the latest scheduling information can be continuously obtained, allowing the video content distribution strategy to respond in real time to changes in user behavior.

[0133] Thus, periodically sending information retrieval requests ensures that business nodes continuously receive the latest primary scheduling information, allowing them to adjust their work strategies promptly in response to changes in system status. Business nodes can flexibly adjust their behavior based on the primary scheduling information, such as adjusting data processing priorities, selecting different data sources, and executing specific business logic. Through primary scheduling information, business nodes can detect potential problems, such as data source unavailability or performance degradation, and take corresponding fault-tolerant measures, such as switching to backup data sources or adjusting load balancing strategies. Primary scheduling information helps business nodes optimize resource usage; for example, when a data source is under high load, workloads can be avoided by avoiding allocation to that data source, thereby improving resource utilization efficiency. Through primary scheduling information, business nodes can ensure continuous data processing, enabling rapid recovery even in the event of system failures or data source unavailability. Regularly receiving primary scheduling information helps ensure that business nodes can adapt to continuous changes in system status, thereby improving the stability of the entire system. The automated information retrieval request and processing mechanism simplifies the management complexity of business nodes, reducing the need for manual intervention. Primary scheduling information helps business nodes ensure that the data and behaviors they process comply with security and compliance requirements, such as avoiding the processing of sensitive data or adhering to specific data processing rules. By periodically receiving the first scheduling information, business nodes can respond to changes in system status in real time, improving the system's real-time performance, flexibility, fault tolerance, resource optimization, business continuity, system stability, management simplification, and security.

[0134] In some embodiments, after performing step 102 above, the following processing may also be performed: performing the data processing based on the second data source.

[0135] In some embodiments, after a business node switches from connecting to a first data source to connecting to a second data source, the business node performs the data processing based on the second data source.

[0136] Thus, during the data processing process where a business node connects to the first data source, if the received first scheduling information differs from the locally stored information on the business node, and since the first scheduling information carries access addresses for multiple data sources, including a second data source, the second data source that the business node needs to switch to can be determined based on the first scheduling information. Based on the access address, the first data source connected to the business node is switched to the second data source, and data processing is performed based on the second data source. Therefore, when the received first scheduling information differs from the second scheduling information stored locally on the business node, the second data source for the business node is determined based on the first scheduling information, and the first data source connected to the business node is switched to the second data source based on the access address, and data processing is performed based on the second data source. This achieves real-time hot switching of data sources during data processing, meeting the data processing needs of different data sources and effectively improving the processing efficiency of the business node.

[0137] See Figure 5 , Figure 5 This is a flowchart illustrating the data processing method provided in the embodiments of this application. Figure 2 , will combine Figure 5 Steps 201 to 202 are described below. The data processing method provided in this application embodiment can be implemented by the server or the terminal alone, or by the server and the terminal working together. The following description will take the implementation by the server alone as an example.

[0138] In step 201, the first scheduling information for the service node is generated.

[0139] In some embodiments, the first scheduling information refers to scheduling instructions or adjustment information dynamically sent to the business node during the business node's processing. This information may include data source switching rules, priority adjustments, changes in data processing order, etc., used to adjust the data processing logic of the business node in real time.

[0140] In some embodiments, during the process of a business node connecting to the first data source for data processing, if the received first scheduling information is inconsistent with the locally stored second scheduling information, the business node needs to determine a new second data source based on the first scheduling information in order to switch to the new data source to continue performing data processing. This process ensures that the business node can flexibly adjust the data processing logic according to the latest scheduling requirements to meet changing business needs.

[0141] In step 202, in response to the information acquisition request sent by the service node, the first scheduling information is sent to the service node.

[0142] In some embodiments, the first scheduling information is used to trigger the service node to determine the second data source and switch the connected first data source to the second data source when the first scheduling information is different from the second scheduling information stored locally by the service node.

[0143] In some embodiments, a service node connects to a first data source for data processing based on second scheduling information stored locally. If the received first scheduling information is different from the second scheduling information stored locally by the service node, it indicates that the service node cannot connect to the first data source for data processing based on the second scheduling information stored locally, and needs to switch to the second data source for data processing based on the received first scheduling information.

[0144] In some embodiments, the generation of the first scheduling information for a business node can be achieved as follows: if an adjustment request for the business node is received, the first scheduling information is generated; if first information is received, indicating that the first data source cannot meet the business requirements of the data processing, the second scheduling information is modified according to the business requirements to obtain the first scheduling information.

[0145] In some embodiments, the system continuously listens for adjustment requests from various parties. These requests may originate from the business nodes themselves, or from other system components or the system management layer. When an adjustment request for a specific business node is received, it first performs preprocessing, such as verifying the validity of the request and parsing the request content. If the verification passes, the system generates first scheduling information based on the content of the request. If the trigger condition for generating first scheduling information is receiving an adjustment request for a business node, the system constructs new scheduling information based on the specific content of the request. If the trigger condition for generating first scheduling information is receiving first information indicating that the first data source cannot meet business needs, the system will assess the current business needs and make necessary modifications to the existing second scheduling information. The system analyzes the data provided in the first information to identify the specific reasons why the first data source cannot meet business needs. Based on business needs, the system may adjust resource allocation, data processing strategies, task execution order, etc., in the second scheduling information to ensure that the new scheduling information can meet business needs. After modifying the second scheduling information, the system generates new scheduling information, namely the first scheduling information. The first scheduling information will contain all the adjusted details to guide the business nodes in subsequent data processing and task execution. The generated first scheduling information is then sent to the relevant business nodes. After receiving the initial scheduling information, the business node will perform corresponding operations based on the new scheduling information, such as resource adjustment and task reallocation. It can flexibly generate new scheduling information based on received adjustment requests or the initial information to ensure that the business node can effectively process data according to current business needs and data source conditions.

[0146] As an example, suppose there is a distributed data processing system containing multiple business nodes, each responsible for processing data from different data sources. Here's a specific scenario: Business node A processes data from data source DS1. DS1 is a high-performance database, but its processing capacity is limited. Additionally, there is a second data source, DS2, which is a distributed file storage system with larger storage space and varying processing capabilities. Business node A receives an adjustment request initiated by the system administrator because DS1's data processing capacity is about to reach its limit. This request is detected, and a first scheduling message is generated based on its content. This message instructs business node A to transfer some data processing tasks to DS2 to alleviate the load on DS1. The first scheduling message details which data processing tasks need to be transferred and how these tasks should be allocated on DS2. Business node A begins to reallocate tasks according to the first scheduling message, migrating some data processing work to DS2. While processing data, business node A discovers that DS1 has failed and cannot provide the necessary data processing capacity, generating a first message. The first message indicates that DS1 cannot meet the current business needs, possibly due to hardware failure or a surge in data volume. Based on the first information, the second scheduling information for business node A is modified. The second scheduling information was originally designed based on the operational status of DS1. Modifying the second scheduling information reassigns data processing tasks originally executed on DS1 to DS2 and other available data sources. The modified second scheduling information becomes the new first scheduling information and is sent back to business node A. Upon receiving the new first scheduling information, business node A executes data processing tasks according to the new scheduling strategy, ensuring business continuity and the correctness of data processing.

[0147] In this way, generating initial scheduling information for business nodes effectively improves the system's flexibility and responsiveness. When the system receives adjustment requests from business nodes, it promptly generates new scheduling information, quickly adapting to changes in business needs, optimizing resource allocation, and improving task execution efficiency. Simultaneously, when the initial data source cannot meet business requirements, the system generates new initial scheduling information by modifying the second scheduling information. This not only ensures the continuity and accuracy of data processing but also enhances the system's adaptability to unforeseen circumstances, preventing business interruptions due to data source issues, thereby improving the overall system's stability and reliability.

[0148] In some embodiments, the service node and the data source have multiple communication modes. The generation of the first scheduling information can be achieved as follows: for each communication mode, if the communication mode is the target communication mode, then generate switch information to indicate that the communication mode is started; if the communication mode is not the target communication mode, then generate switch information to indicate that the communication mode is not started; the first scheduling information is generated based on the switch information of each communication mode.

[0149] In some embodiments, the business node first needs to be able to identify multiple communication modes with the data source. Each communication mode may correspond to different data transmission protocols, data formats, transmission rates, etc. One or more target communication modes are defined; these are the communication modes that the business node expects or needs to activate under specific circumstances. The target communication modes may be determined based on factors such as business requirements, data source availability, and network conditions. The business node checks each communication mode with its data source. For each communication mode, the business node determines whether it is a target communication mode. The business node generates a switch message indicating that this communication mode should be activated. This switch message may be a Boolean value, a status code, or a message containing a start instruction. The business node generates a switch message indicating that this communication mode should not be activated. The business node integrates the switch messages for all communication modes to form a complete set of scheduling instructions. Based on the integrated switch messages, the business node generates first scheduling information. This scheduling information will contain detailed instructions on which communication modes should be activated and which should not. The generated first scheduling information is sent to the relevant data source or scheduling management system so that they can adjust their behavior according to this information, such as activating or deactivating specific communication modes. In this way, business nodes can precisely control communication with the data source, ensuring that only the target communication mode is activated, thereby optimizing the efficiency of data transmission and processing, while avoiding unnecessary data traffic and potential security risks.

[0150] As an example, suppose a business node needs to communicate with a data source, supporting three communication modes: single-machine mode, cluster mode, and sentinel mode. Each mode has its corresponding identifier (key), as follows: Single-machine mode: key="single"; Cluster mode: key="colony"; Sentinel mode: key="sentinel". Now, the business node needs to decide which communication mode to use based on current business requirements and environmental conditions. The business node determines the target communication mode as cluster mode based on the current business load, the data source status, and network conditions. For single-machine mode, the business node generates a switch message indicating that single-machine mode is not activated (because the target communication mode is cluster mode). For cluster mode, the business node generates a switch message indicating that cluster mode is activated. For sentinel mode, the business node generates a switch message indicating that sentinel mode is not activated. The business node integrates the generated switch messages into a scheduling information structure. For example, this structure might be a dictionary or a JSON object containing three fields, each corresponding to a communication mode, with the value indicating whether it is activated. The business node sends the generated initial scheduling information to the data source. Upon receiving this information, the data source activates or deactivates the corresponding communication mode according to the instructions. The data source, upon receiving the initial scheduling information, parses the on / off information within it. Based on the business node's requirements, the data source deactivates the communication services in single-machine and sentinel modes, while simultaneously activating the communication service in cluster mode. Through this process, the business node can dynamically control the communication mode with the data source according to current needs, ensuring data processing efficiency and system stability. In this example, cluster mode may offer higher data processing capabilities and better resource utilization, therefore it is selected as the target communication mode.

[0151] Thus, the dynamic scheduling strategy based on target communication modes improves the responsiveness and flexibility to business needs, enabling rapid adjustments to communication modes based on real-time conditions, thereby optimizing data transmission and processing efficiency. Secondly, through explicit on / off information indications, the system can clearly control the status of each communication mode, avoiding unnecessary communication overhead, reducing potential security risks, and ensuring the rational utilization of resources. Furthermore, this implementation method improves system reliability and stability because it can automatically adjust communication strategies based on data source availability and changes in business needs, thus avoiding service interruptions caused by the failure of a single communication mode. Finally, this dynamic scheduling method also facilitates monitoring and maintenance by system administrators, as they can easily understand the currently active communication modes and perform further management and optimization as needed.

[0152] In some embodiments, the generation of first scheduling information based on the switching information of each communication mode can be achieved as follows: for each communication mode, the switching information of the communication mode and the access address of at least one data source in the communication mode are concatenated to obtain the first sub-scheduling information corresponding to the communication mode; each of the first sub-scheduling information is concatenated to obtain the first scheduling information.

[0153] In some embodiments, for each communication mode, the system generates a switch message indicating whether the communication mode should be activated. Simultaneously, the system extracts the access address of the data source associated with that communication mode. Then, the system concatenates this switch message and the data source address to form a complete sub-scheduling message. This sub-scheduling message includes the switch state and the data source address, for example: {"mode":"cluster", "enabled":true", "address":"http: / / cluster.example.com"}. The first sub-scheduling message corresponding to each communication mode is then concatenated to form a unified scheduling message structure. The concatenation can be done by merging all sub-scheduling messages into an array, list, or JSON object. The resulting concatenated information structure is the first scheduling message, which contains the switch states of all communication modes and their corresponding data source addresses. The first scheduling message can be sent to business nodes or other system components so that they can adjust their behavior based on this information. This clearly represents the state and data source address of each communication mode, facilitating appropriate operations by business nodes or other components. Furthermore, this implementation makes the representation of scheduling information more structured and easier to manage, contributing to improved system maintainability and scalability.

[0154] As an example, suppose a business node has three communication modes with a data source: single-machine mode, cluster mode, and sentinel mode. Each mode has its corresponding on / off information and data source access address. Here's an example of how this information is concatenated into the first scheduling information: The business node needs to select the appropriate communication mode based on the current load and network conditions, and communicate with the corresponding data source. Single-machine mode: On / off information is "off", data source address is "http: / / single.example.com". Cluster mode: On / off information is "on", data source address is "http: / / cluster.example.com". Sentinel mode: On / off information is "off", data source address is "http: / / sentinel.example.com". The on / off information and data source address for single-machine mode are concatenated into the string: "single, off, http: / / single.example.com"; the on / off information and data source address for cluster mode are concatenated into the string: "cluster, on, http: / / cluster.example.com"; the on / off information and data source address for sentinel mode are concatenated into the string: "sentinel, off, http: / / sentinel.example.com"; all sub-scheduling information is concatenated into a single string, with each sub-scheduling information separated by commas, resulting in the first scheduling information: "single, off, http: / / single.example.com, cluster, on, http: / / cluster.example.com, sentinel, off, http: / / sentinel.example.com". This first scheduling information string represents the on / off status and data source address for each communication mode. Service nodes can parse this string to understand which communication modes should be activated and the corresponding data source address for each mode. Service nodes can adjust their communication strategy with the data source based on the instructions in the first scheduling information to ensure that the most appropriate communication mode is used for data processing.

[0155] Thus, by concatenating the switching information of each communication mode and the data source access address into a string to generate the first scheduling information, this implementation method simplifies information transmission and processing. It not only reduces data redundancy and improves data transmission efficiency but also facilitates automated parsing and execution of scheduling instructions. Furthermore, the string-based scheduling information is easily shared and transmitted between different system components, contributing to centralized management and enhancing the flexibility and maintainability of system configuration.

[0156] In some embodiments, the service node and the data source have multiple communication modes, the service requirement includes the requirement to enable a target communication mode among the multiple communication modes, and the second scheduling information includes the on / off information of each of the communication modes.

[0157] In some embodiments, business requirements refer to the specific requirements that business nodes need to meet during task execution. These requirements may include data processing speed, data security, system stability, etc. Data sources refer to the systems, devices, or services that provide data, such as databases, file systems, API interfaces, etc. Data sources are an important source of data required by business nodes to execute tasks.

[0158] In some embodiments, the modification of the second scheduling information according to the business requirements to obtain the first scheduling information can be achieved in the following way: the switch information in the second scheduling information that indicates the corresponding scheduling mode is on is modified to the switch information that indicates the corresponding scheduling mode is off, to obtain the third scheduling information; the switch information in the third scheduling information that indicates the target communication mode is off is modified to the switch information that indicates the target communication mode is on, to obtain the first scheduling information.

[0159] In some embodiments, the system first checks the on / off status of each scheduling mode in the second scheduling information. For scheduling modes that are already enabled, the system changes their on / off status from "enabled" to "disabled," generating third scheduling information. In the third scheduling information, the system identifies the on / off status of the target communication mode. If the target communication mode is displayed as "disabled" in the third scheduling information, it is changed to "enabled." If the target communication mode is already "enabled" in the third scheduling information, it remains unchanged. After the above modifications, the third scheduling information becomes the required first scheduling information. This information reflects the final on / off status of all scheduling modes under the current business requirements, where the target communication mode has been correctly enabled according to the requirements. In this way, the scheduling information can be dynamically adjusted according to changes in business requirements, ensuring that the target communication mode is prioritized, while unnecessary or no longer needed communication modes can be disabled, thereby optimizing resource utilization and improving efficiency.

[0160] As an example, suppose there is a distributed system containing multiple business nodes that need to communicate with a data source. The system supports two communication modes: single mode and colony mode. Currently, the second scheduling information is as follows: single mode: enabled; colony mode: disabled. Now, due to changes in business requirements, the system decides to disable single mode and enable colony mode as the new target communication mode. First, the enabled state of single mode in the second scheduling information is changed to disabled, resulting in the third scheduling information: single mode: disabled; colony mode: disabled. Next, the new target communication mode is identified as cluster mode, and the disabled state of cluster mode in the third scheduling information is changed to enabled, generating the first scheduling information: single mode: disabled; colony mode: enabled. The obtained first scheduling information instructs the business nodes to disable single mode and enable cluster mode. The business nodes will adjust their communication mode according to this scheduling information to meet the new business requirements. You can see how to dynamically adjust the communication mode based on changes in business needs, thereby optimizing system performance and resource utilization.

[0161] Thus, by modifying the second scheduling information according to business needs to obtain the first scheduling information, significant flexibility and adaptability are demonstrated. This ensures that business nodes respond promptly to changes in business requirements, effectively improving resource utilization and system performance by dynamically adjusting the communication mode. This strategy not only reduces the overhead of unnecessary communication modes but also helps avoid potential conflicts and overloads, while ensuring the priority of the target communication mode. Therefore, it optimizes the overall system's operational efficiency and stability while meeting real-time business needs.

[0162] Thus, during the data processing process where a business node connects to the first data source, if the received first scheduling information differs from the second scheduling information stored locally on the business node, and since the first scheduling information carries access addresses for multiple data sources, including the second data source, the second data source that the business node needs to switch to can be determined based on the first scheduling information. Based on the access address of the second data source, the first data source connected to the business node is switched to the second data source. Therefore, when the received first scheduling information differs from the second scheduling information stored locally on the business node, the second data source of the business node is determined based on the access addresses of the multiple data sources included in the first scheduling information, and the first data source connected to the business node is switched to the second data source based on the access address of the second data source. This achieves real-time hot switching of data sources during data processing, meeting the data processing needs of different data sources and effectively improving the processing efficiency of the business node.

[0163] The following will describe an exemplary application of the embodiments of this application in a real-world distributed service system application scenario.

[0164] This application's embodiments can be applied to microservice distributed deployments, allowing modification of the Nacos Redis data source configuration file without restarting the Java Nacos client's listening service, thus achieving dynamic switching of the Redis data source. Furthermore, dynamic configuration switching validation is added to prevent incorrect usernames, passwords, or hosts during configuration modifications. It can be applied to scenarios such as Redis failover, RedSi mode upgrades, and single-machine cluster conversions.

[0165] In some embodiments, see Figure 6 , Figure 6 This is a flowchart illustrating the data processing method provided in the embodiments of this application. Figure 3 The data processing method provided in this application embodiment can be used to... Figure 6 Steps 301 to 306 shown are implemented.

[0166] In step 301, a Nacos listener is registered, and the obtained Redis configuration class is loaded when the service starts, and the host of the configuration class is placed in the local cache.

[0167] In some embodiments, the `RedisProperties` class of `spring-data-redis` is overridden, and the `@RefreshScope` annotation is added to the class. This allows for dynamic, real-time updates to the corresponding properties when configuration changes are made in the Nacos server-side interface. When the Java service integrating the Nacos client initializes and starts, it obtains the overridden `RedisProperties` class. At this point, a custom `RedisConnectionFactory` is implemented. In this embodiment, the custom `RedisConnectionFactory` has two properties: `LettuceConnectionFactory` and `threadLocal`. Previously, `threadLocal` stored a String; now it stores a `LettuceConnectionFactory`. A method is also provided to switch between these properties and the `LettuceConnectionFactory` object. A Spring auto-injection method is then written, injecting the overridden `RedisProperties` class as a parameter, encapsulating it into a `LettuceConnectionFactory` object. When setting the property, threadLocal also stores this LettuceConnectionFactory object. It is important to note that when switching, the threadlocal and remote methods should be executed first, otherwise memory leaks may occur.

[0168] In step 302, modify the Redis configuration information in the Nacos configuration file.

[0169] In some embodiments, the automatic injection method simultaneously caches these RedisProperties properties in a local cache using a map. The key and value are both String types. The key can be single, sentinel, or colour, and the concatenation of the value differs depending on the key. There are three types: For single-machine mode, the key is single, and the value is the host and password properties of the corresponding RedisProperties property concatenated into a string. For cluster mode, the key is colour, and the value is the nodes property of the RedisProperties subclass Cluster concatenated into a string. For sentinel mode, the key is sentinel, and the value is the master, nodes, and password properties of the RedisProperties subclass Sentinel concatenated into a string.

[0170] In step 303, the client listens for changes to the file and compares them with the obtained configuration information and the local cache.

[0171] In some embodiments, the Java service adds a Nacos client listener, specifying that the Nacos file directory contains the Redis configuration file. When the Redis configuration is modified in the Nacos configuration interface, the client listener detects the configuration file change. It then calls a method to compare the local cached and rewritten `redisProperties` properties. There are roughly three scenarios: 1) The retrieved `redisProperties` properties, after string concatenation, show no change in the values ​​of the three keys in the local cache. 2) If the retrieved `redisProperties` properties, after string concatenation, show a change and do not match the value of one of the three keys in the local cache, then a method of the custom `RedisConnectionFactory` is called to encapsulate `redisProperties` into a `LettuceConnectionFactory`, and the properties are re-set, with `ThreadLocal` also assigning values ​​synchronously. 3) If the retrieved `redisProperties` properties, after string concatenation, show a change and do not match the value of two of the three keys in the local cache, then this situation only occurs during mode switching, such as switching from a single machine to a cluster. We will not consider several changes here. As long as there are changes, we will follow the same steps as the second method.

[0172] In step 304, it is determined whether the comparison has changed.

[0173] In some embodiments, when the Nacos listener detects file changes, a verification interface is also synchronously called. This interface verifies the properties of the obtained custom `redsiProperties` object to ensure a proper connection to Redsi. The verification method uses a `LettuceConnectionFactory` object wrapped in `redisProperties` to obtain a `getcconnection` object and execute the `ping` method. If an error occurs, or if the returned value is not `pong`, it indicates that the modified configuration in Nacos is unusable. This also prevents accidental changes from affecting the existing environment.

[0174] No processing is performed in step 305.

[0175] In some embodiments, if it is determined that no change has occurred during the comparison, no action is taken.

[0176] In step 306, a custom MultiRedisLettuceConnectionFactory object is obtained, its constructor method is overridden, and the currentRedisName property value is reset.

[0177] In some embodiments, the `RedisProperties` class of `spring-data-redis` needs to be rewritten with the `@RedisProperties` annotation for dynamic refresh by Nacos. This ensures that the retrieved `RedisProperties` changes in real time, and the `LettuceConnectionFactory` updates accordingly. Here, `MultiRedisLettuceConnectionFactory` has also been rewritten. Previously, `currentRedisName` stored a String type; now it's `LettuceConnectionFactory`, and the `connectionFactoryMap` property has also been changed to `LettuceConnectionFactory`.

[0178] This significantly reduces the workload of operation and maintenance in distributed microservice deployments, improving production efficiency. There's no need to restart the service after modifying configuration files, reducing workload. Because it's hot-update, it ensures uninterrupted service, greatly guaranteeing service stability. A new Redis configuration verification function has been added to prevent accidental operations from affecting the normal operation of existing services.

[0179] In some embodiments, see Figure 7 , Figure 7This is a schematic diagram illustrating the principle of the data processing method provided in this application. Both serviceA and serviceb are deployed in a cluster, with multiple instances of each service. However, they all connect to the same Nacos instance. Each instance contains a Nacos client to communicate with the Nacos server and listens for changes to the server's configuration file. ServiceA and serviceb communicate via a Feign client. As shown in the diagram, they share the same Redis data source. If this Redis database fails, business operations will be disrupted. An emergency switch to a backup Redis data source is required. In this case, the new data source can be configured in the Nacos server's configuration center. Both serviceA and serviceb's clusters have Nacos client listeners that detect configuration file changes and compare them with local cache configurations to replace the custom RedisConnectionFactory. This provides excellent timeliness, greatly ensuring service stability without downtime or re-releases, and also reduces the workload of operations and maintenance. The same principle applies to upgrading Redis's mode. For example, the Redis instance previously used in this application was in single-machine mode, which obviously had poor stability and reliability. In this case, switching to cluster mode is necessary, which can be achieved by updating the Redis data source configuration in the Nacos configuration center. Figure 7 Three diagrams are provided, illustrating the microservice architecture and the relationship between the Nacos server and client.

[0180] It is understood that in the embodiments of this application, data related to traffic and so on are involved. When the embodiments of this application are applied to specific products or technologies, user permission or consent is required, and the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions.

[0181] The following description continues to illustrate the exemplary structure of the data processing apparatus 455 provided in the embodiments of this application as a software module. In some embodiments, such as Figure 2 As shown, the software modules stored in the data processing device 455 of the memory 450 may include: a determining module 4551, configured to determine the second data source of the service node based on the access addresses of multiple data sources included in the first scheduling information in response to the received first scheduling information being different from the second scheduling information stored locally on the service node; and a switching module 4552, configured to switch the first data source of the service node to the second data source based on the access address of the second data source.

[0182] In some embodiments, the service node and the data source have multiple communication modes. The first scheduling information includes first sub-scheduling information corresponding to each communication mode, and the second scheduling information includes second sub-scheduling information corresponding to each communication mode. The determining module is further configured to compare the first sub-scheduling information and the second sub-scheduling information corresponding to each communication mode. If the first sub-scheduling information and the second sub-scheduling information are different, then the first scheduling information and the second scheduling information are determined to be different.

[0183] In some embodiments, the first sub-scheduling information includes at least one first information field, and the second sub-scheduling information includes a second information field that corresponds one-to-one with the first information field; the determining module is further configured to compare the first information field with the corresponding second information field, and if the first information field is different from the corresponding second information field, then determine that the first scheduling information is different from the second scheduling information.

[0184] In some embodiments, the data processing apparatus further includes: an authentication module, configured to perform permission authentication on the first scheduling information based on the second scheduling information to obtain a permission authentication result; the permission authentication result is used to indicate whether the first scheduling information has data source scheduling permission for the business node; the switching module is further configured to, if the permission authentication result indicates that the first scheduling information has data source scheduling permission for the business node, determine the second data source of the business node based on the access addresses of the multiple data sources included in the first scheduling information.

[0185] In some embodiments, the first scheduling information includes first authentication information, and the second scheduling information includes second authentication information. The authentication module is further configured to compare the first authentication information and the second authentication information; if the first authentication information and the second authentication information are the same, the permission authentication result is used to indicate that the first scheduling information has data source scheduling permission for the business node; if the first authentication information and the second authentication information are different, the permission authentication result is used to indicate that the first scheduling information does not have data source scheduling permission for the business node.

[0186] In some embodiments, the switching module is further configured to send a scheduling information acquisition request at preset time intervals; receive the first scheduling information returned in response to the scheduling information acquisition request; store the first scheduling information locally on the service node; and delete the second scheduling information stored locally on the service node.

[0187] The following description continues to illustrate the exemplary structure of the data processing apparatus 555 provided in the embodiments of this application as a software module. In some embodiments, such as... Figure 3 As shown, the software modules stored in the data processing device 555 of the memory 550 may include: a generation module for generating first scheduling information for a service node; a sending module for sending the first scheduling information to the service node in response to an information acquisition request sent by the service node; the first scheduling information is used to trigger the service node to determine a second data source and switch the connected first data source to the second data source when the first scheduling information is different from the second scheduling information stored locally by the service node.

[0188] In some embodiments, the above-mentioned generation module is further configured to generate the first scheduling information if an adjustment request for the service node is received; and to modify the second scheduling information according to the service requirements if first information is received, indicating that the first data source cannot meet the service requirements of the data processing, so as to obtain the first scheduling information.

[0189] In some embodiments, the generation module is further configured to, for each communication mode, generate switch information indicating that the communication mode is started if the communication mode is the target communication mode, and generate switch information indicating that the communication mode is not started if the communication mode is not the target communication mode; and generate the first scheduling information based on the switch information of each communication mode.

[0190] In some embodiments, the generation module is further configured to, for each communication mode, concatenate the switch information of the communication mode and the access address of at least one data source in the communication mode to obtain the first sub-scheduling information corresponding to the communication mode; and concatenate each of the first sub-scheduling information to obtain the first scheduling information.

[0191] In some embodiments, the service node and the data source have multiple communication modes, and the service requirement includes a requirement to enable a target communication mode among the multiple communication modes. The second scheduling information includes on / off information for each of the communication modes. The generation module is further configured to modify the on / off information of the corresponding scheduling mode in the second scheduling information to indicate that the corresponding scheduling mode is enabled, and to obtain third scheduling information; and to modify the on / off information of the target communication mode in the third scheduling information to indicate that the target communication mode is enabled, and to obtain the first scheduling information.

[0192] This application provides a computer program product, which includes a computer program or computer-executable instructions stored in a computer-readable storage medium. The processor of an electronic device reads the computer-executable instructions from the computer-readable storage medium and executes the computer-executable instructions, causing the electronic device to perform the data processing method described in this application.

[0193] This application provides a computer-readable storage medium storing computer-executable instructions. When these computer-executable instructions are executed by a processor, they cause the processor to perform the data processing method provided in this application. For example, ... Figure 4 The data processing method is shown.

[0194] In some embodiments, the computer-readable storage medium may be a memory such as FRAM, ROM, PROM, EPROM, EEPROM, flash memory, magnetic surface memory, optical disk, or CD-ROM; or it may be an electronic device that includes one or any combination of the above-mentioned memories.

[0195] In some embodiments, computer-executable instructions may take the form of programs, software, software modules, scripts, or code, written in any form of programming language (including compiled or interpreted languages, or declarative or procedural languages), and may be deployed in any form, including as stand-alone programs or as modules, components, subroutines, or other units suitable for use in a computing environment.

[0196] As an example, computer-executable instructions may, but do not necessarily, correspond to files in a file system. They may be stored as part of a file that holds other programs or data, for example, in one or more scripts in a HyperText Markup Language (HTML) document, in a single file dedicated to the program in question, or in multiple co-located files (e.g., files that store one or more modules, subroutines, or code sections).

[0197] As an example, computer-executable instructions can be deployed to execute on a single electronic device, or on multiple electronic devices located in one location, or on multiple electronic devices distributed across multiple locations and interconnected via a communication network.

[0198] In summary, the embodiments of this application have the following beneficial effects:

[0199] (1) During the data processing process of a business node connecting to the first data source, if the received first scheduling information differs from the second scheduling information stored locally by the business node, and since the first scheduling information carries access addresses of multiple data sources, including the second data source, the second data source that the business node needs to switch to can be determined based on the first scheduling information. Based on the access address of the second data source, the first data source connected to the business node is switched to the second data source. Thus, when the received first scheduling information differs from the second scheduling information stored locally by the business node, the second data source of the business node is determined based on the access addresses of multiple data sources included in the first scheduling information, and the first data source connected to the business node is switched to the second data source based on the access address of the second data source. This achieves real-time hot switching of data sources during data processing to meet the data processing needs of different data sources, thereby effectively improving the processing efficiency of the business node.

[0200] (2) Business nodes can ensure that they adjust their data processing logic according to the latest first scheduling information, thereby guaranteeing the efficient and accurate operation of the business. At the same time, this also enables business nodes to flexibly adapt to changes in the scheduling system, improving the reliability and flexibility of the system.

[0201] (3) Business nodes can ensure that only the first scheduling information with the correct permissions can schedule the data source, thereby protecting data security and business compliance. At the same time, this also enables business nodes to flexibly adapt to changes in the scheduling system, improving the system's reliability and flexibility.

[0202] (4) Business nodes can ensure that the scheduling operations they perform are within their authorized scope, thereby protecting data security and business compliance. Simultaneously, this allows business nodes to flexibly adapt to changes in the scheduling system, improving system reliability and flexibility. By authenticating the first scheduling information, business nodes can ensure that only information with correct permissions can schedule the data source, thus protecting data security and business compliance. Business nodes can flexibly adjust their processing logic based on the authentication results, responding to updates to the first scheduling information while preventing unauthorized information from interfering with business processes. Even when the first scheduling information changes, business nodes can continue to stably execute data processing based on the first data source according to the authentication results, ensuring business continuity and consistency. Recording the authentication process and processing results facilitates tracking and auditing, enabling rapid troubleshooting and performance analysis when problems arise. Authentication ensures that only authenticated information can schedule business nodes, improving the overall reliability and stability of the system.

[0203] (5) By dynamically switching between different data sources, business nodes can utilize the optimal performance of each data source, thereby improving the overall data processing speed and efficiency. When a new data source is added to the data source cluster, the system can automatically switch based on the first scheduling information without manual intervention, thus easily expanding system capacity and processing capabilities. The first scheduling information provides flexibility in data source selection, allowing business nodes to dynamically adjust data source usage strategies based on actual load, performance, or priority. Even if a data source fails, the first scheduling information can guide business nodes to switch to other normal data sources, thereby reducing the impact of system failures and ensuring business continuity. The first scheduling information can guide business nodes to select data sources based on the current load of the data sources, thereby avoiding resource waste and achieving more efficient resource utilization. Through the first scheduling information, business nodes can automatically manage complex data source configurations and switching, reducing manual intervention and management complexity. The first scheduling information can help business nodes avoid accessing insecure or non-compliant data sources, ensuring data security and compliance with regulatory requirements.

[0204] (6) By disconnecting or establishing connections based on the access address of the second data source, business nodes can utilize resources in the data source cluster more effectively, avoiding resource waste. By maintaining a connection with the second data source and disconnecting connections to other non-second data sources, business nodes can optimize data processing performance and improve system response speed. In cases where the second data source is not included in the data source cluster, business nodes can quickly disconnect from faulty or degraded data source clusters and establish a connection with the second data source, thereby improving the system's fault tolerance. Whether maintaining or establishing a connection with the second data source, business nodes can ensure business continuity and reduce business interruptions caused by data source problems. Data source connection management guided by the first scheduling information reduces the need for manual intervention and simplifies the complexity of system management. By selecting the second data source through the first scheduling information, business nodes can ensure that data access and processing are conducted within a secure scope, protecting data from unauthorized access by data sources.

[0205] (7) Regularly sending information retrieval requests enables business nodes to continuously receive the latest first-order information, allowing them to adjust their work strategies in a timely manner to respond to changes in system status. Business nodes can flexibly adjust their behavior based on the first-order information, such as adjusting data processing priorities, selecting different data sources, and executing specific business logic. Through the first-order information, business nodes can detect potential problems, such as data source unavailability or performance degradation, and take corresponding fault-tolerant measures, such as switching to backup data sources or adjusting load balancing strategies. The first-order information can help business nodes optimize resource usage; for example, when the data source load is high, workloads can be avoided from being allocated to that data source, thereby improving resource utilization efficiency. Through the first-order information, business nodes can ensure that their data processing can continue continuously, and can recover quickly even in the event of system failure or data source unavailability. Regularly receiving the first-order information helps ensure that business nodes can adapt to the continuous changes in system status, thereby improving the stability of the entire system. Through automated information retrieval request and processing mechanisms, the management complexity of business nodes is simplified, reducing the need for manual intervention. The first-order information can help business nodes ensure that the data and behaviors they process comply with security and compliance requirements, such as avoiding the processing of sensitive data or adhering to specific data processing rules. By periodically receiving the first scheduling information, business nodes can respond to changes in system status in real time, improving the system's real-time performance, flexibility, fault tolerance, resource optimization, business continuity, system stability, management simplification, and security.

[0206] (8) Significantly reduces the workload of operation and maintenance in distributed microservice deployments, improving production efficiency. It eliminates the need to restart the service after modifying configuration files, reducing workload. Due to hot updates, it ensures uninterrupted service provision without downtime, greatly guaranteeing service stability. A new Redis configuration verification function has been added to prevent accidental operations from affecting the normal operation of existing services.

[0207] (9) By modifying the second scheduling information according to business needs to obtain the first scheduling information, significant flexibility and adaptability are demonstrated. This ensures that business nodes respond promptly to changes in business needs, and effectively improves resource utilization and system performance by dynamically adjusting the state of communication modes. This strategy not only reduces the overhead of unnecessary communication modes but also helps avoid potential conflicts and overloads, while ensuring the priority of the target communication mode. Thus, while meeting real-time business needs, it optimizes the overall system's operating efficiency and stability.

[0208] (10) By concatenating the switch information of each communication mode and the data source access address into a string to generate the first scheduling information, this implementation method simplifies the information transmission and processing flow. It not only reduces data redundancy and improves the efficiency of data transmission, but also facilitates the automated parsing and execution of scheduling instructions. In addition, the scheduling information in string form is easy to share and transmit between different system components, which helps to achieve centralized management and enhances the flexibility and maintainability of system configuration.

[0209] (11) The dynamic scheduling strategy based on target communication modes improves the responsiveness and flexibility to business needs, enabling rapid adjustment of communication modes according to real-time conditions, thereby optimizing data transmission and processing efficiency. Secondly, through explicit on / off information indications, the system can clearly control the status of each communication mode, avoiding unnecessary communication overhead, reducing potential security risks, and ensuring the rational utilization of resources. Furthermore, this implementation method improves the reliability and stability of the system because it can automatically adjust the communication strategy according to the availability of the data source and changes in business needs, thereby avoiding service interruptions caused by the failure of a single communication mode. Finally, this dynamic scheduling method also facilitates monitoring and maintenance by system administrators, as they can easily understand the currently active communication modes and perform further management and optimization as needed.

[0210] The above description is merely an embodiment of this application and is not intended to limit the scope of protection of this application. Any modifications, equivalent substitutions, and improvements made within the spirit and scope of this application are included within the scope of protection of this application.

Claims

1. A data processing method, characterized in that, The method includes: In response to the difference between the received first scheduling information and the second scheduling information stored locally on the service node, the second data source of the service node is determined based on the access addresses of multiple data sources included in the first scheduling information and the switching information of each communication mode; the second data source refers to the data source in the communication mode when the switching information indicates that the communication mode is started; If the second data source is a data source cluster, and the data source cluster includes the first data source of the business node, then a data connection is established with the third data source other than the first data source in the data source cluster based on the access address of the third data source, while maintaining the connection with the first data source.

2. The method according to claim 1, characterized in that, The service node and the data source have multiple communication modes. The first scheduling information includes first sub-scheduling information corresponding to each communication mode, and the second scheduling information includes second sub-scheduling information corresponding to each communication mode. Before determining the second data source of the service node based on the access addresses of multiple data sources included in the first scheduling information, the method further includes: For each of the communication modes, the first sub-scheduling information and the second sub-scheduling information corresponding to the communication mode are compared. If the first sub-scheduling information and the second sub-scheduling information are different, then it is determined that the first scheduling information and the second scheduling information are different.

3. The method according to claim 2, characterized in that, The first sub-scheduling information includes at least one first information field, and the second sub-scheduling information includes a second information field that corresponds one-to-one with the first information field; The step of comparing the first sub-scheduling information and the second sub-scheduling information corresponding to the communication mode, and determining that the first scheduling information and the second scheduling information are different if the first sub-scheduling information and the second sub-scheduling information are different, includes: The first information field is compared with the corresponding second information field. If the first information field is different from the corresponding second information field, it is determined that the first scheduling information is different from the second scheduling information.

4. The method according to claim 1, characterized in that, Before determining the second data source of the service node based on the access addresses of multiple data sources included in the first scheduling information, the method further includes: Based on the second scheduling information, the first scheduling information is authenticated to obtain the authentication result. The step of determining the second data source of the service node based on the access addresses of multiple data sources included in the first scheduling information includes: If the permission authentication result indicates that the first scheduling information has the permission to schedule the data source of the business node, then the second data source of the business node is determined based on the access addresses of the multiple data sources included in the first scheduling information.

5. The method according to claim 4, characterized in that, The first scheduling information includes first authentication information, and the second scheduling information includes second authentication information. The step of performing permission authentication on the first scheduling information based on the second scheduling information to obtain an authentication result includes: Compare the first authentication information and the second authentication information; If the first authentication information and the second authentication information are the same, then the permission authentication result is used to indicate that the first scheduling information has the data source scheduling permission for the business node; If the first authentication information is different from the second authentication information, then the permission authentication result is used to indicate that the first scheduling information does not have the data source scheduling permission for the business node.

6. The method according to claim 1, characterized in that, Before determining the second data source of the business node based on the access addresses of multiple data sources included in the first scheduling information, in response to the difference between the received first scheduling information and the second scheduling information stored locally on the business node, the method further includes: Send scheduling information retrieval requests at preset time intervals; Receive the first scheduling information returned in response to the scheduling information retrieval request; Before establishing a data connection with the third data source and maintaining the connection with the first data source, the method further includes: The first scheduling information is stored locally on the service node, and the second scheduling information stored locally on the service node is deleted.

7. A data processing method, characterized in that, The method includes: First scheduling information is generated for a business node. In response to an information retrieval request sent by the business node, the first scheduling information is sent to the business node. The first scheduling information is used to trigger the business node to determine a second data source when the first scheduling information is different from the second scheduling information stored locally by the business node. If the second data source is a data source cluster and the data source cluster includes the first data source of the business node, a data connection is established with the third data source based on the access address of the third data source other than the first data source in the data source cluster, and the connection with the first data source is maintained. The first scheduling information includes access addresses of multiple data sources and switch information for each communication mode. The second data source refers to the data source in the communication mode when the switch information indicates that the communication mode is started.

8. The method according to claim 7, characterized in that, The generation of the first scheduling information for the service node includes: If an adjustment request for the service node is received, the first scheduling information is generated; If the first information is received, indicating that the first data source cannot meet the business requirements of the data processing, then the second scheduling information is modified according to the business requirements to obtain the first scheduling information.

9. The method according to claim 8, characterized in that, The service node and the data source have multiple communication modes, and the generation of the first scheduling information includes: For each of the communication modes, if the communication mode is the target communication mode, then switch information is generated to indicate that the communication mode is started; if the communication mode is not the target communication mode, then switch information is generated to indicate that the communication mode is not started. The first scheduling information is generated based on the switching information of each of the communication modes.

10. The method according to claim 9, characterized in that, The generation of the first scheduling information based on the switching information of each of the communication modes includes: For each of the communication modes, the switching information of the communication mode and the access address of at least one data source in the communication mode are concatenated to obtain the first sub-scheduling information corresponding to the communication mode. Each of the first sub-scheduling information is concatenated to obtain the first scheduling information.

11. The method according to claim 8, characterized in that, The service node and the data source have multiple communication modes, the service requirements include the requirement to enable a target communication mode among the multiple communication modes, and the second scheduling information includes the on / off information of each of the communication modes; The step of modifying the second scheduling information according to the business requirements to obtain the first scheduling information includes: The switch information indicating the activation of the corresponding scheduling mode in the second scheduling information is modified to the switch information indicating the deactivation of the corresponding scheduling mode to obtain the third scheduling information; The switch information indicating that the target communication mode is off in the third scheduling information is modified to the switch information indicating that the target communication mode is on, thus obtaining the first scheduling information.

12. A data processing apparatus, characterized in that, The device includes: The determination module is configured to, in response to a difference between the received first scheduling information and the second scheduling information stored locally on the service node, determine the second data source of the service node based on the access addresses of multiple data sources included in the first scheduling information and the switching information of each communication mode; the second data source refers to the data source in the communication mode when the switching information indicates that the communication mode is started; The switching module is used to establish a data connection with the third data source other than the first data source in the data source cluster if the second data source is a data source cluster and the data source cluster includes the first data source of the business node, and maintain the connection with the first data source.

13. A data processing apparatus, characterized in that, The device includes: The generation module is used to generate the first scheduling information for the business nodes; The sending module is used to send the first scheduling information to the service node in response to the information acquisition request sent by the service node; The first scheduling information is used to trigger the service node to determine a second data source when the first scheduling information differs from the second scheduling information stored locally by the service node. If the second data source is a data source cluster, and the data source cluster includes the first data source of the service node, a data connection is established with the third data source based on the access address of a third data source other than the first data source in the data source cluster, while maintaining the connection with the first data source. The first scheduling information includes access addresses of multiple data sources and switch information for each communication mode. The second data source refers to the data source in the communication mode when the switch information indicates that the communication mode is started.

14. An electronic device, characterized in that, The electronic device includes: Memory is used to store executable instructions or computer programs. A processor, when executing computer-executable instructions or computer programs stored in the memory, implements the data processing method according to any one of claims 1 to 6, or 7 to 11.

15. A computer-readable storage medium storing computer-executable instructions, characterized in that, When the computer-executable instructions are executed by a processor, they implement the data processing method according to any one of claims 1 to 6, or 7 to 11.

16. A computer program product comprising a computer program or computer-executable instructions, characterized in that, When the computer program or computer-executable instructions are executed by a processor, they implement the data processing method according to any one of claims 1 to 6, or 7 to 11.

Images