Data processing method and device, equipment, storage medium and program product

Abstract

The invention provides a data processing method and device, equipment, a storage medium and a program product. The method comprises the steps of determining a second data source of a service node based on access addresses of a plurality of data sources included in first scheduling information in response to the fact that the received first scheduling information is different from the second scheduling information locally stored by the service node; and switching the first data source of the service node to the second data source based on the access address of the second data source. According to the invention, the data processing efficiency of the service node can be effectively improved.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a data processing method, device, equipment, storage medium and program product. Background Art

[0002] Cluster technology is a technology that has emerged recently with the development of the Internet. It connects a group of independent computers through a high-speed network to form a cluster system that can be managed and scheduled in a unified manner. It is used to handle the business in each big data scenario, thereby improving the scalability and reliability of the service. In order to ensure that the business in the big data scenario can run normally, the cluster system usually stores a huge amount of data to support each business in the big data scenario.

[0003] In the related art, for the data source connection of the business node, the business node usually connects to a fixed data source for data processing. Since during the data processing process, the fixed data source cannot meet the data requirements of the data processing, the processing efficiency of the data processing of the business node is low. Summary of the invention

[0004] The embodiments of the present application provide a data processing method, device, electronic device, computer-readable storage medium, and computer program product, which can effectively improve the data processing efficiency of service nodes.

[0005] The technical solution of the embodiment of the present application is implemented as follows:

[0006] The present application provides a data processing method, including:

[0007] In response to the received first scheduling information being different from the second scheduling information locally stored in the service node, determining a second data source of the service node based on access addresses of a plurality of data sources included in the first scheduling information;

[0008] Based on the access address of the second data source, the first data source of the service node is switched to the second data source.

[0009] The present application provides a data processing method, including:

[0010] Generate first scheduling information for a business node, and send the first scheduling information to the business node in response to an information acquisition request sent by the business node, wherein the first scheduling information is used to trigger the business node to determine a second data source and switch the connected first data source to the second data source when the first scheduling information is different from second scheduling information locally stored in the business node.

[0011] The present application provides a data processing device, including:

[0012] a determination module, configured to determine, in response to the received first scheduling information being different from the second scheduling information locally stored in the service node, a second data source of the service node based on access addresses of a plurality of data sources included in the first scheduling information;

[0013] A switching module is used to switch the first data source of the service node to the second data source based on the access address of the second data source.

[0014] The present application provides a data processing device, including:

[0015] A generating module, used to generate first scheduling information for a service node;

[0016] A sending module, configured to send the first scheduling information to the service node in response to an information acquisition request sent by the service node;

[0017] The first scheduling information is used to trigger the service node to determine a second data source and switch the connected first data source to the second data source when the first scheduling information is different from second scheduling information locally stored in the service node.

[0018] An embodiment of the present application provides an electronic device, including:

[0019] A memory for storing computer executable instructions or computer programs;

[0020] The processor is used to implement the data processing method provided in the embodiment of the present application when executing the computer executable instructions or computer programs stored in the memory.

[0021] An embodiment of the present application provides a computer-readable storage medium storing computer-executable instructions for causing a processor to execute the instructions to implement the data processing method provided in the embodiment of the present application.

[0022] The embodiment of the present application provides a computer program product, which includes a computer program or a computer executable instruction, and the computer program or the computer executable instruction is stored in a computer-readable storage medium. The processor of the electronic device reads the computer executable instruction from the computer-readable storage medium, and the processor executes the computer executable instruction, so that the electronic device executes the data processing method described above in the embodiment of the present application.

[0023] The embodiments of the present application have the following beneficial effects:

[0024] In the process of the service node connecting to the first data source for data processing, in response to the received first scheduling information being different from the second scheduling information stored locally in the service node, since the first scheduling information carries the access addresses of multiple data sources, and the multiple data sources include the second data source, based on the first scheduling information, the second data source that the service node needs to switch can be determined, and based on the access address of the second data source, the first data source connected to the service node is switched to the second data source. In this way, when the received first scheduling information is different from the second scheduling information stored locally in the service node, the second data source of the service node is determined based on the access addresses of the multiple data sources included in the first scheduling information, and based on the access address of the second data source, the first data source connected to the service node is switched to the second data source, thereby realizing real-time hot switching of the data source during the data processing of the service, so as to meet the data requirements of data processing for different data sources, thereby effectively improving the processing efficiency of data processing by the service node. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 It is a schematic diagram of the architecture of the data processing system provided in the embodiment of the present application;

[0026] Figure 2 This is a schematic diagram of the structure of an electronic device for data processing provided in an embodiment of the present application. Figure 1 ;

[0027] Figure 3 This is a schematic diagram of the structure of an electronic device for data processing provided in an embodiment of the present application. Figure 2 ;

[0028] Figure 4 This is a flow chart of the data processing method provided in the embodiment of the present application. Figure 1 ;

[0029] Figure 5 This is a flow chart of the data processing method provided in the embodiment of the present application. Figure 2 ;

[0030] Figure 6 This is a flow chart of the data processing method provided in the embodiment of the present application. Figure 3 ;

[0031] Figure 7 It is a schematic diagram of the principle of the data processing method provided in the embodiment of the present application. DETAILED DESCRIPTION

[0032] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application will be further described in detail below in conjunction with the accompanying drawings. The described embodiments should not be regarded as limiting the present application. All other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of this application.

[0033] In the following description, reference is made to “some embodiments”, which describe a subset of all possible embodiments, but it will be understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0034] In the following description, the terms "first\second\third" involved are merely used to distinguish similar objects and do not represent a specific ordering of the objects. It can be understood that "first\second\third" can be interchanged with a specific order or sequence where permitted, so that the embodiments of the present application described here can be implemented in an order other than that illustrated or described here.

[0035] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein are only for the purpose of describing the embodiments of this application and are not intended to limit this application.

[0036] Before further describing the embodiments of the present application in detail, the nouns and terms involved in the embodiments of the present application are explained. The nouns and terms involved in the embodiments of the present application are subject to the following interpretations.

[0037] 1) Distributed system: refers to a distributed file system designed to run on commodity hardware. It has a lot in common with existing distributed file systems, but at the same time, it is also very different from other distributed file systems. HDFS is a highly fault-tolerant system that is suitable for deployment on cheap machines. HDFS can provide high-throughput data access and is very suitable for applications on large data sets. HDFS relaxes some POSIX constraints to achieve the purpose of streaming file system data. HDFS has the characteristics of high fault tolerance and is designed to be deployed on low-cost hardware. It also provides high throughput to access application data and is suitable for applications with large data sets. HDFS relaxes POSIX requirements so that streaming access can be achieved to data in the file system.

[0038] 2) Cloud Technology: Cloud technology refers to a hosting technology that unifies hardware, software, network and other resources within a wide area network or local area network to achieve data computing, storage, processing and sharing. Cloud technology is a general term for network technology, information technology, integration technology, management platform technology, application technology, etc. based on the cloud computing business model. It can form a resource pool that can be used on demand and is flexible and convenient. Cloud computing technology will become an important support. The background services of technical network systems require a large amount of computing and storage resources, such as video websites, image websites and more portal websites. With the rapid development and application of the Internet industry, in the future, each item may have its own identification mark, and all need to be transmitted to the background system for logical processing. Data of different levels will be processed separately. Each type of industry data requires a strong system backend support, which can only be achieved through cloud computing.

[0039] 3) In response to: used to indicate the conditions or states on which the executed operations depend. When the dependent conditions or states are met, one or more operations executed may be in real time or have a set delay. Unless otherwise specified, there is no restriction on the order in which the multiple operations executed are executed.

[0040] 4) Cluster technology: It is a technology that has emerged recently with the development of the Internet. It connects a group of independent computers through a high-speed network to form a cluster system that can be managed and scheduled in a unified manner. It is used to handle services in each big data scenario, thereby improving the scalability and reliability of services. In order to ensure that services in big data scenarios can run normally, cluster systems usually store massive amounts of data to support each type of service in big data scenarios.

[0041] 5) Pod: Pod is the smallest unit that can be created and managed in Kubernetes. It is a collection of containers that share the same network namespace and storage volumes. A Pod usually contains one or more containers, as well as some shared resources for communication and storage between containers. The containers in a Pod share the same network interface and storage volumes, which means they can communicate with each other through localhost and can access the same storage resources. Pod has its own life cycle, including states such as creation, running, pause, resume, and termination. Pod is used to deploy and manage applications. It is the basic unit for Kubernetes to schedule and manage computing tasks.

[0042] 6) Business Node: A business node usually refers to a component or service that undertakes a specific data processing function. It is responsible for performing corresponding logical processing according to the access request, such as data verification, business logic calculation, data storage, etc. A business node may be a module inside the application or an external service.

[0043] 7) Remote Dictionary Service: Often referred to as Redis, it is a high-performance key-value storage system. It can be used as a database, cache, and message broker middleware. Redis is open source, written in ANSI C, and follows the BSD protocol. On a technical level, Redis is an in-memory data structure storage system that supports multiple types of data structures, such as strings, lists, sets, ordered sets (ZSet, also known as SortedSet), and hashes. These data structures enable Redis to handle every complex data operation task. These features of Redis make it suitable for scenarios that require fast access and processing of large amounts of data, such as distributed caches, session caches, message queues, leaderboards, social networks, etc. Due to its high performance and high availability, Redis has become an indispensable part of many large systems and applications.

[0044] 8) Nacos: is an open source service discovery and configuration management platform developed by Alibaba Group for building applications based on microservice architecture. The name Nacos comes from the abbreviation of "Naming and Configuration Service". It provides a unified system to manage and configure services, as well as dynamic discovery between services. In a microservice architecture, services usually run in different servers or containers. The role of Nacos is to manage the registration and discovery of these services to ensure that services can communicate with each other correctly. The main functions include: Service discovery and registration, Nacos allows services to self-register at startup, and allows other services to query and discover these registered services for mutual communication. Dynamic configuration management, Nacos can centrally manage service configuration information, support dynamic update and push of configuration, so that configuration changes can take effect in real time without restarting the service. Service management, Nacos provides service grouping, namespace management, as well as service weight adjustment and routing rule setting functions to help developers better control service traffic and load balancing. High availability, Nacos supports cluster mode, multiple Nacos nodes can be deployed together to form a high-availability cluster to ensure efficient and stable service discovery and configuration management. Easy to integrate, Nacos has good integration with mainstream microservice frameworks such as Spring Cloud and Dubbo, which simplifies the complexity of developers introducing Nacos into the existing architecture. Multi-environment support, Nacos supports multi-environment configuration, and can manage and push different configuration information according to different environments (such as development, testing, and production). Lightweight and easy to deploy, as a lightweight platform, Nacos is easy to integrate into each cloud native environment, such as Kubernetes, Docker, etc.

[0045] During the implementation of the embodiments of the present application, the applicant discovered that the related technology has the following problems:

[0046] In the related art, for the data source connection of the business node, the business node usually connects to a fixed data source for data processing. Since during the data processing process, the fixed data source cannot meet the data requirements of the data processing, the processing efficiency of the data processing of the business node is low.

[0047] The embodiments of the present application provide a data processing method, device, electronic device, computer-readable storage medium and computer program product, which can effectively improve the data processing efficiency of business nodes. The following describes an exemplary application of the data processing system provided by the embodiments of the present application.

[0048] See also Figure 1 , Figure 1It is a schematic diagram of the architecture of the data processing system 100 provided in an embodiment of the present application. The terminal (terminal 400 is shown as an example) is connected to the server 200 via the network 300. The network 300 can be a wide area network or a local area network, or a combination of the two.

[0049] The terminal 400 is used for the user to use the client 410 and display the test script on the graphical interface 410-1 (the graphical interface 410-1 is shown as an example). The terminal 400 and the server 200 are connected to each other via a wired or wireless network.

[0050] In some embodiments, the server 200 may be an independent physical server, or a server cluster or business system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDN), and big data and artificial intelligence platforms. The terminal 400 may be a smart phone, a tablet computer, a laptop computer, a desktop computer, a smart speaker, a smart TV, a smart watch, a car terminal, etc., but is not limited thereto. The electronic device provided in the embodiment of the present application may be implemented as a terminal or as a server. The terminal and the server may be directly or indirectly connected via wired or wireless communication, which is not limited in the embodiment of the present application.

[0051] In some embodiments, in response to the received first scheduling information being different from the second scheduling information locally stored in the business node, the server 200 determines the second data source of the business node based on the first scheduling information, and switches the first data source connected to the business node to the second data source based on the access address.

[0052] In other embodiments, in response to the received first scheduling information being different from the second scheduling information locally stored in the business node, the terminal 400 determines the second data source of the business node based on the first scheduling information, and switches the first data source connected to the business node to the second data source based on the access address.

[0053] In other embodiments, the embodiments of the present application can be implemented with the aid of cloud technology. Cloud technology refers to a hosting technology that unifies a series of resources such as hardware, software, and networks within a wide area network or a local area network to achieve data computing, storage, processing, and sharing.

[0054] Cloud technology is a general term for network technology, information technology, integration technology, management platform technology, and application technology based on the cloud computing business model. It can form a resource pool that can be used on demand and is flexible and convenient. Cloud computing technology will become an important support. The background services of the technical network system require a large amount of computing and storage resources.

[0055] See also Figure 2 , Figure 2 The structure of the electronic device 500 for data processing provided in the embodiment of the present application is shown in FIG. Figure 1 ,in, Figure 2 The electronic device 500 shown may be Figure 1 The server 200 or the terminal 400 in Figure 2 The electronic device 500 shown includes: at least one processor 430, a memory 450, and at least one network interface 420. Each component in the electronic device 500 is coupled together via a bus system 440. It is understood that the bus system 440 is used to achieve connection and communication between these components. In addition to the data bus, the bus system 440 also includes a power bus, a control bus, and a status signal bus. However, for the sake of clarity, the bus system 440 is not described in detail. Figure 2 Each bus is labeled as bus system 440 .

[0056] The processor 430 can be an integrated circuit chip with signal processing capabilities, such as a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc., where the general-purpose processor can be a microprocessor or any conventional processor, etc.

[0057] The memory 450 may be removable, non-removable, or a combination thereof. Exemplary hardware devices include solid-state memory, hard drives, optical drives, etc. The memory 450 may optionally include one or more storage devices that are physically remote from the processor 430.

[0058] The memory 450 includes a volatile memory or a non-volatile memory, and may also include both volatile and non-volatile memories. The non-volatile memory may be a read-only memory (ROM), and the volatile memory may be a random access memory (RAM). The memory 450 described in the embodiments of the present application is intended to include any suitable type of memory.

[0059] In some embodiments, the memory 450 can store data to support each operation, examples of which include programs, modules, and data structures, or a subset or superset thereof, as exemplarily described below.

[0060] Operating system 451, including system programs for processing each basic system service and performing hardware-related tasks, such as a framework layer, a core library layer, a driver layer, etc., for implementing each basic business and processing hardware-based tasks;

[0061] The network communication module 452 is used to reach other electronic devices via one or more (wired or wireless) network interfaces 420. Exemplary network interfaces 420 include: Bluetooth, Wireless Fidelity (WiFi), and Universal Serial Bus (USB).

[0062] In some embodiments, the data processing device provided in the embodiments of the present application can be implemented in software. Figure 2 The data processing device 455 stored in the memory 450 is shown, which can be software in the form of a program and a plug-in, etc., including the following software modules: a determination module 4551, a switching module 4552, these modules are logical, and therefore can be arbitrarily combined or further split according to the functions implemented. The function of each module will be described below.

[0063] See also Figure 3 , Figure 3 This is a schematic diagram of the structure of an electronic device for data processing provided in an embodiment of the present application. Figure 2 ,in, Figure 3 The electronic device 600 shown may be Figure 1 The server 200 or the terminal 400 in Figure 3 The electronic device 600 shown includes: at least one processor 530, a memory 550, and at least one network interface 520. Each component in the electronic device 600 is coupled together via a bus system 540. It is understood that the bus system 540 is used to achieve connection and communication between these components. In addition to the data bus, the bus system 540 also includes a power bus, a control bus, and a status signal bus. However, for the sake of clarity, the bus system 540 is not described in detail. Figure 3 Each bus is labeled as bus system 540 .

[0064] The processor 530 can be an integrated circuit chip with signal processing capabilities, such as a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc., where the general-purpose processor can be a microprocessor or any conventional processor, etc.

[0065] The memory 550 may be removable, non-removable, or a combination thereof. Exemplary hardware devices include solid-state memory, hard disk drives, optical disk drives, etc. The memory 550 may optionally include one or more storage devices that are physically remote from the processor 530.

[0066] The memory 550 includes a volatile memory or a non-volatile memory, and may also include both volatile and non-volatile memories. The non-volatile memory may be a read-only memory (ROM), and the volatile memory may be a random access memory (RAM). The memory 550 described in the embodiments of the present application is intended to include any suitable type of memory.

[0067] In some embodiments, the memory 550 can store data to support each operation, examples of which include programs, modules, and data structures, or a subset or superset thereof, as exemplarily described below.

[0068] Operating system 551, including system programs for processing each basic system service and performing hardware-related tasks, such as a framework layer, a core library layer, a driver layer, etc., for implementing each basic business and processing hardware-based tasks;

[0069] The network communication module 552 is used to reach other electronic devices via one or more (wired or wireless) network interfaces 520. Exemplary network interfaces 520 include: Bluetooth, Wireless Fidelity (WiFi), and Universal Serial Bus (USB).

[0070] In some embodiments, the data processing device provided in the embodiments of the present application can be implemented in software. Figure 3 The data processing device 555 stored in the memory 550 is shown, which can be software in the form of a program and a plug-in, etc., including the following software modules: a generation module 5551 and a sending module 5552. These modules are logical, so they can be arbitrarily combined or further split according to the functions implemented. The function of each module will be explained below.

[0071] In other embodiments, the data processing device provided in the embodiments of the present application can be implemented in hardware. As an example, the data processing device provided in the embodiments of the present application can be a processor in the form of a hardware decoding processor, which is programmed to execute the data processing method provided in the embodiments of the present application. For example, the processor in the form of a hardware decoding processor can adopt one or more application-specific integrated circuits (ASICs), DSPs, programmable logic devices (PLDs), complex programmable logic devices (CPLDs), field programmable gate arrays (FPGAs), or other electronic components.

[0072] In some embodiments, the terminal or server can implement the data processing method provided in the embodiment of the present application by running a computer program or a computer executable instruction. For example, the computer program can be a native program (e.g., a dedicated data processing program) or a software module in an operating system, for example, a data processing module that can be embedded in any program (such as an instant messaging client, an album program, an electronic map client, a navigation client); for example, it can be a native application (APP, Application), that is, a program that needs to be installed in the operating system to run. In short, the above-mentioned computer program can be an application, module or plug-in in any form.

[0073] The data processing method provided in the embodiment of the present application will be explained in combination with the exemplary application and implementation of the server or terminal provided in the embodiment of the present application.

[0074] See also Figure 4 , Figure 4 This is a flow chart of the data processing method provided in the embodiment of the present application. Figure 1 , will combine Figure 4 Steps 101 to 102 are shown for illustration. The data processing method provided in the embodiment of the present application can be implemented by a server or a terminal alone, or by a server and a terminal in collaboration. The following description will be given by taking the implementation by a server alone as an example.

[0075] In step 101, in response to the received first scheduling information being different from the second scheduling information locally stored in the service node, a second data source of the service node is determined based on access addresses of multiple data sources included in the first scheduling information.

[0076] In some embodiments, the first scheduling information carries access addresses of multiple data sources, the multiple data sources include a second data source, and the first scheduling information is generated during the process of the service node performing service processing based on connecting to an initial data source.

[0077] In some embodiments, the first data source refers to the data source to which the service node is connected when it starts processing services. It may be a database, a file system, an API interface, etc., which is used to provide data required for data processing.

[0078] In some embodiments, the first data source refers to the data source to which the service node is connected when it starts processing services. It may be a database, a file system, an API interface, etc., which is used to provide data required for data processing.

[0079] In some embodiments, the second scheduling information is a set of scheduling rules or configuration information stored locally by the service node, which determines how the service node reads and writes data sources when processing services, including the order of data reading, the target of data writing, etc.

[0080] In some embodiments, the first scheduling information refers to scheduling instructions or adjustment information dynamically sent to the service node during the service node processing. Such information may include data source switching rules, priority adjustment, data processing order changes, etc., and is used to adjust the data processing logic of the service node in real time.

[0081] In some embodiments, when a business node connects to a first data source for data processing, if the received first scheduling information is inconsistent with the second scheduling information stored locally, the business node needs to determine a new second data source based on the first scheduling information so as to switch to the new data source to continue data processing. This process ensures that the business node can flexibly adjust the data processing logic according to the latest scheduling requirements to meet changing business needs.

[0082] In some embodiments, the business node connects to the first data source for data processing based on the second scheduling information stored locally. When the received first scheduling information is different from the second scheduling information stored locally at the business node, it means that the business node cannot connect to the first data source for data processing based on the second scheduling information stored locally, and needs to switch to the second data source for data processing based on the received first scheduling information.

[0083] In some embodiments, there are multiple communication modes between the service node and the data source, the first scheduling information includes first sub-scheduling information corresponding to each communication mode, and the second scheduling information includes second sub-scheduling information corresponding to each communication mode.

[0084] In some embodiments, the communication mode of a business node is used to indicate the connection method of the business node to a data source. For example, the communication mode may be a single-node communication mode, which is used to indicate that the business node can only connect to a single data source. The communication mode may also be a cluster communication mode, which is used to indicate that the business node can connect to a data source cluster including multiple data sources.

[0085] As an example, the service node includes a first communication mode and a second communication mode, the first scheduling information includes first sub-scheduling information corresponding one-to-one to the first communication mode and first sub-scheduling information corresponding one-to-one to the second communication mode, and the above-mentioned second scheduling information includes second sub-scheduling information corresponding one-to-one to the first communication mode and second sub-scheduling information corresponding to the second communication mode.

[0086] In some embodiments, before determining the second data source of the service node based on the access addresses of multiple data sources included in the first scheduling information, it can be determined whether the first scheduling information is the same as the second scheduling information in the following manner: for each of the communication modes, the first sub-scheduling information corresponding to the communication mode and the second sub-scheduling information corresponding to the communication mode are compared; if the first sub-scheduling information and the second sub-scheduling information are different, it is determined that the first scheduling information is different from the second scheduling information.

[0087] In some embodiments, if the first sub-scheduling information and the second sub-scheduling information in each of the communication modes are the same, it is determined that the first scheduling information is the same as the second scheduling information.

[0088] In some embodiments, the first sub-scheduling information includes at least one first information field, and the second sub-scheduling information includes a second information field corresponding one-to-one to the first information field.

[0089] In some embodiments, the above-mentioned comparing the first sub-scheduling information and the second sub-scheduling information corresponding to the communication mode, if the first sub-scheduling information and the second sub-scheduling information are different, then determining that the first scheduling information is different from the second scheduling information, can be achieved as follows: comparing the first information field with the corresponding second information field, if the first information field is different from the corresponding second information field, then determining that the first scheduling information is different from the second scheduling information.

[0090] In some embodiments, two data structures are defined, which are used to represent the first sub-scheduling information and the second sub-scheduling information, respectively. Each sub-scheduling information includes a set of information fields, and these information fields should correspond one to one in the two sub-scheduling information. A comparison logic is implemented, which compares the corresponding information fields in the first sub-scheduling information and the second sub-scheduling information in sequence according to a certain order or rule (such as the order of the fields). When comparing each corresponding information field, check whether their values ​​are equal. If it is found that any pair of field values ​​are not equal, the comparison operation is stopped immediately, because it is sufficient to prove that the first sub-scheduling information and the second sub-scheduling information are different. If all the information fields have been compared and are equal, it can be determined that the two sub-scheduling information are the same. If any field is not equal, according to the comparison logic, we can determine that the first sub-scheduling information is different from the second sub-scheduling information. According to the comparison results of the sub-scheduling information, it can be inferred whether the first scheduling information is different from the second scheduling information. If the sub-scheduling information is different, it can be concluded that the first scheduling information is also different from the second scheduling information. By comparing each information field in the two sub-scheduling information one by one, it is possible to quickly and accurately determine whether the scheduling information is different. This comparison method ensures that as long as there are differences, conclusions can be drawn in the shortest possible time, thus improving efficiency.

[0091] In some embodiments, it is necessary to determine all possible communication modes of the service node. The communication mode may be directly connected to a data source, or connected to multiple data sources through middleware, agents, etc. For each communication mode, it is necessary to obtain the current first sub-scheduling information from the real-time scheduling system, which may include the current state, priority, load, and other information of the data source. At the same time, it is necessary to obtain the corresponding second sub-scheduling information from the local storage of the service node. This information is loaded from the global scheduling system when the service node is started or at some time before. For each communication mode, the first sub-scheduling information obtained is compared with the second sub-scheduling information. The comparison is based on factors such as scheduling rules, data source status, and priority. Generate an information comparison result. This result is a representation that tells the system whether the first sub-scheduling information and the second sub-scheduling information are the same. Check all information comparison results. If there is any information comparison result indicating that the first sub-scheduling information and the second sub-scheduling information are different, then it can be determined that the first scheduling information is different from the second scheduling information. If all information comparison results indicate that the first sub-scheduling information and the second sub-scheduling information are the same, then it can be determined that the first scheduling information is the same as the second scheduling information. If it is determined that the first scheduling information is different from the second scheduling information, the business node needs to update its second scheduling information according to the first scheduling information. This may include updating the priority of the data source, switching to a new data source, adjusting the data processing logic, etc. If the first scheduling information is the same as the second scheduling information, the business node can continue to use the current data source and scheduling rules for data processing without making any changes. Once the business node updates the second scheduling information according to the first scheduling information, it needs to update the new scheduling information to the local storage so that the correct scheduling information can be loaded the next time it is started or executed.

[0092] In this way, the service node can ensure that it adjusts the data processing logic according to the latest first scheduling information, thereby ensuring efficient and accurate operation of the service. At the same time, this also enables the service node to flexibly adapt to changes in the scheduling system and improve the reliability and flexibility of the system.

[0093] In some embodiments, during the process of the service node connecting to the first data source to perform the data processing, the following processing may also be performed: in response to the received first scheduling information being the same as the second scheduling information, continuing to perform data processing based on the first data source.

[0094] In some embodiments, during the process of a business node connecting to a first data source for data processing, in response to the received first scheduling information being the same as the second scheduling information, when the received first scheduling information is the same as the second scheduling information stored locally at the business node, it indicates that the business node can continue to connect to the first data source for data processing based on the locally stored second scheduling information.

[0095] In some embodiments, if the comparison result shows that the first scheduling information is the same as the second scheduling information, it means that the current scheduling information has not changed. This may be because the new scheduling information has not yet been released, or the current business node is already in the latest scheduling state. Since the scheduling information has not changed, the business node can continue to use the first data source for data processing. This means that the business node does not need to switch the data source and can perform tasks according to the original scheduling rules and data processing logic. Although the current scheduling information has not changed, the business node usually needs to continue to monitor the first scheduling information so that it can respond in time once the scheduling information changes. Even if the first scheduling information is the same as the second scheduling information, the business node should update the second scheduling information regularly or according to specific trigger conditions. This ensures that the business node always has the latest scheduling rules and configurations. The business node can ensure that its processing logic is consistent with the latest scheduling requirements, and can also quickly respond to any changes in scheduling information, thereby improving the flexibility and reliability of the business.

[0096] In some embodiments, before determining the second data source of the business node based on the access addresses of multiple data sources included in the first scheduling information, the following processing can also be performed: based on the second scheduling information, the first scheduling information is authenticated to obtain an authentication result.

[0097] In some embodiments, the authority authentication result is used to indicate whether the first scheduling information has the data source scheduling authority for the service node.

[0098] In some embodiments, the service node reads the currently stored scheduling information from the local storage. This information includes the access rights, operation rights, priority, etc. of the data source. The service node receives the latest first scheduling information through the scheduling system interface or other mechanisms. This information may include updates on the status, priority, processing logic, etc. of the data source. The service node uses the locally stored scheduling information as a reference to perform permission authentication on the first scheduling information. This involves verifying whether the permission parameters in the first scheduling information match the permission configuration stored locally. The permission authentication result is an indication that tells the service node whether the first scheduling information has the data source scheduling authority for the current service node. If the permission parameters of the first scheduling information match the permission configuration in the second scheduling information, the authentication result is "passed," otherwise it is "failed." If the permission authentication result is "passed," it means that the first scheduling information has the data source scheduling authority for the service node. The service node can perform corresponding scheduling operations based on this, such as switching the data source, adjusting the processing logic, etc. If the permission authentication result is "failed," it means that the first scheduling information does not have the data source scheduling authority for the service node. The service node should not perform any scheduling operations based on the first scheduling information, but needs to continue to follow the locally stored scheduling information for data processing.

[0099] In some embodiments, the above-mentioned determination of the second data source of the business node based on the access addresses of multiple data sources included in the first scheduling information can be achieved in the following way: if the authority authentication result indicates that the first scheduling information has the data source scheduling authority for the business node, then based on the access addresses of multiple data sources included in the first scheduling information, the second data source of the business node is determined.

[0100] In some embodiments, the service node checks the result of the authority authentication to determine whether the first scheduling information has the data source scheduling authority for the current service node. If the result of the authority authentication indicates that the first scheduling information has the scheduling authority, the service node further analyzes the first scheduling information to determine whether the second data source needs to be switched. This may involve comparing factors such as the data source priority, load, and availability in the first scheduling information. If the first scheduling information indicates that the data source needs to be switched, the service node determines the new second data source based on the access addresses of the multiple data sources included in the first scheduling information, which may be a data source with the highest priority, the lowest load, or the highest availability. Once the service node determines the new second data source, it needs to update the scheduling information in the local storage to ensure that the correct scheduling information can be loaded the next time it is started or executed. The service node performs a data source switching operation based on the new second data source. This may include disconnecting the connection with the current data source, establishing a connection with the new data source, reinitializing the data access interface, etc. For tracking and auditing, the service node should record the process and results of the data source switching. This helps troubleshooting and performance analysis when problems arise. After the data source switching is completed, the service node continues to process data based on the new second data source. The business node can ensure that the scheduling operation it performs is performed within the scope of authority, and can flexibly switch the data source according to the first scheduling information when necessary to improve the reliability and flexibility of the system. At the same time, this also enables the business node to quickly respond to changes in the scheduling system and ensure the efficient and accurate operation of the business.

[0101] In some embodiments, the first scheduling information includes first authentication information, and the second scheduling information includes second authentication information.

[0102] In some embodiments, the above-mentioned performing authority authentication on the first scheduling information based on the second scheduling information to obtain an authority authentication result can be achieved in the following manner: comparing the first authentication information and the second authentication information; if the first authentication information and the second authentication information are the same, then the authority authentication result is used to indicate that the first scheduling information has the data source scheduling authority for the business node; if the first authentication information is different from the second authentication information, then the authority authentication result is used to indicate that the first scheduling information does not have the data source scheduling authority for the business node.

[0103] In some embodiments, the service node obtains information to be authenticated from the real-time scheduling system. This information may include a signature of the scheduling information, encrypted authentication information, or other data used to verify identity and authority. The service node obtains the second authentication information from the local storage or authentication service. This information is pre-defined and is used as a reference standard for comparison with the first authentication information. The service node compares the first authentication information with the second authentication information. The comparison process may involve verifying the legitimacy of the signature, the integrity of the authentication information, or other security verification steps. If the information comparison result indicates that the first authentication information is the same as the second authentication information, the service node can determine that the first scheduling information has the data source scheduling authority for the service node. At this time, the authority authentication result is "passed." If the information comparison result indicates that the first authentication information is different from the second authentication information, the service node can determine that the first scheduling information does not have the data source scheduling authority for the service node. At this time, the authority authentication result is "failed." According to the authority authentication result, the service node updates its authority authentication status. If the authentication passes, the service node can continue to perform the scheduling operation; if the authentication fails, the service node should comply with the existing scheduling rules and not perform any new scheduling operations. For tracking and auditing, the service node should record the authentication process and results. This helps with troubleshooting and performance analysis when problems arise.

[0104] In this way, the business node can ensure that only the first scheduling information with the correct authority can schedule the data source, thereby protecting the security of the data and the compliance of the business. At the same time, this also enables the business node to flexibly adapt to changes in the scheduling system and improve the reliability and flexibility of the system.

[0105] In some embodiments, the above-mentioned authorization authentication is performed on the first scheduling information based on the second scheduling information. After obtaining the authorization authentication result, the following processing can also be performed: if the authorization authentication result indicates that the first scheduling information does not have the data source scheduling authority for the business node, then continue to perform the data processing based on the first data source.

[0106] In some embodiments, the service node reads the currently stored scheduling information from the local storage. This information includes the access rights, operation rights, priority, etc. of the data source. The service node receives the latest first scheduling information through the scheduling system interface or other mechanisms. This information may include updates on the status, priority, processing logic, etc. of the data source. The service node uses the locally stored scheduling information as a reference to perform permission authentication on the first scheduling information. This involves verifying whether the permission parameters in the first scheduling information match the permission configuration stored locally. If the permission parameters of the first scheduling information match the permission configuration in the second scheduling information, the authentication result is "passed," otherwise it is "failed." If the permission authentication result is "failed," it means that the first scheduling information does not have the data source scheduling authority for the service node. The service node should not perform any scheduling operation based on the first scheduling information. Since the permission authentication result is "failed," the service node continues to process data based on the first data source. This means that the service node will not switch the data source according to the first scheduling information, but will continue to use the first data source to execute its business logic. Although the current permission authentication result is "failed," the service node usually needs to continuously monitor the changes in the first scheduling information so that it can respond in time once the scheduling information changes. Business nodes should record the results of permission authentication and the data processing performed based on these authentication results. This helps with troubleshooting and performance analysis when problems arise.

[0107] In this way, the business node can ensure that the scheduling operation it performs is carried out within the scope of authority, thereby protecting the security of data and the compliance of business. At the same time, this also enables the business node to flexibly adapt to changes in the scheduling system and improve the reliability and flexibility of the system. By performing authority authentication on the first scheduling information, the business node can ensure that only information with the correct authority can schedule the data source, thereby protecting the security of data and the compliance of business. The business node can flexibly adjust its processing logic according to the authority authentication result, which can not only respond to the update of the first scheduling information, but also prevent unauthorized information from interfering with the business process. Even when the first scheduling information changes, the business node can continue to stably perform data processing based on the first data source according to the authority authentication result to ensure the continuity and consistency of the business. The record of the authority authentication process and processing results is helpful for tracking and auditing, and troubleshooting and performance analysis can be quickly performed when problems arise. Authority authentication ensures that only authenticated information can schedule the business node, improving the overall reliability and stability of the system.

[0108] In some embodiments, the service node includes multiple communication modes, the first scheduling information includes first sub-scheduling information corresponding to each of the communication modes, the first sub-scheduling information includes switch information used to characterize whether the communication mode is started, and the access address of at least one data source under the communication mode.

[0109] In some embodiments, the above-mentioned determination of the second data source of the service node based on the first scheduling information can be achieved in the following manner: for each first sub-scheduling information, if the switch information in the first sub-scheduling information indicates that the communication mode is started, then at least one data source under the communication mode is determined as the second data source.

[0110] In some embodiments, the business node sets multiple communication modes at startup, and each mode corresponds to a different data source access strategy and priority. The business node receives the first scheduling information containing the first sub-scheduling information of each communication mode through a scheduling system or other mechanism. The business node parses the first sub-scheduling information in the first scheduling information, and each communication mode has a set of corresponding first sub-scheduling information. For the first sub-scheduling information of each communication mode, the business node checks the switch information. If the switch information in the first sub-scheduling information of a communication mode indicates that the mode is started, then at least one data source in the communication mode will be determined as the second data source. This means that the business node will connect to the data source for data processing. Once the second data source is determined, the business node will connect to the data source and perform data processing and business logic based on the data source. The business node records the information of connecting to the second data source and the data processing performed for subsequent auditing and analysis.

[0111] In some embodiments, before executing the above step 101, the first scheduling information may also be obtained in the following manner: sending a scheduling information acquisition request at a preset time interval; and receiving the first scheduling information returned in response to the scheduling information acquisition request.

[0112] In some embodiments, the service node first receives the first scheduling information, which includes the access addresses of multiple data sources. The service node compares the received first scheduling information with the second scheduling information stored locally. By comparison, the service node detects that there is a difference between the first scheduling information and the second scheduling information. Based on the detected difference, the service node will start a scheduling information acquisition process. The first step of this process is to send a scheduling information acquisition request at a preset time interval. This time interval can be fixed, such as sending a request every 10 seconds, or it can be a dynamic interval based on certain conditions. The purpose of sending a scheduling information acquisition request is to obtain the latest scheduling information from the relevant scheduling management node or other service nodes to ensure that the local scheduling information is the latest, or to resolve the inconsistency between the local scheduling information and the received scheduling information. The service node receives a response to the scheduling information acquisition request, that is, the new first scheduling information. This information should contain the latest or correct scheduling information. After receiving the new first scheduling information, the service node will process it, such as updating the local second scheduling information to ensure that the local scheduling information is consistent with the latest scheduling information. After obtaining the new scheduling information, the service node may determine the second data source based on the data source access address contained in the new scheduling information.

[0113] In some embodiments, before executing the following step 102, the scheduling information stored locally in the service node may be updated in the following manner: the first scheduling information is stored locally in the service node, and the second scheduling information stored locally in the service node is deleted.

[0114] In some embodiments, the service node needs to determine why the scheduling information needs to be updated. This may be because the received first scheduling information is inconsistent with the second scheduling information stored locally, or because there is a new scheduling instruction that needs to be executed. Before deciding to update, the service node needs to verify the validity of the received first scheduling information. This may include checking the source, format, signature, etc. of the information to ensure that the information is reliable and correct. The service node writes the complete content of the first scheduling information to the local storage to ensure that all data is correctly saved. After the storage operation is completed, the service node may need to perform some confirmation operations, such as verifying whether the stored information is consistent with the original information to ensure the integrity of the data. The service node needs to locate the location of the second scheduling information stored locally. Once the second scheduling information is found, the service node will perform a deletion operation. This may include deleting records from the database, deleting files from the file system, etc. After performing the deletion operation, the service node needs to confirm that the second scheduling information has been completely deleted and no data is left. The service node may need to update the local status flag to indicate that the current scheduling information is already the latest version of the first scheduling information. It can ensure that the service node can safely and effectively update its locally stored scheduling information and maintain the consistency and normal operation of the entire system.

[0115] In step 102, based on the access address of the second data source, the first data source connected to the service node is switched to the second data source.

[0116] In some embodiments, the above step 102 can be implemented as follows: if the second data source is a data source cluster, and the data source cluster includes the first data source, then based on the access address of a third data source other than the first data source in the data source cluster, a data connection with the third data source is established, and the connection with the first data source is maintained; if the second data source is the data source cluster, and the data source cluster does not include the first data source, the data connection with the first data source is disconnected, and based on the access address, a data connection with the data source cluster is established.

[0117] In some embodiments, the service node determines the second data source according to the first scheduling information. If the second data source is a data source cluster, and the data source cluster includes the first data source, then the connection with the first data source is maintained. If the second data source is a data source cluster, and the data source cluster does not include the first data source, then the connection with the first data source needs to be disconnected. Based on the access addresses of the third data sources other than the first data source in the data source cluster, data connections with these third data sources are established. After establishing a connection with the third data source in the data source cluster, the active state of these data source connections is maintained so that the service node can read or write data from them at any time. The service node records the current data source connection state, including which data source is connected, and the state information of the connection (such as the activity of the connection, read and write performance, etc.). After completing the connection switching of the data source, the service node performs corresponding data processing based on the new data source connection. This may include operations such as data reading, processing, and writing. The service node continuously monitors the connection state with each data source in the data source cluster to ensure the stability of the connection and the availability of the data. The service node remains sensitive to changes in the first scheduling information, and once receiving new scheduling information, it immediately switches the connection of the data source according to its representation.

[0118] As an example, for an online shopping platform application scenario, its order processing system is connected to a database cluster. When a database in the database cluster degrades due to high load, the real-time scheduling system may instruct the business node to switch to another database in the database cluster. At this time, the business node will disconnect from the initial database and establish a new connection based on the access address of the other database to ensure the continuity and performance of order processing.

[0119] As an example, for the application scenario of content distribution network, a website's server may be connected to a CDN cluster. When a node in the CDN cluster fails, the real-time scheduling system will instruct the business node to switch to other healthy nodes. The business node will disconnect from the failed node and re-establish the connection based on the access address of other nodes to maintain the fast distribution and availability of website content.

[0120] As an example, in a cloud computing application scenario, an application may be deployed on multiple cloud service instances. When a cloud service instance has a problem, the real-time scheduling system may indicate that the application should be migrated to other healthy instances. The business node will disconnect from the failed cloud service instance and establish a new connection based on the access address of other healthy instances to ensure the continuous operation of the application and the availability of the service.

[0121] As an example, in the application scenario of a financial transaction system, transaction records may be stored on multiple data centers or cloud services to improve the reliability and scalability of the system. When a problem occurs in a data center or cloud service, the real-time scheduling system will instruct the business node to switch to other normal data centers or cloud services. The business node will disconnect from the faulty data center and re-establish the connection based on the access address of other normal data centers or cloud services to ensure the accuracy of transaction records and the continuity of transactions.

[0122] In this way, by dynamically switching between different data sources, the business node can utilize the best performance of each data source, thereby improving the overall data processing speed and efficiency. When a new data source is added to the data source cluster, the system can automatically switch according to the first scheduling information without manual intervention, thereby conveniently expanding the system capacity and processing power. The first scheduling information provides flexibility in data source selection, allowing business nodes to dynamically adjust data source usage policies based on actual load, performance or priority. Even if a data source fails, the first scheduling information can guide the business node to switch to other normal data sources, thereby reducing the impact of system failures and ensuring business continuity. The first scheduling information can guide the business node to select a data source based on the current load of the data source, thereby avoiding resource waste and achieving more efficient resource utilization. Through the first scheduling information, the business node can automatically manage complex data source configuration and switching, reducing manual intervention and management complexity. The first scheduling information can help business nodes avoid accessing unsafe or non-compliant data sources, ensuring data security and compliance with regulatory requirements.

[0123] In some embodiments, the above step 102 can also be implemented in the following manner: if the first data source is a data source cluster, and the data source cluster includes the second data source, then based on the access address, disconnect the data connection with the third data source other than the second data source in the data source cluster, and maintain the data connection with the second data source; if the first data source is the data source cluster, and the data source cluster does not include the second data source, disconnect the data connection with the data source cluster, and establish a data connection with the second data source based on the access address.

[0124] In some embodiments, the service node identifies the second data source based on the first scheduling information. The service node checks whether the first data source is part of the data source cluster and whether it contains the second data source. If the first data source is part of the data source cluster and contains the second data source, then the data connection with the second data source is maintained and the data connection with other non-second data sources is disconnected. If the first data source is part of the data source cluster but does not contain the second data source, then the data connection with the entire data source cluster is disconnected and a data connection with the second data source is established. Based on the access address of the second data source, the corresponding data connection is disconnected or established. This may include closing the old connection, starting a new connection, configuring connection parameters, etc. After completing the connection switching of the data source, the service node performs corresponding data processing based on the new data source connection. This may include operations such as data reading, processing, and writing. The service node continuously monitors the connection status with the second data source to ensure the stability of the connection and the availability of the data. The service node is sensitive to changes in the first scheduling information, and once new scheduling information is received, the connection switching of the data source is immediately performed according to its representation.

[0125] As an example, in an online retail platform, the order processing system is connected to a database cluster. When a database node's performance degrades due to high load, the real-time scheduling system may instruct the business node to switch the second data source to other database nodes with better performance. At this time, the business node will disconnect from other database nodes in the database cluster except the target database, and maintain the connection with the target database to optimize order processing performance.

[0126] As an example, in a cloud computing environment, a web application may be deployed on multiple cloud service instances. When a cloud service instance fails, the real-time scheduling system may instruct the business node to switch the second data source to another healthy cloud service instance. The business node will disconnect from the failed cloud service instance and establish a new connection with the target cloud service instance to ensure the continuity of the web application service.

[0127] As an example, in a content delivery network (CDN) scenario, a website's server may be connected to a CDN cluster. When a node in the CDN cluster fails, the real-time scheduling system will instruct the business node to switch to other healthy nodes. The business node will disconnect from the failed node and establish a connection with the target healthy node to ensure fast distribution and availability of website content.

[0128] As an example, in a financial transaction system, transaction records may be stored on multiple data centers or cloud services to improve the reliability and scalability of the system. When a data center or cloud service has a problem, the real-time scheduling system will instruct the business node to switch to another normal data center or cloud service. The business node will disconnect from the faulty data center and establish a connection with the target normal data center or cloud service to ensure the accuracy of transaction records and the continuity of transactions.

[0129] In this way, by disconnecting or establishing a connection according to the access address of the second data source, the business node can more effectively utilize the resources in the data source cluster and avoid wasting resources. By maintaining the connection with the second data source and disconnecting the connections of other non-second data sources, the business node can optimize the performance of data processing and improve the response speed of the system. In the case where the second data source is not included in the data source cluster, the business node can quickly disconnect from the failed or degraded data source cluster and establish a connection with the second data source, thereby improving the fault tolerance of the system. Whether maintaining a connection with the second data source or establishing a connection with the second data source, the business node can ensure business continuity and reduce business interruptions caused by data source problems. Data source connection management under the guidance of the first scheduling information reduces the need for manual intervention and simplifies the complexity of system management. By selecting the second data source through the first scheduling information, the business node can ensure that data access and processing are carried out within a safe range and protect the data from being accessed by unauthorized data sources.

[0130] In some embodiments, during the process of the business node connecting to the first data source to perform the data processing, the following processing may also be performed: sending an information acquisition request to the scheduling node at a preset time interval, and receiving the first scheduling information returned by the scheduling node in response to the information acquisition request.

[0131] In some embodiments, the business node sends an information acquisition request to the scheduling node through the network or other communication mechanism at a preset time interval. This may be an HTTP request, a custom communication protocol message, or other forms of network data packets. After receiving the information acquisition request from the business node, the scheduling node will process the request and then return the corresponding first scheduling information. The response of the scheduling node may be sent back to the business node in the form of a data packet through the network. The business node receives the first scheduling information returned by the scheduling node, and receives and parses it. This may include steps such as reading the network data packet, parsing the returned data format, and extracting the first scheduling information. The business node processes the received first scheduling information. This may include updating the internal state, triggering corresponding logical processing, recording logs, etc. After completing the process of information acquisition request and response, the business node will periodically repeat the information acquisition request at a preset time interval to continuously receive updates to the first scheduling information.

[0132] As an example, suppose there is an online video streaming platform whose backend system consists of multiple data processing nodes, which need to adjust the distribution strategy of video content according to real-time user viewing data. In order to achieve this function, the platform maintains a scheduling node, which is responsible for collecting and analyzing data from user behavior and generating first scheduling information, including the distribution priority of video content, the selection of data sources in hot spots, etc. Each data processing node sends an HTTP GET request to the scheduling node at a preset time interval (for example, every minute), and the request may contain the node's own ID or other identification information. After receiving the information acquisition request, the scheduling node extracts the latest user viewing data and video content distribution strategy from the database or real-time analysis system, and generates the corresponding first scheduling information. Then, the scheduling node returns the first scheduling information to the data processing node as a response, and this response is usually a data packet in JSON or XML format. After receiving the first scheduling information returned by the scheduling node, the data processing node will parse the data packet and extract the required scheduling information, such as the distribution priority of the video content. The data processing node will adjust its own working strategy according to the received first scheduling information. For example, if the scheduling information shows that a certain video content is very popular in a specific area, the data processing node will give priority to distributing the video content to improve the user experience. By regularly repeating the request, it can be ensured that the latest first scheduling information can be continuously obtained, so that the video content distribution strategy can respond to changes in user behavior in real time.

[0133] In this way, the regular sending of information acquisition requests enables the business node to continuously receive the latest first scheduling information, so that it can adjust its work strategy in time to respond to changes in the system state. The business node can flexibly adjust its behavior according to the first scheduling information, such as adjusting the data processing priority, selecting different data sources, executing specific business logic, etc. Through the first scheduling information, the business node can detect potential problems, such as the unavailability or performance degradation of the data source, and take corresponding fault-tolerant measures, such as switching to a backup data source or adjusting the load balancing strategy. The first scheduling information can help the business node optimize the use of resources. For example, when the data source load is high, it can avoid allocating the workload to the data source, thereby improving resource utilization efficiency. Through the first scheduling information, the business node can ensure that its data processing can be carried out continuously and can be quickly restored even when encountering a system failure or the data source is unavailable. Regularly receiving the first scheduling information helps ensure that the business node can adapt to the continuous changes in the system state, thereby improving the stability of the entire system. Through the automated information acquisition request and processing mechanism, the management complexity of the business node is simplified, reducing the need for manual intervention. The first scheduling information can help the business node ensure that the data and behavior it processes meet security and compliance requirements, such as avoiding processing sensitive data or complying with specific data processing rules. By regularly receiving the first scheduling information, the business node can respond to changes in the system status in real time, improving the system's real-time performance, flexibility, fault tolerance, resource optimization, business continuity, system stability, management simplification and security.

[0134] In some embodiments, after executing the above step 102, the following processing may also be performed: performing the data processing based on the second data source.

[0135] In some embodiments, after the service node switches from connecting to the first data source to connecting to the second data source, the service node performs the data processing based on the second data source.

[0136] In this way, in the process of the service node connecting to the first data source for data processing, in response to the fact that the received first scheduling information is different from the local storage information stored locally by the service node, since the first scheduling information carries the access addresses of multiple data sources, and the multiple data sources include the second data source, based on the first scheduling information, it is possible to determine the second data source that the service node needs to switch, and based on the access address, the first data source connected to the service node is switched to the second data source, and data processing is performed based on the second data source. In this way, when the received first scheduling information is different from the second scheduling information stored locally by the service node, the second data source of the service node is determined based on the first scheduling information, and based on the access address, the first data source connected to the service node is switched to the second data source, and data processing is performed based on the second data source, thereby realizing real-time hot switching of the data source during the data processing of the service, so as to meet the data requirements of data processing for different data sources, thereby effectively improving the processing efficiency of data processing by the service node.

[0137] See also Figure 5 , Figure 5 This is a flow chart of the data processing method provided in the embodiment of the present application. Figure 2 , will combine Figure 5 Steps 201 to 202 are shown for illustration. The data processing method provided in the embodiment of the present application can be implemented by a server or a terminal alone, or by a server and a terminal in collaboration. The following description will be given by taking the implementation by a server alone as an example.

[0138] In step 201, first scheduling information for a service node is generated.

[0139] In some embodiments, the first scheduling information refers to scheduling instructions or adjustment information dynamically sent to the service node during the service node processing. Such information may include data source switching rules, priority adjustment, data processing order changes, etc., and is used to adjust the data processing logic of the service node in real time.

[0140] In some embodiments, when a business node connects to a first data source for data processing, if the received first scheduling information is inconsistent with the second scheduling information stored locally, the business node needs to determine a new second data source based on the first scheduling information so as to switch to the new data source to continue data processing. This process ensures that the business node can flexibly adjust the data processing logic according to the latest scheduling requirements to meet changing business needs.

[0141] In step 202, in response to the information acquisition request sent by the service node, the first scheduling information is sent to the service node.

[0142] In some embodiments, the first scheduling information is used to trigger the service node to determine a second data source and switch the connected first data source to the second data source when the first scheduling information is different from second scheduling information locally stored in the service node.

[0143] In some embodiments, the business node connects to the first data source for data processing based on the second scheduling information stored locally. When the received first scheduling information is different from the second scheduling information stored locally at the business node, it means that the business node cannot connect to the first data source for data processing based on the second scheduling information stored locally, and needs to switch to the second data source for data processing based on the received first scheduling information.

[0144] In some embodiments, the above-mentioned generation of the first scheduling information for the business node can be achieved in the following manner: if an adjustment request for the business node is received, the first scheduling information is generated; if the first information is received, and the first information indicates that the first data source cannot meet the business requirements of the data processing, then the second scheduling information is modified according to the business requirements to obtain the first scheduling information.

[0145] In some embodiments, adjustment requests from all parties are continuously monitored. These adjustment requests may be issued by the business node itself, or by other system components or system management layers. When an adjustment request for a specific business node is received, it will first perform preprocessing, such as verifying the legitimacy of the request, parsing the request content, etc. If the verification is passed, the system will generate the first scheduling information according to the content of the request. If the trigger condition for generating the first scheduling information is receiving an adjustment request for the business node, the system will construct new scheduling information according to the specific content of the request. If the trigger condition for generating the first scheduling information is receiving the first information, indicating that the first data source cannot meet the business requirements, the system will evaluate the current business requirements and make necessary modifications to the existing second scheduling information. The data provided in the first information will be analyzed to identify the specific reasons why the first data source cannot meet the business requirements. According to the business requirements, the resource allocation, data processing strategy, task execution order, etc. in the second scheduling information may be adjusted to ensure that the new scheduling information can meet the business requirements. After modifying the second scheduling information, the system will generate new scheduling information, namely the first scheduling information. The first scheduling information will contain all adjusted details to guide the business node to perform subsequent data processing and task execution. The generated first scheduling information is sent to the relevant business node. After receiving the first scheduling information, the service node will perform corresponding operations according to the new scheduling information, such as resource adjustment, task reallocation, etc. New scheduling information can be flexibly generated according to the received adjustment request or the first information to ensure that the service node can perform effective data processing according to the current business needs and data source conditions.

[0146] As an example, assume that there is a distributed data processing system that includes multiple business nodes, each of which is responsible for processing data from different data sources. The following is a specific scenario: Business node A is responsible for processing data from data source DS1. DS1 is a high-performance database, but its processing capacity is limited. In addition, there is a second data source DS2, which is a distributed file storage system with larger storage space and different processing capacity. Business node A receives an adjustment request, which is initiated by the system administrator because the data processing capacity of DS1 is about to reach the upper limit. This request is detected and a first scheduling information is generated based on the content of the request. The information instructs business node A to transfer part of the data processing tasks to DS2 to reduce the load on DS1. The first scheduling information details which data processing tasks need to be transferred and how to allocate these tasks on DS2. Business node A starts to reallocate tasks according to the first scheduling information and migrates part of the data processing work to DS2. When processing data, business node A finds that DS1 has a fault and cannot provide the necessary data processing capacity, generating the first information. The first information indicates that DS1 cannot meet the current business needs, which may be due to hardware failure or a surge in data volume. According to the first information, the second scheduling information of business node A is modified. The second scheduling information was originally designed based on the operating status of DS1. The second scheduling information is modified to reallocate the data processing tasks originally executed on DS1 to DS2 and other available data sources. The modified second scheduling information is converted into new first scheduling information and sent back to business node A. After receiving the new first scheduling information, business node A executes data processing tasks according to the new scheduling strategy to ensure business continuity and correctness of data processing.

[0147] In this way, generating the first scheduling information for the business node can effectively improve the flexibility and responsiveness of the system. When the system receives an adjustment request from a business node, it generates new scheduling information in a timely manner, which can quickly adapt to changes in business needs, optimize resource allocation, and improve the efficiency of task execution. At the same time, when the first data source cannot meet business needs, the system generates new first scheduling information by modifying the second scheduling information, which not only ensures the continuity and correctness of data processing, but also enhances the system's adaptability to emergencies, avoids business interruptions caused by data source problems, and thus improves the stability and reliability of the overall system.

[0148] In some embodiments, there are multiple communication modes between the business node and the data source. The above-mentioned generation of the first scheduling information can be implemented in the following manner: for each of the communication modes, if the communication mode is the target communication mode, then generate switch information for indicating that the communication mode is started; if the communication mode is not the target communication mode, then generate switch information for indicating that the communication mode is not started; based on the switch information of each of the communication modes, generate the first scheduling information.

[0149] In some embodiments, the service node first needs to be able to identify multiple communication modes with the data source. Each communication mode may correspond to different data transmission protocols, data formats, transmission rates, etc. One or more target communication modes are defined, which are the communication modes that the service node expects or needs to activate in specific situations. The target communication mode may be determined based on business needs, availability of data sources, network conditions, etc. The service node will check each communication mode with its data source. For each communication mode, the service node will determine whether it is the target communication mode. The service node generates a switch information, which indicates that this communication mode should be started. This switch information may be a Boolean value, a status code, or a message containing a start instruction. The service node generates a switch information, which indicates that this communication mode should not be started. The service node integrates the switch information of all communication modes to form a complete set of scheduling instructions. Based on the integrated switch information, the service node generates a first scheduling information. This scheduling information will contain detailed instructions on which communication modes should be started and which should not be started. The generated first scheduling information is sent to the relevant data source or scheduling management system so that they can adjust their behavior based on this information, such as activating or shutting down a specific communication mode. In this way, business nodes can precisely control communication with data sources, ensuring that only target communication modes are activated, thereby optimizing the efficiency of data transmission and processing while avoiding unnecessary data traffic and potential security risks.

[0150] As an example, suppose there is a business node that needs to communicate with a data source. They support three communication modes: stand-alone mode, cluster mode, and sentinel mode. Each mode has its corresponding identifier (key), as follows: stand-alone mode: key = "single"; cluster mode: key = "colony"; sentinel mode: key = "sentinel". Now, the business node needs to decide which communication mode to use based on the current business needs and environmental conditions. The business node determines that the target communication mode is the cluster mode based on the current business load, the state of the data source, and the network conditions. For the stand-alone mode, the business node generates switch information, indicating that the stand-alone mode is not started (because the target communication mode is the cluster mode). For the cluster mode, the business node generates switch information, indicating that the cluster mode is started. For the sentinel mode, the business node generates switch information, indicating that the sentinel mode is not started. The business node integrates the generated switch information into a scheduling information structure. For example, this structure may be a dictionary or a JSON object, containing three fields, each of which corresponds to a communication mode, and its value indicates whether it is started. The service node sends the generated first scheduling information to the data source. After receiving this information, the data source will activate or shut down the corresponding communication mode according to the instructions. After receiving the first scheduling information, the data source will parse the switch information therein. According to the requirements of the service node, the data source will shut down the communication services in the stand-alone mode and sentinel mode, and start the communication service in the cluster mode. Through this process, the service node can dynamically control the communication mode with the data source according to the current needs to ensure the efficiency of data processing and the stability of the system. In this example, the cluster mode may provide higher data processing capabilities and better resource utilization, so it is selected as the target communication mode.

[0151] In this way, the dynamic scheduling strategy based on the target communication mode improves the response speed and flexibility to business needs, making it possible to quickly adjust the communication mode according to the real-time situation, thereby optimizing data transmission and processing efficiency. Secondly, through clear switch information indication, the system can clearly control the state of each communication mode, avoid unnecessary communication overhead, reduce potential security risks, and ensure that resources are used reasonably. In addition, this implementation method also improves the reliability and stability of the system, because it can automatically adjust the communication strategy according to the availability of data sources and changes in business needs, thereby avoiding service interruptions caused by the failure of a single communication mode. Finally, this dynamic scheduling method is also convenient for system administrators to monitor and maintain, because they can easily understand the currently activated communication mode and perform further management and optimization as needed.

[0152] In some embodiments, the generation of the first scheduling information based on the switch information of each communication mode can be achieved in the following manner: for each of the communication modes, the switch information of the communication mode and the access address of at least one data source under the communication mode are spliced ​​to obtain the first sub-scheduling information corresponding to the communication mode; each of the first sub-scheduling information is spliced ​​to obtain the first scheduling information.

[0153] In some embodiments, for each communication mode, the system will generate a switch information indicating whether the communication mode should be started. At the same time, the system will extract the access address of the data source associated with the communication mode. Then, the system will splice these switch information and data source addresses to form a complete sub-scheduling information. This sub-scheduling information will include the switch status and the data source address, for example: {"mode":"cluster", "enabled":true, "address":"http: / / cluster.example.com"}. The first sub-scheduling information corresponding to each communication mode is spliced ​​to form an overall scheduling information structure. The splicing method can be to merge all sub-scheduling information into an array, list or JSON object. The information structure obtained by the above splicing is the first scheduling information, which contains the switch status of all communication modes and the corresponding data source address. The first scheduling information can be sent to business nodes or other system components so that they can adjust their behavior based on this information. It can clearly represent the status and data source address of each communication mode, so as to facilitate business nodes or other components to perform corresponding operations based on this information. In addition, this implementation method also makes the representation of scheduling information more structured and easy to manage, which helps to improve the maintainability and scalability of the system.

[0154] As an example, suppose there are three communication modes between a business node and a data source: stand-alone mode, cluster mode, and sentinel mode. Each mode has its corresponding switch information and data source access address. The following is an example of how to splice this information into the first scheduling information: The business node needs to select the appropriate communication mode according to the current load and network conditions, and communicate with the corresponding data source. Stand-alone mode: the switch information is "off", and the data source address is "http: / / single.example.com". Cluster mode: the switch information is "on", and the data source address is "http: / / cluster.example.com". Sentinel mode: the switch information is "off", and the data source address is "http: / / sentinel.example.com". The switch information and data source address of the stand-alone mode are concatenated into a string: "single, off, http: / / single.example.com"; the switch information and data source address of the cluster mode are concatenated into a string: "cluster, on, http: / / cluster.example.com"; the switch information and data source address of the sentinel mode are concatenated into a string: "sentinel, off, http: / / sentinel.example.com"; all sub-scheduling information are concatenated into a string, and each sub-scheduling information is separated by a comma to obtain the first scheduling information: "single, off, http: / / single.example.com, cluster, on, http: / / cluster.example.com, sentinel, off, http: / / sentinel.example.com. The first scheduling information string indicates the switch status and data source address of each communication mode. The service node can parse this string to understand which communication modes should be activated currently and the data source address corresponding to each mode. The service node can adjust the communication strategy between it and the data source according to the instructions in the first scheduling information to ensure that the most appropriate communication mode is used for data processing.

[0155] In this way, by splicing the switch information of each communication mode and the data source access address into a string to generate the first scheduling information, this implementation method is beneficial to simplify the information transmission and processing process. It not only reduces data redundancy and improves the efficiency of data transmission, but also facilitates the automatic parsing and execution of scheduling instructions. In addition, the scheduling information in the form of a string is easy to share and transmit between different system components, which helps to achieve centralized management and enhances the flexibility and maintainability of system configuration.

[0156] In some embodiments, there are multiple communication modes between the service node and the data source, the service demand includes a demand for starting a target communication mode among the multiple communication modes, and the second scheduling information includes switch information of each of the communication modes.

[0157] In some embodiments, business requirements refer to specific requirements that business nodes need to meet in the process of performing tasks. These requirements may include data processing speed, data security, system stability, etc. Data sources refer to systems, devices or services that provide data, such as databases, file systems, API interfaces, etc. Data sources are important sources of data required for business nodes to perform tasks.

[0158] In some embodiments, the above-mentioned modification of the second scheduling information according to the business needs to obtain the first scheduling information can be achieved in the following way: modifying the switch information in the second scheduling information indicating that the corresponding scheduling mode is turned on, to switch information indicating that the corresponding scheduling mode is turned off, to obtain third scheduling information; modifying the switch information in the third scheduling information indicating that the target communication mode is turned off, to switch information indicating that the target communication mode is turned on, to obtain the first scheduling information.

[0159] In some embodiments, the switch status of each scheduling mode in the second scheduling information is first checked. For those scheduling modes that have been turned on, the system modifies the switch status of these scheduling modes from "on" to "off," and generates third scheduling information. In the third scheduling information, the system identifies the switch status of the target communication mode. If the target communication mode is displayed as "off" in the third scheduling information, it is modified to "on." If the target communication mode is already "on" in the third scheduling information, it remains unchanged. After the above modifications, the third scheduling information is the required first scheduling information. This information reflects the final switch status of all scheduling modes under the current business needs, where the target communication mode has been correctly turned on according to the needs. In this way, the scheduling information can be dynamically adjusted according to changes in business needs to ensure that the target communication mode is given priority, and at the same time, unnecessary or no longer needed communication modes can be turned off, thereby optimizing resource utilization and improving efficiency.

[0160] As an example, assume that there is a distributed system that contains multiple business nodes that need to communicate with data sources. The system supports two communication modes: single mode and colony mode. Currently, the second scheduling information is as follows: single mode: on; colony mode: off. Now, due to changes in business requirements, the system decides to turn off the single mode and turn on the colony mode as the new target communication mode. First, the on state of the single mode in the second scheduling information is modified to the off state, and the third scheduling information is obtained: single mode: off; colony mode: off; next, the colony mode is identified as the new target communication mode, and the off state of the colony mode in the third scheduling information is modified to the on state, generating the first scheduling information: single mode: off; colony mode: on. The obtained first scheduling information instructs the business node to turn off the single mode and turn on the colony mode. The business node will adjust its communication mode according to this scheduling information to meet the new business requirements. You can see how to dynamically adjust the state of the communication mode according to changes in business needs, thereby optimizing system performance and resource usage.

[0161] In this way, by modifying the second scheduling information according to business needs to obtain the first scheduling information, significant flexibility and adaptability are demonstrated. It can ensure that business nodes respond to changes in business needs in a timely manner, and effectively improve resource utilization and system performance by dynamically adjusting the state of the communication mode. This strategy not only reduces the overhead of unnecessary communication modes, but also helps avoid potential conflicts and overloads, while ensuring the priority of the target communication mode, thereby optimizing the operating efficiency and stability of the overall system while meeting real-time business needs.

[0162] In this way, in the process of the service node connecting to the first data source for data processing, in response to the received first scheduling information being different from the second scheduling information stored locally in the service node, since the first scheduling information carries the access addresses of multiple data sources, and the multiple data sources include the second data source, based on the first scheduling information, the second data source that the service node needs to switch can be determined, and based on the access address of the second data source, the first data source connected to the service node is switched to the second data source. In this way, when the received first scheduling information is different from the second scheduling information stored locally in the service node, the second data source of the service node is determined based on the access addresses of the multiple data sources included in the first scheduling information, and based on the access address of the second data source, the first data source connected to the service node is switched to the second data source, thereby realizing real-time hot switching of the data source during the data processing of the service, so as to meet the data requirements of data processing for different data sources, thereby effectively improving the processing efficiency of data processing by the service node.

[0163] The following describes an exemplary application of the embodiments of the present application in an actual application scenario of a distributed service system.

[0164] The embodiment of the present application can be applied to the distributed deployment of microservices, modify the configuration file content of the redis data source of nacos, and realize the dynamic switching of the redis data source without restarting the listening service of the java nacos client. And the verification of the dynamic switching configuration is added to prevent the account password and host from being incorrect when modifying the configuration. It can be applied to the application scenarios such as redis fault switching, redsi mode upgrade, and single machine to cluster.

[0165] In some embodiments, see Figure 6 , Figure 6 This is a flow chart of the data processing method provided in the embodiment of the present application. Figure 3 The data processing method provided in the embodiment of the present application can be Figure 6 Steps 301 to 306 are shown to be implemented.

[0166] In step 301, register the nacos listener, load the acquired redis configuration class when the service starts, and put the host of the configuration class into the local cache.

[0167] In some embodiments, rewrite the RedisProperties class of spring-data-redis, and add the @RefreshScope annotation on the class. When the configuration is changed in the server interface of nacos, the corresponding content will also be modified synchronously on the corresponding properties. Only in this way can it take effect dynamically and in real time. When the java service of the integrated nacos client is initialized and started, the class of redisProperties that we rewrite is obtained. At this time, the custom RedisConnectionFactory is implemented. The custom RedisConnectionFactory in the embodiment of the present application has two properties, namely LettuceConnectionFactory and threadLocal. The existing technical method threadlocal stores String, and now LettuceConnectionFactory is stored. And it also provides a switching method to the outside, and the LettuceConnectionFactory object is switched. Write another spring automatic injection method, in which the rewritten redisProperti es are injected as parameters and encapsulated into a LettuceConnectionFactory object. It is set as a property, and threadLocal also stores the LettuceConnectionFactory object. It should be noted here that when switching, the threadlocal and remote methods must be executed first, otherwise memory leaks will occur easily when adding them.

[0168] In step 302, the configuration information of redis is modified in the configuration file of nacos.

[0169] In some embodiments, in the automatic injection method, these redisProperties attributes are cached in the local cache at the same time. Map is used here, the key is of String type, and the value is also of String type. The key is single, sentinel, colony, and we concatenate the values ​​differently according to the different keys. There are three types in total. If the redis connection is in stand-alone mode, the key is single, and the value is the corresponding redisProperties host and password attributes concatenated into a string. If it is cluster mode, the key is colony, and the value is the nodes attribute of the redisProperties subclass cluster concatenated into a string. If it is sentinel mode, the key is sentinel, and the value is the master, nodes, and password attributes of the redisProperties subclass Sentinel concatenated into a string.

[0170] In step 303, the client monitors the file change and compares the acquired configuration information with the local cache.

[0171] In some embodiments, the Java service adds a listener for the nacos client, and specifies that the file directory for listening to nacos contains the redis configuration file. When the redis configuration is modified in the nacos configuration interface. The client listener obtains the changes in the configuration file. Call a method to compare the local cache with the rewritten redisProperties. There are roughly three types here. The obtained redsiProperties attribute is concatenated with the value of the three keys in the local cache, and the values ​​have not changed. Then do not do anything. The obtained redsiPrope rties attribute is concatenated with the value of the three keys in the local cache, and one of them has changed and does not match. Then call a method of the custom RedisConnectionFactory, encapsulate redisProperties into LettuceConnectionFactory, and then set it again. At the same time, threadlocal is also assigned synchronously. The obtained redsiProperties attribute is concatenated with the value of the three keys in the local cache, and two of them have changed and do not match. It should be noted that this situation only occurs when switching modes, such as switching from a single machine to a cluster. It does not matter how many changes occur here. As long as there is a change, follow the same steps as the second method.

[0172] In step 304, it is determined whether the comparison has changed.

[0173] In some embodiments, when the nacos listener listens to file changes, a verification interface will be called synchronously here, that is, according to the obtained custom redsiProperties object, the properties are verified to see if they are correct and whether redsi can be connected and used normally. The verification method is to use the LettuceConnectionFactory object encapsulated by redisProperties to obtain the getcconnection object and execute the ping method. If an error is reported, or if the returned value is not pong, it means that the configuration modified in nacos cannot be used. It can also prevent the impact of misoperation on the existing environment.

[0174] In step 305, no processing is performed.

[0175] In some embodiments, if it is determined that there is no change, no processing is performed.

[0176] In step 306, a customized MultiRedisLettuceConnectionFactory object is obtained, the constructor method thereof is rewritten, and the currentRedisName attribute value thereof is reset.

[0177] In some embodiments, the RedisProperties class of spring-data-redis needs to be rewritten to add the dynamic refresh annotation of nacos, so that the obtained RedisProperties will change in real time and LettuceConnectionFactory will change. Here, MultiRedisLettuceConnectionFactory is also rewritten. The currentRedisName previously stored the String type, and now it is LettuceConnectionFactory, and the connectionFactoryMap attribute has also become LettuceConnectionFactory.

[0178] In this way, the operation and maintenance work under distributed microservice deployment is greatly reduced, and production efficiency is improved. There is no need to restart the service after modifying the configuration file, which reduces the workload. Because it is a hot update, the service can be provided uninterruptedly without stopping the service, which greatly ensures the stability of the service. A new redis configuration verification function is added to prevent misoperation and affect the normal operation of existing services.

[0179] In some embodiments, see Figure 7 , Figure 7It is a schematic diagram of the principle of the data processing method provided by the embodiment of the present application. ServiceA and serviceb are both clustered deployments, and each service has multiple instances. However, the connected nacos is the same. And each instance contains a nacos client to communicate with the nacos server. And it also listens to the configuration file of the server to obtain the changes of the configuration file in real time. ServiceA and serviceb are called by feign client. As can be seen from the legend, they share the same redis data source. Assuming that the redis database is down, the business cannot proceed normally at this time. It is necessary to switch to the backup redis data source urgently. At this time, the new data source can be configured in the configuration center of the nacos server. At this time, the clusters of serviceA and serviceb b are equipped with nacos client listeners, which monitor the changes in the configuration file and compare them with the local cache configuration to replace the custom RedisConnnectionFactory. It has good timeliness and greatly guarantees the stability of the service, without downtime or re-release. It also reduces the workload of operation and maintenance. The same is true for the upgrade of Redis mode. For example, the Redis used before the embodiment of this application is a stand-alone mode, which has obvious stability and reliability. At this time, you need to switch to cluster mode and update the Redis data source configuration in the configuration center of Nacos. Figure 7 There are three diagrams, including the architecture of microservices, and also the relationship between the nacos server and client.

[0180] It is understandable that in the embodiments of the present application, data related to traffic, etc. is involved. When the embodiments of the present application are applied to specific products or technologies, user permission or consent is required, and the collection, use and processing of relevant data need to comply with relevant laws, regulations and standards of relevant countries and regions.

[0181] The following is a description of an exemplary structure of the data processing device 455 provided in the embodiment of the present application implemented as a software module. In some embodiments, Figure 2 As shown, the software modules stored in the data processing device 455 of the memory 450 may include: a determination module 4551, used to determine the second data source of the business node based on the access addresses of multiple data sources included in the first scheduling information in response to the received first scheduling information being different from the second scheduling information locally stored in the business node; a switching module 4552, used to switch the first data source of the business node to the second data source based on the access address of the second data source.

[0182] In some embodiments, there are multiple communication modes between the business node and the data source, the first scheduling information includes first sub-scheduling information corresponding to each of the communication modes, and the second scheduling information includes second sub-scheduling information corresponding to each of the communication modes; the above-mentioned determination module is also used to compare the first sub-scheduling information and the second sub-scheduling information corresponding to each of the communication modes, and if the first sub-scheduling information and the second sub-scheduling information are different, it is determined that the first scheduling information is different from the second scheduling information.

[0183] In some embodiments, the first sub-scheduling information includes at least one first information field, and the second sub-scheduling information includes a second information field corresponding one-to-one to the first information field; the above-mentioned determination module is also used to compare the first information field with the corresponding second information field. If the first information field is different from the corresponding second information field, it is determined that the first scheduling information is different from the second scheduling information.

[0184] In some embodiments, the above-mentioned data processing device also includes: an authentication module, which is used to perform authority authentication on the first scheduling information based on the second scheduling information to obtain an authority authentication result; the authority authentication result is used to indicate whether the first scheduling information has the data source scheduling authority for the business node; the above-mentioned switching module is also used to determine the second data source of the business node based on the access addresses of multiple data sources included in the first scheduling information if the authority authentication result indicates that the first scheduling information has the data source scheduling authority for the business node.

[0185] In some embodiments, the first scheduling information includes first authentication information, and the second scheduling information includes second authentication information. The above-mentioned authentication module is also used to compare the first authentication information and the second authentication information; if the first authentication information and the second authentication information are the same, the authority authentication result is used to indicate that the first scheduling information has the data source scheduling authority for the business node; if the first authentication information is different from the second authentication information, the authority authentication result is used to indicate that the first scheduling information does not have the data source scheduling authority for the business node.

[0186] In some embodiments, the switching module is further configured to send a scheduling information acquisition request at a preset time interval, receive the first scheduling information returned in response to the scheduling information acquisition request, store the first scheduling information locally in the service node, and delete the second scheduling information locally stored in the service node.

[0187] The following further describes an exemplary structure of the data processing device 555 provided in the embodiment of the present application implemented as a software module. In some embodiments, Figure 3 As shown, the software modules stored in the data processing device 555 of the memory 550 may include: a generating module, used to generate first scheduling information for a business node; a sending module, used to send the first scheduling information to the business node in response to an information acquisition request sent by the business node; the first scheduling information is used to trigger the business node to determine a second data source and switch the connected first data source to the second data source when the first scheduling information is different from the second scheduling information locally stored in the business node.

[0188] In some embodiments, the above-mentioned generation module is also used to generate the first scheduling information if an adjustment request for the business node is received; if the first information is received, and the first information indicates that the first data source cannot meet the business requirements of the data processing, then the second scheduling information is modified according to the business requirements to obtain the first scheduling information.

[0189] In some embodiments, the above-mentioned generation module is also used to generate switch information for indicating that the communication mode is started for each of the communication modes if the communication mode is the target communication mode, and to generate switch information for indicating that the communication mode is not started if the communication mode is not the target communication mode; and to generate the first scheduling information based on the switch information of each of the communication modes.

[0190] In some embodiments, the above-mentioned generation module is also used to splice the switch information of the communication mode and the access address of at least one data source under the communication mode for each of the communication modes to obtain the first sub-scheduling information corresponding to the communication mode; and splice each of the first sub-scheduling information to obtain the first scheduling information.

[0191] In some embodiments, there are multiple communication modes between the service node and the data source, the service demand includes a demand for turning on a target communication mode among the multiple communication modes, and the second scheduling information includes switch information of each of the communication modes. The above-mentioned generation module is also used to modify the switch information indicating that the corresponding scheduling mode is turned on in the second scheduling information to switch information indicating that the corresponding scheduling mode is turned off, so as to obtain the third scheduling information; and modify the switch information indicating that the target communication mode is turned off in the third scheduling information to switch information indicating that the target communication mode is turned on, so as to obtain the first scheduling information.

[0192] The embodiment of the present application provides a computer program product, which includes a computer program or a computer executable instruction, and the computer program or the computer executable instruction is stored in a computer-readable storage medium. The processor of the electronic device reads the computer executable instruction from the computer-readable storage medium, and the processor executes the computer executable instruction, so that the electronic device executes the data processing method described above in the embodiment of the present application.

[0193] The embodiment of the present application provides a computer-readable storage medium storing computer-executable instructions, wherein the computer-executable instructions are stored. When the computer-executable instructions are executed by a processor, the processor will be caused to execute the data processing method provided by the embodiment of the present application, for example, Figure 4 The data processing method is shown.

[0194] In some embodiments, the computer-readable storage medium may be a memory such as FRAM, ROM, PROM, EPROM, EEPROM, flash memory, magnetic surface storage, optical disk, or CD-ROM; or it may be any electronic device including one or any combination of the above memories.

[0195] In some embodiments, computer executable instructions may be in the form of a program, software, software module, script or code, written in any form of programming language (including compiled or interpreted languages, or declarative or procedural languages), and may be deployed in any form, including as a stand-alone program or as a module, component, subroutine or other unit suitable for use in a computing environment.

[0196] As an example, computer-executable instructions may, but need not, correspond to a file in a file system, may be stored as part of a file storing other programs or data, such as in one or more scripts in a HyperText Markup Language (HTML) document, in a single file dedicated to the program in question, or in multiple coordinated files (e.g., files storing one or more modules, subroutines, or code portions).

[0197] As an example, computer executable instructions may be deployed to be executed on one electronic device, or on multiple electronic devices located at one site, or on multiple electronic devices distributed at multiple sites and interconnected by a communication network.

[0198] In summary, the embodiments of the present application have the following beneficial effects:

[0199] (1) In the process of the service node connecting to the first data source for data processing, in response to the received first scheduling information being different from the second scheduling information stored locally in the service node, since the first scheduling information carries the access addresses of multiple data sources, the multiple data sources include the second data source, based on the first scheduling information, the second data source that the service node needs to switch can be determined, and based on the access address of the second data source, the first data source connected to the service node is switched to the second data source. In this way, when the received first scheduling information is different from the second scheduling information stored locally in the service node, the second data source of the service node is determined based on the access addresses of the multiple data sources included in the first scheduling information, and based on the access address of the second data source, the first data source connected to the service node is switched to the second data source, thereby realizing real-time hot switching of the data source during the data processing of the service, so as to meet the data requirements of data processing for different data sources, thereby effectively improving the processing efficiency of data processing by the service node.

[0200] (2) The service node can ensure that it adjusts the data processing logic according to the latest first scheduling information, thereby ensuring efficient and accurate operation of the service. At the same time, this also enables the service node to flexibly adapt to changes in the scheduling system and improve the reliability and flexibility of the system.

[0201] (3) The business node can ensure that only the first scheduling information with the correct authority can schedule the data source, thereby protecting the security of the data and the compliance of the business. At the same time, this also enables the business node to flexibly adapt to changes in the scheduling system and improve the reliability and flexibility of the system.

[0202] (4) The business node can ensure that the scheduling operation it performs is within the scope of authority, thereby protecting the security of data and the compliance of business. At the same time, this also enables the business node to flexibly adapt to changes in the scheduling system and improve the reliability and flexibility of the system. By performing authority authentication on the first scheduling information, the business node can ensure that only information with correct authority can schedule the data source, thereby protecting the security of data and the compliance of business. The business node can flexibly adjust its processing logic according to the authority authentication result, which can not only respond to the update of the first scheduling information, but also prevent unauthorized information from interfering with the business process. Even when the first scheduling information changes, the business node can continue to stably perform data processing based on the first data source according to the authority authentication result, ensuring the continuity and consistency of the business. The record of the authority authentication process and processing results facilitates tracking and auditing, and can quickly perform troubleshooting and performance analysis when problems arise. Authority authentication ensures that only authenticated information can schedule the business node, improving the overall reliability and stability of the system.

[0203] (5) By dynamically switching between different data sources, business nodes can utilize the best performance of each data source, thereby improving the overall data processing speed and efficiency. When a new data source is added to the data source cluster, the system can automatically switch according to the first scheduling information without manual intervention, thereby conveniently expanding the system capacity and processing power. The first scheduling information provides flexibility in data source selection, allowing business nodes to dynamically adjust data source usage policies based on actual load, performance or priority. Even if a data source fails, the first scheduling information can guide the business node to switch to other normal data sources, thereby reducing the impact of system failures and ensuring business continuity. The first scheduling information can guide the business node to select a data source based on the current load of the data source, thereby avoiding resource waste and achieving more efficient resource utilization. Through the first scheduling information, the business node can automatically manage complex data source configuration and switching, reducing manual intervention and management complexity. The first scheduling information can help business nodes avoid accessing unsafe or non-compliant data sources, ensuring data security and compliance with regulatory requirements.

[0204] (6) By disconnecting or establishing a connection based on the access address of the second data source, the business node can more effectively utilize the resources in the data source cluster and avoid wasting resources. By maintaining the connection with the second data source and disconnecting other non-second data sources, the business node can optimize the performance of data processing and improve the response speed of the system. In the case where the second data source is not included in the data source cluster, the business node can quickly disconnect from the failed or degraded data source cluster and establish a connection with the second data source, thereby improving the fault tolerance of the system. Whether maintaining a connection with the second data source or establishing a connection with the second data source, the business node can ensure business continuity and reduce business interruptions caused by data source problems. Data source connection management under the guidance of the first scheduling information reduces the need for manual intervention and simplifies the complexity of system management. By selecting the second data source through the first scheduling information, the business node can ensure that data access and processing are carried out within a safe range and protect the data from being accessed by unauthorized data sources.

[0205] (7) Regularly sending information acquisition requests enables the business node to continuously receive the latest first scheduling information, so that it can adjust its work strategy in time to respond to changes in the system status. The business node can flexibly adjust its behavior according to the first scheduling information, such as adjusting the data processing priority, selecting different data sources, executing specific business logic, etc. Through the first scheduling information, the business node can detect potential problems, such as the unavailability or performance degradation of the data source, and take corresponding fault-tolerant measures, such as switching to the backup data source or adjusting the load balancing strategy. The first scheduling information can help the business node optimize the use of resources. For example, when the data source load is high, it can avoid allocating the workload to the data source, thereby improving resource utilization efficiency. Through the first scheduling information, the business node can ensure that its data processing can be carried out continuously and can be quickly restored even in the event of system failure or data source unavailability. Regularly receiving the first scheduling information helps ensure that the business node can adapt to the continuous changes in the system status, thereby improving the stability of the entire system. Through the automated information acquisition request and processing mechanism, the management complexity of the business node is simplified, reducing the need for manual intervention. The first scheduling information can help the business node ensure that the data and behavior it processes meet security and compliance requirements, such as avoiding processing sensitive data or complying with specific data processing rules. By regularly receiving the first scheduling information, the business node can respond to changes in the system status in real time, improving the system's real-time performance, flexibility, fault tolerance, resource optimization, business continuity, system stability, management simplification and security.

[0206] (8) Greatly reduce the maintenance work under distributed microservice deployment and improve production efficiency. There is no need to restart the service after modifying the configuration file, which reduces the workload. Because it is a hot update, it can provide services without stopping the service, which greatly ensures the stability of the service. Added redis configuration verification function to prevent misoperation and affect the normal operation of existing services.

[0207] (9) By modifying the second scheduling information according to business needs to obtain the first scheduling information, it shows significant flexibility and adaptability. It can ensure that the business nodes respond to changes in business needs in a timely manner, and effectively improve resource utilization and system performance by dynamically adjusting the state of the communication mode. This strategy not only reduces the overhead of unnecessary communication modes, but also helps to avoid potential conflicts and overloads, while ensuring the priority of the target communication mode, thereby optimizing the operating efficiency and stability of the overall system while meeting real-time business needs.

[0208] (10) By concatenating the switch information of each communication mode and the data source access address into a string to generate the first scheduling information, this implementation method is beneficial to simplify the information transmission and processing process. It not only reduces data redundancy and improves the efficiency of data transmission, but also facilitates the automatic parsing and execution of scheduling instructions. In addition, the scheduling information in the form of a string is easy to share and transmit between different system components, which helps to achieve centralized management and enhances the flexibility and maintainability of system configuration.

[0209] (11) The dynamic scheduling strategy based on the target communication mode improves the response speed and flexibility to business needs, making it possible to quickly adjust the communication mode according to the real-time situation, thereby optimizing data transmission and processing efficiency. Secondly, through clear switch information indication, the system can clearly control the status of each communication mode, avoid unnecessary communication overhead, reduce potential security risks, and ensure that resources are used reasonably. In addition, this implementation method also improves the reliability and stability of the system because it can automatically adjust the communication strategy according to the availability of data sources and changes in business needs, thereby avoiding service interruptions caused by the failure of a single communication mode. Finally, this dynamic scheduling method is also convenient for system administrators to monitor and maintain, because they can easily understand the currently activated communication mode and perform further management and optimization as needed.

[0210] The above is only an embodiment of the present application and is not intended to limit the protection scope of the present application. Any modifications, equivalent substitutions and improvements made within the spirit and scope of the present application are included in the protection scope of the present application.

Claims

1. A data processing method, characterized in that: The method comprises: In response to the received first scheduling information being different from the second scheduling information locally stored in the service node, determining a second data source of the service node based on access addresses of a plurality of data sources included in the first scheduling information; Based on the access address of the second data source, the first data source of the service node is switched to the second data source.

2. The method according to claim 1, characterized in that There are multiple communication modes between the service node and the data source, the first scheduling information includes first sub-scheduling information corresponding to each of the communication modes, and the second scheduling information includes second sub-scheduling information corresponding to each of the communication modes; Before determining the second data source of the service node based on the access addresses of the multiple data sources included in the first scheduling information, the method further includes: For each of the communication modes, the first sub-scheduling information and the second sub-scheduling information corresponding to the communication mode are compared. If the first sub-scheduling information and the second sub-scheduling information are different, it is determined that the first scheduling information and the second scheduling information are different.

3. The method according to claim 2, characterized in that The first sub-scheduling information includes at least one first information field, and the second sub-scheduling information includes a second information field corresponding to the first information field one by one; The comparing the first sub-scheduling information and the second sub-scheduling information corresponding to the communication mode, and if the first sub-scheduling information is different from the second sub-scheduling information, determining that the first scheduling information is different from the second scheduling information includes: The first information field is compared with the corresponding second information field. If the first information field is different from the corresponding second information field, it is determined that the first scheduling information is different from the second scheduling information.

4. The method according to claim 1, characterized in that: Before determining the second data source of the service node based on the access addresses of the multiple data sources included in the first scheduling information, the method further includes: Based on the second scheduling information, performing authority authentication on the first scheduling information to obtain an authority authentication result; The determining the second data source of the service node based on the access addresses of the plurality of data sources included in the first scheduling information includes: If the authority authentication result indicates that the first scheduling information has the data source scheduling authority for the service node, a second data source of the service node is determined based on access addresses of multiple data sources included in the first scheduling information.

5. The method according to claim 4, characterized in that The first scheduling information includes first authentication information, the second scheduling information includes second authentication information, and the first scheduling information is authenticated based on the second scheduling information to obtain an authentication result, including: comparing the first authentication information and the second authentication information; If the first authentication information and the second authentication information are the same, the authority authentication result is used to indicate that the first scheduling information has the data source scheduling authority for the service node; If the first authentication information is different from the second authentication information, the authority authentication result is used to indicate that the first scheduling information does not have the data source scheduling authority for the service node.

6. The method according to claim 1, characterized in that In response to the received first scheduling information being different from the second scheduling information locally stored in the service node, before determining the second data source of the service node based on the access addresses of the multiple data sources included in the first scheduling information, the method further includes: Send a scheduling information acquisition request at a preset time interval; Receiving the first scheduling information returned in response to the scheduling information acquisition request; Before switching the first data source of the service node to the second data source based on the access address of the second data source, the method further includes: The first scheduling information is stored locally in the service node, and the second scheduling information stored locally in the service node is deleted.

7. A data processing method, characterized in that: The method comprises: Generate first scheduling information for a business node, and send the first scheduling information to the business node in response to an information acquisition request sent by the business node, wherein the first scheduling information is used to trigger the business node to determine a second data source and switch the connected first data source to the second data source when the first scheduling information is different from second scheduling information locally stored in the business node.

8. The method according to claim 7, characterized in that The generating first scheduling information for the service node includes: If an adjustment request for the service node is received, generating the first scheduling information; If first information is received, and the first information indicates that the first data source cannot meet the business requirements of the data processing, the second scheduling information is modified according to the business requirements to obtain the first scheduling information.

9. The method according to claim 8, characterized in that There are multiple communication modes between the service node and the data source, and the generating the first scheduling information includes: For each of the communication modes, if the communication mode is a target communication mode, switch information for indicating that the communication mode is activated is generated, and if the communication mode is not the target communication mode, switch information for indicating that the communication mode is not activated is generated; The first scheduling information is generated based on the switch information of each of the communication modes.

10. The method according to claim 9, characterized in that The generating the first scheduling information based on the switch information of each of the communication modes includes: For each of the communication modes, concatenate the switch information of the communication mode and the access address of at least one data source in the communication mode to obtain first sub-scheduling information corresponding to the communication mode; Each of the first sub-scheduling information is concatenated to obtain the first scheduling information.

11. The method according to claim 8, characterized in that There are multiple communication modes between the service node and the data source, the service demand includes a demand for starting a target communication mode among the multiple communication modes, and the second scheduling information includes switch information of each of the communication modes; The modifying the second scheduling information according to the service requirement to obtain the first scheduling information includes: Modify the switch information indicating that the corresponding scheduling mode is turned on in the second scheduling information to switch information indicating that the corresponding scheduling mode is turned off, to obtain third scheduling information; The switch information indicating that the target communication mode is turned off in the third scheduling information is modified to switch information indicating that the target communication mode is turned on, so as to obtain the first scheduling information.

12. A data processing device, characterized in that: The device comprises: a determination module, configured to determine, in response to the received first scheduling information being different from the second scheduling information locally stored in the service node, a second data source of the service node based on access addresses of a plurality of data sources included in the first scheduling information; A switching module is used to switch the first data source of the service node to the second data source based on the access address of the second data source.

13. A data processing device, characterized in that: The device comprises: A generating module, used to generate first scheduling information for a service node; A sending module, configured to send the first scheduling information to the service node in response to an information acquisition request sent by the service node; The first scheduling information is used to trigger the service node to determine a second data source and switch the connected first data source to the second data source when the first scheduling information is different from second scheduling information locally stored in the service node.

14. An electronic device, characterized in that: The electronic device comprises: A memory for storing computer executable instructions or computer programs; The processor is used to implement the data processing method described in any one of claims 1 to 6, or 7 to 11 when executing the computer executable instructions or computer programs stored in the memory.

15. A computer-readable storage medium storing computer-executable instructions, characterized in that: When the computer executable instructions are executed by a processor, the data processing method described in any one of claims 1 to 6, or 7 to 11 is implemented.

16. A computer program product comprising a computer program or computer executable instructions, characterized in that When the computer program or computer executable instructions are executed by a processor, the data processing method according to any one of claims 1 to 6, or 7 to 11 is implemented.

Images