A multi-source computing power data integration and intelligent scheduling system and method

Abstract

The application discloses a kind of multi-source computing power data integration and intelligent scheduling system and method, it is related to electric digital data processing technical field.The application includes segmented collection according to data source to multi-source computing power dataset;Associated resource feature description file of single computing power data in multi-source computing power dataset is constructed, when single computing power data content changes, feature information in resource feature description file is updated synchronously;Single computing power data and resource feature description file are encapsulated into scheduling unit.The application carries out segmented collection according to data source to computing power dataset, ensures the accuracy and pertinence of data acquisition process, adapts the characteristics of dynamic and changeable computing power resources under multi-source heterogeneous environment, at the same time, constructs resource feature description file and updates feature information synchronously, so that scheduling system can master computing power resource state and its change situation in real time, to effectively improve the accuracy and response speed of scheduling decision.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of digital data processing, in particular to a multi-source computing power data integration and intelligent scheduling system and method. BACKGROUND

[0002] In the cloud computing and edge computing environment, computing power resources are often scattered in multiple heterogeneous data sources, such as distributed data centers, terminal devices and public cloud platforms, so that data integration often faces problems such as non-uniform protocols and inconsistent feature descriptions, resulting in low resource utilization. Therefore, a multi-source computing power data integration and intelligent scheduling system and method need to be designed.

[0003] Through retrieval, the Chinese invention patent application with the application publication number "CN111078700A" discloses a "data synchronization method and system based on block chain". The application stores data in the form of JSON string, which has small data volume, low requirements for storage space and bandwidth, ensures the real-time of transmission, and the index data is encrypted before chaining. The data synchronization is ciphertext data, which ensures the security of data in the transmission process.

[0004] In addition, the Chinese invention patent application with the application publication number "CN114968285A" discloses an "application updating method, device and equipment based on block chain". The application stores application installation package and corresponding application version information through block chain, so that corresponding information can be obtained from block chain and compared. When the version needs to be updated, the application installation package is automatically obtained from the block chain and updated, which ensures the security of the application installation process and reduces the cumbersome steps of manual updating of the user. At the same time of ensuring that the application is installed in time, the user's experience is improved.

[0005] However, the above two patents and similar prior art solutions cannot segment the data collection according to the data source, lack a structured integration mechanism for heterogeneous resource differences, and are difficult to adapt to changes in scheduling demand caused by dynamic changes in computing power resources, so that the scheduling system still relies on static information in the task allocation process, the decision basis is lagging behind, and it is difficult to support efficient intelligent scheduling in the multi-node and multi-task concurrent scene. SUMMARY

[0006] The purpose of the present application is to provide a multi-source computing power data integration and intelligent scheduling system and method to solve the problems raised in the background art.

[0007] To achieve the above purpose, the present application provides the following technical solutions.

[0008] In a first aspect, a multi-source computing power data integration and intelligent scheduling method is provided, comprising:

[0009] According to the data source, the multi-source computing power data set is segmented and collected;

[0010] A resource feature description file associated with a single computing power data in the multi-source computing power data set is constructed, and when the content of the single computing power data changes, the feature information in the resource feature description file is updated synchronously;

[0011] The single computing power data and the resource feature description file are encapsulated into a scheduling unit, and the scheduling unit generates a scheduling node after reading the feature information;

[0012] An authority regulation channel is constructed between multiple scheduling nodes through a node cooperation protocol, and different computing power data corresponding contents are transmitted through the authority regulation channel in turn;

[0013] The priority order of the authority regulation channel is arranged according to the data source;

[0014] The content scheduling of the single computing power data is performed by the corresponding scheduling node, and the corresponding scheduling node realizes content modification by traversing the scheduling node from bottom to top according to the priority order;

[0015] When the scheduling unit reads the changed feature information, a regulation threshold value of a manually set value is set at the corresponding scheduling node;

[0016] When the regulation threshold value is triggered, the authority order of the authority regulation channel is reinitialized.

[0017] As a further preferred technical solution, the construction method of the resource feature description file comprises:

[0018] The resource parameters associated with the single computing power data are collected;

[0019] The running state information and the historical scheduling record of the scheduling node are obtained;

[0020] The resource parameters, the running state information and the historical scheduling record are encapsulated as structured data;

[0021] A corresponding identifier is set for each structured data to form a resource feature description file.

[0022] As a further preferred technical solution, the feature information synchronous updating method in the resource feature description file comprises:

[0023] When the scheduling unit detects that the feature information changes, an adjustment instruction is sent to the corresponding scheduling node, and the authority regulation channel connection between the corresponding scheduling node and the remaining scheduling nodes is interrupted;

[0024] After the feature information changes end, the corresponding regulation node reconnects the authority regulation channel, and the updated resource feature description file is propagated to the remaining scheduling nodes.

[0025] The remaining nodes perform a verification operation on the received updated feature information, and confirm the consistency of the content change by comparing the feature attributes of the resource feature description file before and after the change;

[0026] If the verification is passed, the scheduling node marks the updated feature information as a valid state identifier, and triggers the scheduling unit to reload the feature file;

[0027] If the verification fails, the node generates an exception diagnosis report, performs version rollback to the feature state before the change, and reinitiates the feature synchronization update process.

[0028] As a further preferred embodiment of the technical solution, the generation method of the scheduling node comprises:

[0029] According to the data source, a preset node generation template is matched for each computing power data in the multi-source computing power data set;

[0030] According to the node generation template, a corresponding scheduling node instance is generated by calling a bottom layer resource allocation interface;

[0031] For each created scheduling node instance, a unique node identifier is allocated, and it is registered in the global scheduling node directory for unified indexing and management;

[0032] The running state parameters of each newly registered scheduling node are initialized, and its state is marked as "to be activated;

[0033] When the scheduling unit first loads the task feature information to the node, the self-checking process of the scheduling node is automatically triggered, and the self-checking content includes the integrity verification of the scheduling node function module and the access permission verification of the target computing power resource;

[0034] If the self-checking of the scheduling node is passed, the scheduling node state is updated to "ready state", and the scheduling node is added to the candidate queue of the permission control channel.

[0035] As a further preferred embodiment of the technical solution, the content of the node coordination protocol comprises:

[0036] The starting scheduling node sends a verification code to the branch scheduling node;

[0037] The branch scheduling node performs identity verification after receiving the verification code;

[0038] After the verification is passed, the branch scheduling node initiates a connection request to the starting scheduling node;

[0039] After receiving the connection request, the starting scheduling node establishes a permission control channel with the branch scheduling node.

[0040] As a further preferred technical solution, the content scheduling of single computing power data is performed by a method of the corresponding scheduling node, which comprises:

[0041] According to the priority order, the node identifier of the target computing power data corresponding to the current scheduling node is obtained;

[0042] Based on the node identifier, the corresponding scheduling node instance is located in the global scheduling node directory;

[0043] Determine whether the state of the scheduling node instance is "ready state";

[0044] If the state is "ready state", send a scheduling instruction to the scheduling node instance according to the priority order;

[0045] After the scheduling node instance receives the instruction, it performs access, transmission or modification operation on the target computing power data;

[0046] When the scheduling operation involves cross-node data interaction, the scheduling node acts as the starting scheduling node or the branch scheduling node, and initiates or responds to the establishment request of the authority control channel through the node cooperation protocol;

[0047] After the scheduling operation is completed, the scheduling node instance updates its historical scheduling record, and feeds back the operation state and result to the scheduling unit;

[0048] If the state of the scheduling node instance is not "ready state", the next candidate scheduling node in the priority order is automatically transferred for task allocation.

[0049] As a further preferred technical solution, the method for constructing the adjustment threshold comprises:

[0050] The scheduling unit configures a preset life parameter for each scheduling node according to the data source type, and the life parameter includes the maximum allowed synchronous update times, the scheduling frequency limit and the resource use period;

[0051] The actual life of the scheduling node is dynamically accumulated according to the synchronous update, resource access and task scheduling behavior of the computing power data in the transmission and scheduling process;

[0052] When the real-time life of the scheduling node reaches or exceeds the preset threshold, the scheduling unit marks the node exceeding the preset threshold as "termination state", and terminates the connection between the scheduling unit and the authority control channel.

[0053] In the second aspect, in order to improve the above technical solution, the present application provides a system capable of running the above-mentioned multi-source computing power data integration and intelligent scheduling method, comprising: a multi-source data acquisition module for segmenting and collecting multi-source computing power data set according to data source;

[0054] The resource feature description module is configured to collect resource parameters, scheduling node running state information and historical scheduling records associated with single computing power data, construct a resource feature description file, and synchronously update feature information in the resource feature description file.

[0055] The scheduling unit management module is configured to encapsulate single computing power data and corresponding resource feature description files into scheduling units, generate scheduling nodes based on feature information, and manage generation, self-checking and state updating of the scheduling nodes.

[0056] The permission control channel module is configured to construct permission control channels between multiple scheduling nodes through a node cooperation protocol, and realize transmission of different computing power data contents.

[0057] The priority scheduling control module is configured to arrange priority orders of the permission control channels according to data sources, and perform scheduling operations of the computing power data contents according to the priority orders.

[0058] The feature synchronous updating module is configured to monitor changes of the resource feature description file, and control synchronous updating, verification and exception handling of feature information between the scheduling nodes.

[0059] The adjustment threshold management module is configured to calculate an adjustment threshold based on a scheduling frequency, a computing power resource fluctuation range and a node response delay, and reinitialize permission orders of the permission control channels when the threshold is triggered.

[0060] The global scheduling node directory module is configured to store and manage node identifiers, states and related information of all scheduling nodes, and realize unified indexing and scheduling node management.

[0061] As a further preferred technical solution, the modules are structurally connected through ordered data channels and control interfaces, the multi-source data collection module is directly connected with the resource feature description module, the collected data is transmitted to the resource feature description module to generate a feature file, the resource feature description module is connected with the scheduling unit management module through a communication interface, and feature data is provided to the scheduling unit management module to generate scheduling units, the scheduling unit management module and the node generation and management module are bidirectionally connected, used for triggering creation of the scheduling nodes and state management of the scheduling nodes, the scheduling nodes are connected with the permission control channel module through a unified interface after being generated, a communication link between the scheduling nodes is constructed, the priority scheduling control module is connected with the scheduling unit management module, used for controlling a node task execution order according to a scheduling instruction, the feature synchronous updating module is connected with the resource feature description module and the permission control channel module, used for monitoring changes and coordinating node synchronization, and the adjustment threshold management module is connected with the scheduling unit management module, used for dynamically controlling reordering of the permission channels according to a task condition.

[0062] Compared with the prior art, the technical solution has the following beneficial effects:

[0063] The multi-source computing power data integration and intelligent scheduling system and method, by segmenting the computing power data set according to the data source for collection, ensures the accuracy and pertinence of the data collection process, adapts to the dynamic and variable characteristics of computing power resources in a multi-source heterogeneous environment, at the same time, constructs a resource feature description file and synchronously updates the feature information, so that the scheduling system can master the computing power resource state and its change in real time, thereby effectively improving the accuracy and response speed of scheduling decision;

[0064] In addition, by encapsulating the computing power data and the resource feature description file into the scheduling unit, and constructing the permission control channel between the scheduling nodes through the node cooperation protocol, the ordered deployment and safe transmission of the computing power resources are realized, different computing power data is transmitted in priority order under the permission control, avoiding resource conflicts and access bottlenecks, and ensuring the scheduling efficiency and overall operation stability;

[0065] Thirdly, in the scheduling node management aspect, by generating a template and a global scheduling node directory, the automatic generation, identification and state management of the scheduling node are realized, each scheduling node performs self-checking before receiving the task, including resource access permission verification and function module integrity check, which guarantees the execution reliability of the scheduling node from the source, and through the state management mechanism, the "to be activated" and "ready state" states of the node are accurately controlled, providing a basic guarantee for the stable operation of the system. BRIEF DESCRIPTION OF DRAWINGS

[0066] Figure 1 The step flowchart of the present application is disclosed;

[0067] Figure 2 The auxiliary explanatory diagram of step S200 of the present application is provided;

[0068] Figure 3 The auxiliary explanatory diagram of step S600 of the present application is provided;

[0069] Figure 4 The module composition diagram of the system disclosed by the present application is provided. DETAILED DESCRIPTION

[0070] The technical solutions in the embodiments of the present application will be described clearly and completely in combination with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0071] It should be clear that the application scenario in which the technical solutions in the present application are proposed is an office scenario, and the multi-source computing power data set is used to represent an application document, and the data sources are mainly various department devices capable of adjusting the content of the application document.

[0072] For details, please refer to Figure 1 As can be seen, this application proposes a method for multi-source computing power data integration and intelligent scheduling, including: steps S100-S800.

[0073] Step S100: Collect data from the multi-source computing power dataset in segments according to the data source.

[0074] Specifically, step S100, which involves segmented data collection, refers to collecting data in segments according to the adjustment content of the corresponding department in the application document.

[0075] Step S200: Construct a resource feature description file associated with a single computing power data point within the multi-source computing power dataset.

[0076] It should be noted that in step S200, when the content of a single computing power data changes, the feature information in the resource feature description file is updated synchronously.

[0077] It is worth noting that in step S200, the application of individual computing power data to the actual office scenario is a single department's restriction on the location and content of modifications to the application document.

[0078] As a supplement to step S200, the method for constructing the resource feature description file includes steps S201-S204.

[0079] Step S201: Collect the resource parameters associated with a single computing power data point.

[0080] Step S202: Obtain the running status information and historical scheduling records of the scheduling node.

[0081] Step S203: Encapsulate resource parameters, running status information, and historical scheduling records into structured data.

[0082] Step S204: Set a corresponding identifier for each structured data to form a resource feature description file.

[0083] It should be noted that when the content of steps S201 to S204 is applied to actual office scenarios, when a department's equipment makes modifications to a specific location in the application document, the resource feature description file will record the change information of computing power data (corresponding location in the application document), including resource parameters, scheduling node running status, and historical records, to ensure that the feature description accurately reflects the current office needs. For example, when the finance department adjusts the budget form, the resource feature description file will automatically update the relevant identifiers to provide a reliable basis for subsequent scheduling.

[0084] As a preferred implementation scheme, refer to Figure 2It can be seen that the embodiment is mainly used for supplementing the feature information synchronization updating method in the resource feature description file in the step S200, and specifically includes steps S210-S250.

[0085] Step S210: When the scheduling unit detects that the feature information changes, the corresponding scheduling node sends an adjustment instruction and interrupts the authority control channel connection with the remaining scheduling nodes.

[0086] Step S220: After the feature information changes are terminated, the corresponding control node reconnects the authority control channel and propagates the updated resource feature description file to the remaining scheduling nodes.

[0087] Step S230: The remaining nodes perform verification operations on the received updated feature information, and confirm the consistency of the content changes by comparing the feature attributes of the resource feature description files before and after the changes.

[0088] Step S240: If the verification is passed, the scheduling node marks the updated feature information as a valid state identifier, and triggers the scheduling unit to reload the feature file.

[0089] Step S250: If the verification fails, the node generates an exception diagnosis report, performs version rollback to the feature state before the change, and reinitiates the feature synchronization updating process.

[0090] It should be noted that steps S210-S250 are applied to actual office scenarios, for example, when the personnel department modifies the employee archive table, the feature information change is monitored, the editing authority channel of other department devices is interrupted, and after the modification is completed and verified, the updated feature is automatically reconnected and propagated. Ensure that all nodes access the same version of application documents to avoid data conflicts.

[0091] Step S300: Encapsulate the single computing power data and the resource feature description file into the scheduling unit, and generate the scheduling node after the scheduling unit reads the feature information.

[0092] It should be noted that step S300 is applied to actual office scenarios, and after a department device modifies a specific location of an application document, the scheduling unit integrates the encapsulated computing power data and the feature description file into an independent node, which is convenient for subsequent scheduling operations. For example, when the marketing department updates the promotion strategy part of the marketing plan document, the scheduling unit reads the feature information to generate a corresponding scheduling node, which ensures that the node can respond to subsequent requirements in real time, such as automatically allocating resources to other departments for collaborative review. It should be noted that in the actual office scenario, the scheduling unit refers to a central scheduling service station, and the scheduling node refers to a processing computer of each department, which is convenient for subsequent resource allocation and collaborative control.

[0093] Specifically, the generation method of the scheduling node in step S300 includes steps S301-S306.

[0094] Step S301: According to the data source, a preset node generation template is matched for each computing power data in the multi-source computing power data set.

[0095] Step S302: According to the node generation template, a corresponding scheduling node instance is generated by calling a bottom layer resource allocation interface.

[0096] Step S303: For each created scheduling node instance, a unique node identifier is assigned, and it is registered in a global scheduling node directory for unified indexing and management.

[0097] Step S304: The running state parameters of each newly registered scheduling node are initialized, and its state is marked as “to be activated.

[0098] Step S305: When the scheduling unit first loads task characteristic information to the node, the self-checking process of the scheduling node is automatically triggered, and the self-checking content includes integrity verification of the scheduling node function module and access permission verification of the target computing power resource.

[0099] Step S306: If the self-checking of the scheduling node passes, the state of the scheduling node is updated to “ready state”, and the scheduling node is added to the candidate queue of the permission control channel.

[0100] It is worth noting that steps S301-S306 are applied to actual office scenarios. For example, when the marketing department updates the promotion strategy part of the marketing plan document, the scheduling unit (central scheduling service station) will match a preset node generation template for the department's device according to the data source (the department corresponding to the corresponding part in the application document), call a bottom layer resource allocation interface to generate a corresponding scheduling node instance, assign a unique node identifier to each instance and register it in a global scheduling node directory, initialize the running state parameter to “to be activated”, automatically trigger integrity verification of the scheduling node function module and access permission verification when first loading task characteristic information such as document modification requirements, update the state to “ready state” if the self-checking passes, and add it to the candidate queue of the permission control channel, so that the scheduling node (processing computer) can process the collaborative review task in real time.

[0101] Step S400: A permission control channel is constructed between multiple scheduling nodes through a node collaboration protocol, and different computing power data corresponding contents are transmitted through the permission control channel in sequence.

[0102] It is worth noting that the content of the node coordination protocol in step S400 includes: the starting scheduling node sends a verification code to the branch scheduling node, the branch scheduling node performs identity verification after receiving the verification code, and the branch scheduling node initiates a connection request to the starting scheduling node after the verification is passed. After the starting scheduling node receives the connection request, the authority control channel with the branch scheduling node is established.

[0103] It should be noted that when step S400 is applied to an actual office scenario, for example, when the research and development department initiates a collaborative editing request for an application document, the starting scheduling node (the processing computer of the research and development department) sends a dynamic verification code to the device of the legal department (the branch scheduling node) that needs to participate in the audit. After receiving the verification code, the device of the legal department needs to complete identity verification and authority matching through the scheduling unit (central scheduling service station). After the verification is passed, the device of the legal department initiates an encrypted connection request to the device of the research and development department. The two devices establish a secure authority control channel under the supervision of the central scheduling service station. The authority control channel will ensure that the modification content of a specific chapter of the application document is encrypted end-to-end during transmission, and the real-time verification of whether the legal department has the review authority of the current version of the document is performed. If the authority verification fails (for example, the legal department does not have the access authorization of the latest version), the channel will automatically interrupt the transmission and trigger the authority alarm mechanism.

[0104] Step S500: Arrange the priority order of the authority control channel according to the data source.

[0105] It is worth noting that when step S500 is applied to an actual office scenario, for example, when multiple departments (such as research and development, legal, and finance) simultaneously initiate collaborative editing requests for a multi-source computing power data set (application document), the scheduling unit (central scheduling service station) will dynamically arrange the priority order of the authority control channel according to the data source (such as department level), ensuring that the requests of high-priority departments (such as management) are given priority in establishing connection channels, while low-priority departments (such as support departments) need to wait in the queue for processing, thereby optimizing overall office efficiency and avoiding resource contention.

[0106] Step S600: The content scheduling of a single computing power data is performed by the corresponding scheduling node, and the corresponding scheduling node implements content modification by traversing the scheduling nodes from bottom to top according to the priority order.

[0107] It should be noted that referring to Figure 3 It can be seen that step S600, when actually used, includes steps S601-S608.

[0108] Step S601: According to the priority order, obtain the node identifier of the target computing power data corresponding to the current scheduling node.

[0109] Step S602: Based on the node identifier, locate the corresponding scheduling node instance in the global scheduling node directory.

[0110] Step S603: Determine whether the state of the scheduling node instance is "ready state".

[0111] Step S604: If the state is "ready state", send a scheduling instruction to the scheduling node instance according to the priority order.

[0112] Step S605: After the scheduling node instance receives the instruction, it performs access, transmission or modification operations on the target computing power data.

[0113] Step S606: When the scheduling operation involves cross-node data interaction, the scheduling node acts as the starting scheduling node or the branch scheduling node, and initiates or responds to the establishment request of the authority regulation channel through the node coordination protocol.

[0114] Step S607: After the scheduling operation is completed, the scheduling node instance updates its historical scheduling record and feeds back the operation state and result to the scheduling unit.

[0115] Step S608: If the state of the scheduling node instance is not "ready state", automatically switch to the next candidate scheduling node in the priority order for task allocation.

[0116] It is worth noting that steps S601-S608 apply to actual office scenarios, for example, when the R&D department needs to modify the core algorithm description content in the multi-source computing power data set (application document), the corresponding scheduling node (the processing computer of the R&D department) will first obtain the node identifier of the target computing power data (algorithm description part) according to the priority order preset by the scheduling unit (central scheduling service station) (for example, the priority of legal compliance review is higher than that of technical detail modification), locate the scheduling node (the processing computer of the legal department) instance of the legal department in the global scheduling node directory, and then determine that the scheduling node (the processing computer of the legal department) instance is in the "ready state" before sending a scheduling instruction to it. After receiving the instruction, the scheduling node (the processing computer of the legal department) of the legal department performs access and compliance review operation on the algorithm description part, and after the review operation is completed, the (processing computer of the legal department) updates the historical scheduling record and feeds back the result to the scheduling unit (central scheduling service station). Subsequently, the scheduling unit (central scheduling service station) sends a scheduling instruction to the next order scheduling node instance (i.e. the processing computer of the R&D department) according to the preset priority order. The scheduling node (the processing computer of the R&D department) of the R&D department as a branch scheduling node initiates an authority regulation channel establishment request to the legal department node (which is the starting scheduling node at this time) after receiving the instruction, obtains the modification authority, and after verification, the R&D department scheduling node (the processing computer of the R&D department) performs the final modification operation on the algorithm description part, updates the record after completion and feeds back the result. If it is detected in the process that the legal department node is not in the "ready state" (such as the node is processing other high-priority tasks), the scheduling unit (central scheduling service station) will automatically transfer the task to the next order candidate node (such as the scheduling node of technical audit) for preliminary review according to the preset priority order, so as to ensure that the modification process can proceed according to the preset rules.

[0117] It should be noted that in the actual office scenario, the "ready state" means that the department processing computer has successfully accessed the central scheduling service network, the document editing function component is running normally, and the specified modification area access authorization of the current version of the application document has been obtained. If the scheduling node instance does not meet any of the above conditions (such as network interruption or authority change), the state will be automatically downgraded to the "ready state", and the self-checking process of step S305 needs to be performed again to restore the ready state.

[0118] Step S700: When the scheduling unit reads the feature information change, a regulation threshold value of a manually set value is set at the corresponding scheduling node.

[0119] It is worth noting that in the actual operation process, the construction method of the regulation threshold value includes steps S701-S703.

[0120] Specifically, step S701: the scheduling unit configures a preset life parameter for each scheduling node according to the data source type, and the life parameter includes the maximum allowed synchronous update times, the scheduling frequency limit, and the resource use period;

[0121] Step S702: The actual life of the scheduling node is updated synchronously according to the dynamic accumulation of the computing power data in the transmission and scheduling process, resource access, and task scheduling behavior;

[0122] Step S703: When the real-time life of the scheduling node reaches or exceeds the preset threshold, the scheduling unit marks the node exceeding the preset threshold as “termination state” and terminates the connection of the scheduling unit and the authority control channel.

[0123] It should be noted that steps S701-S703 are applied to actual office scenarios, for example, the scheduling node (processing computer) of the research and development department configures the preset life parameter when editing the core algorithm in the computing power data (application document), the maximum allowed synchronous update times is 200 times (considering high-frequency technical iteration), the scheduling frequency limit is no more than 5 key operations per minute (to prevent resource overload), and the resource use period is 8 hours of continuous work (to match the regular working hours). When the scheduling node (processing computer) of the research and development department frequently triggers document synchronous update (real-time saving of debugging logs) in the algorithm debugging process, and the cumulative update times reaches the upper limit of 200 times, the scheduling unit (central scheduling service station) will detect that the actual life of the scheduling node (processing computer) of the research and development department is exhausted in real time, immediately mark it as “termination state”, and forcibly disconnect the scheduling node (processing computer) of the research and development department from the scheduling unit (central scheduling service station) and all authority control channels (such as cooperative channels with the test department and the document management department). The connection is required to be restored after the scheduling node (processing computer) of the research and development department is re-registered or replaced.

[0124] Step S800: When the adjustment threshold is triggered, the authority sequence of the authority control channel is reinitialized.

[0125] It should be noted that step S800 is applied to an actual office scenario, for example, when the scheduling node (processing computer) of the R&D department is marked as "terminated state" and disconnected by the scheduling unit (central scheduling service station) due to the exhaustion of the cumulative update times, the scheduling unit (central scheduling service station) immediately detects that the adjustment threshold is triggered, and the authority sequence of all active authority regulation channels is recalculated and initialized according to the preset department weight values (such as the R&D department weight is 0.75, the legal department weight is 0.85, and the management layer node weight is 1.0). During this process, the channels that originally rely on the R&D department node as the starting or branch scheduling node (such as the R&D-test collaboration channel) will be temporarily frozen, and the scheduling unit (central scheduling service station) will preferentially rebuild the channel connection according to the new authority sequence (such as management > legal > test > R&D) for high-weight department nodes. After the R&D department node completes re-registration and recovers to "ready state" through self-checking, its scheduling node identifier will be re-inserted at the end of the authority regulation channel candidate queue, waiting for subsequent participation in task scheduling according to the new sequence.

[0126] As a preferred embodiment, reference is made to Figure 4 It can be seen that the application also proposes a multi-source computing power data integration and intelligent scheduling system, uses a multi-source computing power data integration and intelligent scheduling method, and includes:

[0127] The multi-source data acquisition module is used for segmenting and collecting the multi-source computing power data set according to the data sources, optimizing the data collection efficiency through an intelligent segmentation mechanism, and ensuring the independence and integrity of different source data.

[0128] The resource feature description module is used for collecting resource parameters, scheduling node running state information and historical scheduling records associated with a single computing power data, constructing a resource feature description file, and synchronously updating the feature information in the resource feature description file, including real-time monitoring of resource changes and maintaining file consistency.

[0129] The scheduling unit management module is used for encapsulating a single computing power data and a corresponding resource feature description file into a scheduling unit, generating a scheduling node based on the feature information, managing the generation, self-checking and state updating of the scheduling node, and realizing the life cycle management and fault recovery of the node.

[0130] The authority regulation channel module constructs an authority regulation channel between multiple scheduling nodes through a node collaboration protocol, realizes the transmission of different computing power data contents, and includes an encryption transmission mechanism and an identity authentication process.

[0131] The priority scheduling control module arranges the priority order of the authority regulation channel according to the data sources, executes the scheduling operation of the computing power data content according to the priority order, and dynamically adjusts the task queue to optimize the resource utilization rate.

[0132] A feature synchronization updating module is configured to monitor changes in the resource feature description file, control synchronization updating, verification and abnormality processing of feature information between scheduling nodes, and ensure data consistency and error isolation between nodes.

[0133] A regulation threshold management module is configured to calculate a regulation threshold based on a scheduling frequency, a fluctuation amplitude of computing power resources and a node response delay, and re-initialize a permission sequence of the permission control channel when the threshold is triggered, so as to realize adaptive control and system stability maintenance.

[0134] A global scheduling node directory module is configured to store and manage node identifiers, states and related information of all scheduling nodes, realize unified indexing and scheduling node management, and provide fast query and state monitoring functions.

[0135] As a preferred embodiment, it is to be noted that in the embodiment, the modules are connected in a structural manner through ordered data channels and control interfaces. The multi-source data acquisition module is directly connected to the resource feature description module, and the acquired data is transmitted to the resource feature description module in a standardized data format to generate a feature file. The resource feature description module is connected to the scheduling unit management module through a communication interface, and provides feature data to the scheduling unit management module to generate a scheduling unit. The scheduling unit management module and the node generation and management module are bidirectionally connected, and are configured to trigger creation of the scheduling node and state management of the scheduling node, including node registration and heartbeat detection. After the scheduling node is generated, the scheduling node is connected to the permission control channel module through a unified interface, a communication link between the scheduling nodes is constructed, low-delay data transmission is supported, the priority scheduling control module is connected to the scheduling unit management module, the node task execution sequence is controlled according to a scheduling instruction, dynamic allocation of task priorities is realized, the feature synchronization updating module is connected to the resource feature description module and the permission control channel module, and is configured to monitor changes and coordinate node synchronization, and ensure real-time updating through an event-driven mechanism. The regulation threshold management module is connected to the scheduling unit management module, and is configured to dynamically control reordering of the permission channel according to a task condition, and automatically execute a reconfiguration process when a threshold is triggered.

[0136] Although the embodiments of the present application have been shown and described, it can be understood by those skilled in the art that various changes, modifications, replacements and variations can be made to the embodiments without departing from the principles and spirits of the present application, and the scope of the present application is defined by the appended claims and their equivalents.

Claims

1. A method for multi-source computing power data integration and intelligent scheduling, characterized in that, The application relates to a method for constructing a multi-source computing power data set and a scheduling node generation method. The method comprises the following steps: Segmented collection of a multi-source computing power data set according to data sources; Construction of a resource feature description file associated with single computing power data in the multi-source computing power data set, wherein when the content of the single computing power data changes, the feature information in the resource feature description file is synchronously updated; Encapsulation of the single computing power data and the resource feature description file into a scheduling unit, wherein the scheduling unit reads the feature information to generate a scheduling node; Construction of an authority control channel between multiple scheduling nodes through a node cooperation protocol, wherein the contents corresponding to different computing power data are sequentially transmitted through the authority control channel; Arrangement of a priority order of the authority control channel according to data sources; Content scheduling of single computing power data is performed by a corresponding scheduling node, and the corresponding scheduling node realizes content modification through bottom-up traversal of the scheduling node according to the priority order; When the scheduling unit reads the changed feature information, a manually set adjustment threshold value is set at the corresponding scheduling node; When the adjustment threshold value is triggered, the authority order of the authority control channel is reinitialized; The generation method of the scheduling node comprises the following steps: According to data sources, a preset node generation template is matched for each computing power data in the multi-source computing power data set; According to the node generation template, a bottom-layer resource allocation interface is called to generate a corresponding scheduling node instance; For each created scheduling node instance, a unique node identifier is allocated, and the scheduling node instance is registered in a global scheduling node directory for unified indexing and management; For each newly registered scheduling node, its running state parameters are initialized, and its state is marked as "to be activated"; When the scheduling unit first loads task feature information to the node, the self-checking process of the scheduling node is automatically triggered, and the self-checking content comprises integrity verification of a function module of the scheduling node and access authority verification of target computing power resources; 2. The multi-source computing power data integration and intelligent scheduling method of claim 1, wherein: If the self-checking of the scheduling node is passed, the state of the scheduling node is updated to "ready state", and the scheduling node is added to a candidate queue of the authority control channel. The construction method of the resource feature description file comprises the following steps: Resource parameters associated with single computing power data are collected; Running state information and historical scheduling records of the scheduling node are obtained; The resource parameters, the running state information and the historical scheduling records are encapsulated as structured data; 3. The multi-source computing power data integration and intelligent scheduling method of claim 2, wherein: For each structured data, a corresponding identifier is set to form a resource feature description file. The synchronous updating method of the feature information in the resource feature description file comprises the following steps: When the scheduling unit monitors that the feature information changes, an adjustment instruction is sent to the corresponding scheduling node, and the connection of the authority control channel between the corresponding scheduling node and the rest of the scheduling nodes is interrupted; After the feature information changes end, the corresponding scheduling node reconnects the authority control channel, and the updated resource feature description file is propagated to the rest of the scheduling nodes; The rest of the nodes perform verification operations on the received updated feature information, confirm the consistency of content changes by comparing the feature attributes of the resource feature description files before and after the changes, and if the verification is passed, the scheduling node marks the updated feature information as a valid state identifier, and triggers the scheduling unit to reload the feature description file. If the verification fails, the node generates an exception diagnosis report, performs version rollback to the feature state before the change, and reinitiates the feature synchronization update process.

4. The multi-source computing power data integration and intelligent scheduling method of claim 1, wherein: The content of the node coordination protocol includes: The starting scheduling node sends a verification code to the branch scheduling node; The branch scheduling node performs identity verification after receiving the verification code; After verification, the branch scheduling node initiates a connection request to the starting scheduling node; After receiving the connection request, the starting scheduling node establishes an authority control channel with the branch scheduling node.

5. The multi-source computing power data integration and intelligent scheduling method of claim 1, wherein: The method for content scheduling of a single computing power data includes: According to the priority order, obtain the node identifier of the target computing power data corresponding to the current scheduling node; Based on the node identifier, locate the corresponding scheduling node instance in the global scheduling node directory; Determine whether the state of the scheduling node instance is "ready state"; If the state is "ready state", send a scheduling instruction to the scheduling node instance according to the priority order; After receiving the instruction, the scheduling node instance performs access, transmission or modification operation on the target computing power data; When the scheduling operation involves cross-node data interaction, the current scheduling node acts as the starting scheduling node or the branch scheduling node, and initiates or responds to the establishment request of the authority control channel through the node coordination protocol; After the scheduling operation is completed, the scheduling node instance updates its historical scheduling record and feeds back the operation state and result to the scheduling unit; If the state of the scheduling node instance is not "ready state", automatically switch to the next candidate scheduling node according to the priority order for task allocation.

6. The multi-source computing power data integration and intelligent scheduling method of claim 1, wherein: The method for constructing the threshold value includes: The scheduling unit configures a preset life parameter for each scheduling node according to the data source type, and the life parameter includes the maximum allowed synchronization update times, the scheduling frequency limit and the resource usage period; The real-time life of the scheduling node is dynamically accumulated according to the synchronization update, resource access and task scheduling behavior of the computing power data in the transmission and scheduling process; When the real-time life of the scheduling node reaches or exceeds the preset threshold value, the scheduling unit marks the node exceeding the preset threshold value as "termination state" and terminates the connection between the scheduling unit and the authority control channel.

7. A multi-source computing power data integration and intelligent scheduling system, which runs any one of the multi-source computing power data integration and intelligent scheduling methods of claims 1-6. It includes: A multi-source data acquisition module for segmenting and collecting multi-source computing power data sets according to data sources; A resource feature description module for collecting resource parameters, scheduling node running state information and historical scheduling records associated with a single computing power data, constructing a resource feature description file, and synchronously updating the feature information in the resource feature description file; A scheduling unit management module for encapsulating a single computing power data and a corresponding resource feature description file into a scheduling unit, generating a scheduling node based on the feature information, and managing the generation, self-checking and state updating of the scheduling node; An authority control channel module for building an authority control channel between multiple scheduling nodes through a node coordination protocol to realize the transmission of different computing power data contents; A priority scheduling control module for arranging the priority order of the authority control channel according to the data sources, and performing scheduling operation of the computing power data content according to the priority order; A feature synchronization update module for monitoring changes in the resource feature description file, and controlling the synchronization update, verification and exception handling of feature information between scheduling nodes; The adjustment threshold management module calculates an adjustment threshold based on the scheduling frequency, the fluctuation amplitude of computing power resources, and the node response delay, and reinitializes the permission sequence of the permission control channel when the threshold is triggered. The global scheduling node directory module is used for storing and managing the node identifiers, states, and related information of all scheduling nodes, and realizes unified indexing and scheduling node management. 8.The multi-source computing power data integration and intelligent scheduling system of claim 7, wherein: The modules are structurally connected through ordered data channels and control interfaces. The multi-source data acquisition module is directly connected with the resource feature description module, transmits the collected data to the resource feature description module to generate a feature file, and the resource feature description module is connected with the scheduling unit management module through a communication interface, provides feature data to the scheduling unit management module to generate a scheduling unit. The scheduling unit management module and the scheduling node generation and management module are bidirectionally connected, used for triggering the creation of the scheduling node and the management of the state of the scheduling node. After the scheduling node is generated, it is connected with the permission control channel module through a unified interface, builds a communication link between the scheduling nodes, and the priority scheduling control module is connected with the scheduling unit management module, controls the node task execution sequence according to the scheduling instruction. The feature synchronization update module is connected with the resource feature description module and the permission control channel module, used for monitoring changes and coordinating node synchronization. The adjustment threshold management module is connected with the scheduling unit management module, used for dynamically controlling the permission channel rearrangement processing according to the task condition.

Images