A department data resource scheduling management method and system

Abstract

The application relates to the technical field of data processing and management, and discloses a department data resource scheduling management method and system. The method comprises the following steps: a data node actively broadcasts a data credit beacon containing endogenous credit points and an authoritative update timestamp; a data demander terminal preliminarily sorts the received beacon, combines the result of real-time service health detection on the authoritative candidate data source, finally determines the unique authoritative data source, and initiates a data request. Through the establishment of a dynamic credit competition and service capability checking mechanism, the application changes the administrative examination and approval mode in the traditional data scheduling into a cooperative mode in which each data department actively maintains data quality and service stability so as to make its data be preferentially adopted, thereby stimulating the internal driving force of data sharing.

Description

TECHNICAL FIELD

[0001] The application relates to a department data resource scheduling management method and belongs to the technical field of data processing and management. BACKGROUND

[0002] In current data-intensive management and business activities, in order to realize the integration and utilization of internal data of an organization, two data resource scheduling modes are usually adopted. One is to establish a data pipeline to collect data scattered in various business departments to a unified data center through an extraction, transformation and loading process. The other is to provide data to other departments in the form of an application program interface by a data holding department after authorization. These two modes provide a basic framework for realizing centralized control and orderly access of data.

[0003] When the timeliness and flexibility of data are required by a business scenario, for example, the market department needs to obtain the latest transaction data of the sales department to quickly respond to market dynamics, the above scheduling mode will produce a delay in the actual operation process. This is because the basis of the operation mode is a fixed process of first application, then authorization and then access. This requires communication and coordination between the data demand side and the data holding side, and the execution of the corresponding internal approval procedures. These interaction processes themselves need time. A direct response idea is to increase the frequency of data extraction or expand the bandwidth of the data interface. However, this way increases the processing load and maintenance complexity of the center system and does not change the essence of scheduling decision relying on central authorization. Another idea is to relax the permission limit of data access, but this directly conflicts with the data security protection responsibility borne by the data holding department. Therefore, under the existing technical framework, there is a real operational tension between the organization's demand for flexible data flow and the department's responsibility for data security control.

[0004] Specifically, the existing technology has the following conditions: 1. The core of the existing scheduling mechanism is to verify the identity of the requester and plan the access path. This mechanism does not contain components for measuring the quality of shared data by the data provider, nor does it have corresponding links to feedback the value of the sharing behavior to the provider; 2. Under this mechanism, providing data is built as a one-way task of the data holding department, and the maintenance cost and the value of data contribution cannot be reflected in the system, thereby affecting the active supply of high-quality and high-timeliness data. Therefore, how to establish a data resource scheduling management method that no longer relies on the fixed application-approval process, but can dynamically judge the authority of data acquisition according to the verifiable information published by each data source, and then form a collaborative environment to promote the active sharing of data, is the technical problem to be solved by the application. SUMMARY

[0005] The application provides a department data resource scheduling management method, which mainly aims to solve the problem of long cross-department data cooperation process and insufficient timeliness caused by the fact that the existing data scheduling mode cannot effectively promote the active sharing of data in mechanism due to the fixed application-approval process.

[0006] To achieve the above object, the application provides a department data resource scheduling management method, which comprises the following steps:

[0007] Step a, after each of the multiple data nodes holding a copy of the same data item completes an authoritative update, generating and broadcasting a data reputation beacon to the network, the data reputation beacon containing the unique identification of the data item, the timestamp of the authoritative update and an endogenous reputation score determined according to preset ownership level information;

[0008] Step b, the data demand side terminal receives the data reputation beacon and preliminarily sorts the multiple data nodes according to the endogenous reputation score and the timestamp of the authoritative update to determine one or more authoritative candidate data sources;

[0009] Step c, the data demand side terminal sends a query intention signaling to the one or more authoritative candidate data sources and receives a service health degree score returned by the authoritative candidate data sources according to their own running load states;

[0010] Step d, the data demand side terminal uniquely determines a most authoritative data source from the authoritative candidate data sources based on the preliminary sorting and in combination with the service health degree score;

[0011] Step e, the data demand side terminal initiates a data request to the most authoritative data source to obtain the data item.

[0012] Preferably, the most authoritative data source is uniquely determined in step d, specifically: the authoritative candidate data sources are sorted in descending order according to the service health degree score, and the first authoritative candidate data source after the sorting is determined as the most authoritative data source.

[0013] Preferably, before step b, the data demand side terminal also comprises: statistically calculating the time interval of the arrival of the historical data reputation beacon from each data node to calculate a dynamic reliability weight representing the stability of the beacon broadcasting rhythm of the data node; and in step b, the preliminary sorting is based on the weighted reputation score obtained by multiplying the endogenous reputation score and the dynamic reliability weight, and the calculation of the weighted reputation score is: wherein, is the weighted reputation score, is the dynamic reliability weight, is the endogenous reputation score.

[0014] Preferably, further comprising: setting a trigger for the service rejection operation in the downstream service application; generating and broadcasting a credit instruction containing the rejected data source identification when the trigger is activated by the service rejection operation; and temporarily attenuating the endogenous credit points of the rejected data source according to the received credit instruction when the data demander terminal performs the preliminary sorting of step b.

[0015] Preferably, in step c, the query intention signaling is a pre-request detection signal containing the unique identification of the data item; and the service health score is calculated by a health agent of the authoritative candidate data source according to at least one performance indicator of the real-time monitored central processor load, memory usage and current database connection number of the agent, through a preset weighting formula.

[0016] Preferably, further comprising: in step a, the at least one data demander terminal collects the data reputation beacons according to the service domain field contained in the data reputation beacons when receiving the data reputation beacons; in step b, the data demander terminal statistics the broadcast frequency of the data reputation beacons of each service domain in time windows, and compares the broadcast frequency with the historical frequency baseline of the service domain; in step c, when the broadcast frequency exceeds the predetermined range of the historical frequency baseline, a service trend early warning signal is generated and sent.

[0017] Preferably, before step b, further comprising an input purification step: the data demander terminal performs frequency domain feature analysis on the timestamp value stream of the authoritative update in the data reputation beacons received from different data nodes; when it is identified that the power spectral density of the timestamp value stream of a certain data node presents an abnormality in the high frequency region, the timestamp values of the data node are normalized before the preliminary sorting of step b is performed.

[0018] Preferably, further comprising: setting an emergency mode trigger for listening to the emergency broadcast channel at the data demander terminal; when the trigger receives an emergency signaling containing an event keyword and geographic fence information, suspending the execution of steps b to e and switching to an emergency arbitration mode; in the emergency arbitration mode, selecting the data node with the closest geographic position to the center of the geographic fence as the final data source according to the geographic fence information in the emergency signaling and the real-time geographic position of each data node; and establishing a beacon cache by the data demander terminal for temporarily storing the data reputation beacons received in the near future; when it is necessary to dispatch the same data item again, preferentially performing steps b to e in the beacon cache, and only when the data reputation beacons in the beacon cache do not meet the timeliness requirement, listening to the network broadcast again.

[0019] Preferably, the endogenous credit score is temporarily attenuated, specifically, the data demander terminal locally maintains a temporary credit score for each data source; when a credit instruction for a certain data source is received, the temporary credit score of the data source is reduced under the constraint of a preset attenuation coefficient and an upper limit of attenuation; the temporary credit score is automatically restored over time according to a preset recovery rule.

[0020] A departmental data resource scheduling management system, comprising:

[0021] A beacon receiving module configured to receive a data credit beacon broadcast by multiple data nodes holding a copy of the same data item, the data credit beacon containing a unique identifier of the data item, a timestamp of a completion of an authoritative update by the data node, and an endogenous credit score determined according to preset ownership level information;

[0022] A preliminary arbitration module configured to preliminarily sort the multiple data nodes according to the endogenous credit score and the timestamp of the authoritative update, to determine one or more authoritative candidate data sources;

[0023] A health degree detection module configured to send a query intention signaling to the one or more authoritative candidate data sources, and receive a service health degree score returned by the authoritative candidate data sources according to their own running load states;

[0024] A final arbitration module configured to uniquely determine a most authoritative data source from the authoritative candidate data sources based on the result of the preliminary sorting and in combination with the service health degree score;

[0025] A data acquisition module configured to initiate a data request to the most authoritative data source to acquire the data item.

[0026] Compared with the prior art, the present application has the following beneficial effects:

[0027] 1. By constructing a data collaboration mode, the fixed mode of application-approval in traditional data scheduling is changed; in the present application, the data nodes actively broadcast credit beacons containing their ownership level and update time, and the demander determines the authoritative source locally according to the public beacons; this mode changes the passive administrative task of data sharing into a competitive behavior of actively maintaining data quality and update timeliness by each data holder in order to make the data be adopted preferentially, thereby stimulating the internal driving force of cross-departmental data collaboration.

[0028] 2. By combining the static ownership level of the data source with the dynamic update time two dimensions, and introducing the detection of the real-time running state of the service node on this basis, a multi-level dynamic source seeking decision mechanism is established, which not only solves the problem of how to select the most authoritative party in the management sense among multiple seemingly credible data sources, but also avoids the risk of initiating invalid requests to a authoritative but currently insufficient service node through the health degree detection of the pre-request, so that the whole process of data acquisition is guaranteed in the correctness of decision and the reliability of execution.

[0029] 3. The data scheduling management method constructed by the application has the ability of self-regulation and continuous optimization. By introducing the consideration of the stability of the beacon arrival rhythm, the arbitration logic can perceive and adapt to the changes of the network environment, preferentially relying on the more reliable data source of the transmission channel. At the same time, by capturing the business veto behavior of the downstream application and generating a reputation instruction, a distributed negative feedback loop is established, so that the real use experience of data quality can correct the reputation evaluation of the data source in reverse, thereby continuously suppressing the spread of low-quality data without human intervention, and maintaining the health of the entire data ecosystem.

[0030] 4. The method of the application shows adaptability and resilience when dealing with irregular and unexpected situations. By setting an emergency mode trigger and coupling an emergency broadcast channel, the method can suspend the regular arbitration logic based on reputation in the extreme scenario of damaged regular communication network, and automatically switch to an emergency decision mode that prioritizes geographical location proximity, ensuring that the data with the most on-site significance can be obtained in time, providing data support for emergency command. BRIEF DESCRIPTION OF DRAWINGS

[0031] Fig. 1 Fig. 1 is a schematic diagram of the running process of the department data resource scheduling management method of the application;

[0032] Fig. 2 Fig. 4 is a performance comparison test result graph of the method of the application and the traditional method under different working conditions;

[0033] Fig. 3 Fig. 5 is a scheduling decision and state switching logic flowchart of the data demand side terminal of the application. DETAILED DESCRIPTION

[0034] In order to make the purpose, technical scheme and advantages of the application clearer, the technical scheme of the application will be described in detail below in combination with embodiments. Obviously, the described embodiments are part of the embodiments of the application, rather than all the embodiments. Based on the embodiments in the application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the application.

[0035] The department data resource scheduling management method and system disclosed by the application aims to be applied in a large organization, when multiple business departments all hold a copy of the same data item, and a data demand party needs to identify and obtain the unique authoritative version in the situation; the core operation process of the method mainly includes the beacon generation and broadcast step executed by the data node, and the dynamic arbitration and source finding step and the data acquisition step executed by the data demand party terminal, wherein the dynamic arbitration and source finding step is configured as a multi-stage confirmation process including preliminary sorting and health degree final selection; in a specific application scenario, for example, in a large retail group, the marketing department needs to obtain the recent high-value customer consumption details generated by the sales department in real time to formulate marketing strategies, and the finance department also holds a copy of the data processed with T+1 delay for cost accounting; in this situation, the existing calling method based on static authorization cannot meet the requirements of timeliness, authority and availability of data acquisition, therefore, the method of the application provides a dynamic discovery mechanism based on data reputation beacon; in the beacon generation step, the business systems of the sales department and the finance department holding the copy of the recent high-value customer consumption details data item are configured as data nodes after completing an update recognized as authoritative by the business logic, and each generates a data reputation beacon containing only metadata; the data structure of the data reputation beacon is determined to contain at least three key fields: the unique identifier of the data item, used to indicate the data content corresponding to the beacon; the timestamp of the authoritative update, which records not the arbitrary database write time, but the specific moment when the data owner confirms that the update content is complete and accurate, and an endogenous reputation score, which is preset by the data governance committee at system initialization according to the ownership level of each department for a specific data item, for example, the endogenous reputation score of the sales department for the recent high-value customer consumption details is 90, and the endogenous reputation score of the finance department is 70; by quantifying the management level of the data source into a calculable weight, the first judgment basis for subsequent arbitration is provided.

[0036] Subsequently, in the beacon broadcast step, the data nodes of the sales department and the finance department are configured to broadcast the respective generated data credit beacons to a predetermined subnet using the user datagram protocol. This broadcast behavior can be achieved by mounting lightweight scripts on the update triggers of the database, thereby announcing the respective data states to the network without intruding on the core logic of the existing business system. This interaction mode from point-to-point request to multi-to-multi broadcast constitutes the basis of the entire collaboration process. Furthermore, in the preliminary sorting stage of the dynamic arbitration and source-seeking step performed on the terminals of the data demand side, i.e., the marketing department, to address the problem of network environment fluctuations leading to delays in the arrival of authoritative beacons or loss, thereby affecting the fairness of arbitration, the method introduces a dynamic reliability weighting mechanism based on beacon arrival rhythm. The terminals of the marketing department are configured to continuously count the time intervals of the arrival of historical data credit beacons from each data node such as the sales department and the finance department, and use the exponential weighted moving average algorithm to calculate the jitter value of the beacon arrival interval for each data source , and generate a dynamic reliability weight accordingly. The certainty procedure of the dynamic reliability weight is set as an inverse proportional function: , where is a sensitivity coefficient that can be determined through offline calibration experiments. This function ensures that the more unstable the data source's beacon arrival rhythm, the lower its reliability weight . Accordingly, when performing preliminary sorting, the terminals of the marketing department do not directly use static endogenous credit scores, but use a weighted credit score obtained by multiplying the dynamic reliability weight and the endogenous credit score . The calculation of the weighted credit score follows the deterministic formula: , where is the endogenous credit score. The terminal sorts the received beacons in descending order according to the , and when In the same time, according to the time stamp of the authority update in descending order, so, by combining the dynamic weight reflecting the reliability of the channel with the static score reflecting the authority of the data source, the arbitration logic can compensate for the uncertainty of the network transmission layer, so as to select one or more authoritative candidate data sources with the highest comprehensive reputation in the current network environment; At the same time, in order to deal with the challenge that a high-reputation data source may start to continuously output legal but incorrect data due to system failure or human error, causing trust pollution, the method also provides a reputation score dynamic decay mechanism based on downstream application implicit feedback; In the BI tool used by market analysts and other downstream business applications, a trigger is preset for the rejection, correction and other business veto operations on the interface; When the analyst executes such veto operation due to the discovery of data quality problems, the trigger is activated, and a reputation instruction containing the identification of the rejected data source is generated and broadcasted; All data demander terminals in the network, including the market department itself, will temporarily and dynamically decay the endogenous reputation score of the rejected data source after listening to the reputation instruction, and the certainty procedure of the decay is: maintain a temporary reputation score for each data source locally, when receiving a reputation instruction for a certain data source, reduce the temporary reputation score of the data source according to a preset decay coefficient and decay upper limit, at the same time, the temporary reputation score is configured to automatically recover over time according to a preset recovery rule; This kind of distributed negative feedback loop makes the real user experience of data quality be able to correct the reputation evaluation of data source in reverse.

[0037] After the initial sorting and the determination of the sales department as the first authoritative candidate data source, the system faces a new problem: a creditworthy data source whose server may be in a sub-healthy state due to high load at the moment, and a direct request may cause delay or failure; To avoid this risk, the market department terminal is configured to send a query intention signal as a pre-request probe signal to all authoritative candidate data sources such as the sales department; Correspondingly, a parallel lightweight health agent is deployed on the data node of the sales department, which monitors at least one performance indicator of the central processor load, memory usage, and current database connection number of the node in real time, and calculates a service health score in real time according to a preset weighted formula, and returns the score directly to the market department terminal after bypassing the business logic upon receiving the query intention signal; Finally, the market department terminal determines the most authoritative data source based on the preliminary sorting result formed in the above steps and the service health score received; The certainty logic is: sort the authoritative candidate data sources in descending order according to the service health score, and determine the first sorted authoritative candidate data source as the most authoritative data source; This mechanism inserts a feasibility check link between decision and execution, ensuring that the final data request is initiated to a node that is not only authoritative but also has the ability to provide services at the moment; The market department terminal initiates a standard data pull request to the uniquely determined most authoritative data source such as the sales department to obtain the recent high-value customer consumption details, thereby completing the entire scheduling process; In addition, to improve system efficiency, the market department terminal is also configured to establish a beacon cache for temporarily storing recent received data credit beacons, and when the same data item needs to be scheduled again, the above arbitration and source finding logic is executed in the cache first, and only when the cache beacon does not meet the preset timeliness requirement, the network broadcast is listened to again.

[0038] To further expand the management and supervision capabilities of the method, the system can also be configured to perceive business trends, add an optional business domain field in the beacon structure, and the data demand terminal can collect beacons according to the business domain while receiving the beacons, and in a fixed time window, the beacon broadcast frequency of each business domain is counted, and by comparing the real-time frequency with a dynamically updated historical frequency baseline, when the frequency of a certain business domain deviates significantly from its historical baseline, for example, more than 3 standard deviations, the system determines that a trend anomaly has occurred, and automatically generates and sends a business trend warning signal; To enhance the robustness of the system in a heterogeneous environment, the method also provides an input purification step; At the front end of the demand terminal receiving the beacon, a frequency domain purification gateway is deployed, which performs concomitant frequency domain feature analysis on the timestamp value stream of the authoritative update in the beacon from different data nodes and about the same data item, the identification logic of which is that a value stream with a dimensional anomaly caused by a unit inconsistency such as milliseconds to seconds will have a significantly higher energy ratio in the high-frequency region of the power spectral density; When such anomalies are identified through a deterministic judgment logic, the gateway automatically performs a preset normalization on the field value of the data node before executing arbitration, thereby ensuring the consistency of all key elements of the input arbitration algorithm; Finally, to deal with major emergencies such as earthquakes and floods that cause disruptions in conventional IP networks, the method provides an emergency mode switching mechanism; An emergency mode trigger for listening to the national emergency broadcast channel is set on the data demand terminal, when the trigger receives emergency signaling containing preset event keywords and geofence information, the original regular arbitration logic based on data credibility beacons is suspended, and the emergency arbitration mode is switched; In this mode, the terminal's decision logic is switched to the data node whose physical location is closest to the geofence center as the final data source according to the geofence information in the emergency signaling and the real-time geographic location of each data node. This design, through coupling with national-level infrastructure, enables the technical solution to still complete its core data source finding task under extreme survival conditions.

[0039] Example 1: In the operational environment of a city-level emergency command center responding to a public health emergency, the command platform, as the data demand terminal, is tasked with mapping the geographical locations of newly confirmed cases in real time to support resource allocation and regional lockdown. The objective challenge in this scenario lies in the fact that the data item of the location of newly confirmed cases exists simultaneously on two data nodes: the internal database of the Municipal Health Commission, whose data is reported by medical institutions and reviewed by experts, possessing the highest management authority, but with time delays in the update process; and the frontline patrol system of the Municipal Public Security Bureau, whose data is entered immediately by on-duty personnel, possessing the highest timeliness, but without medical confirmation. Both data nodes broadcast data reputation beacons after completing their respective authority updates, as required. When the emergency command enters a high-intensity operational phase, network communication traffic surges, causing varying jitter and delays in the path from each data node's beacon to the command platform terminal. After listening to the data reputation beacons from the Municipal Health Commission and the Municipal Public Security Bureau, the command platform initiates dynamic authority source arbitration logic. In the initial sorting stage, the command platform uses continuously statistically analyzed beacon arrival interval jitter values... Dynamic reliability weights were calculated for the two data sources: the Municipal Health Commission and the Municipal Public Security Bureau. Given that the core network of the Municipal Health Commission is more congested, its beacon arrival rhythm stability is lower than that of the Municipal Public Security Bureau, thus resulting in a relatively lower value. Subsequently, the command platform... The procedures will reflect the reliability of the channel. Reflecting the authority of data sources The scores are combined to calculate a weighted credit score, despite the Municipal Health Commission's... The value is higher, but... After the correction, its Both the score and the score of the Municipal Public Security Bureau are on the same order of magnitude, and the timestamp of the Municipal Public Security Bureau is updated. Both were judged to be authoritative candidate data sources.

[0040] At this point, the system faced a conflict between the authority of the source and the real-time availability of the service. While the Municipal Health Commission node was authoritative, its server was under high load due to the need to simultaneously respond to a large number of internal queries. To address this, the command platform immediately entered the final health score selection stage. It simultaneously sent a query intent signal to both the Municipal Health Commission and the Municipal Public Security Bureau, two authoritative candidate data sources. The health score proxy of the Municipal Health Commission node, detecting that its own central processing unit load had exceeded the warning threshold, returned a lower service health score. The health score proxy of the Municipal Public Security Bureau node, however, returned a higher score due to normal system load. After receiving the two scores, the command platform's final source selection mechanism was triggered, uniquely identifying the Municipal Public Security Bureau node with the highest service health score as the final authoritative data source, and thus initiating data requests only to it. The command platform's electronic map then received the data from the Municipal Public Security Bureau node and the latest suspected case location data, highlighting them for the command center. The immediate data input for determining the patrol route for the next minute was provided, while data requests to the Municipal Health Commission node were proactively avoided due to its sub-health status, preventing potential delays in command and decision-making caused by request timeouts. The entire data scheduling process transformed the traditional fixed process of application-approval-access into a dynamic discovery process based on reputation competition and service capability verification. This allowed the original choice between prioritizing timeliness and prioritizing authority to be handled under a unified arbitration logic. Within tens of minutes of the completion of this scheduling, when the expert review process of the Municipal Health Commission was completed and its database was updated, a new data reputation beacon with a higher authority update timestamp was broadcast. At the same time, since the peak of server queries had passed, its service health score also returned to normal levels. In the next scheduling cycle of the command platform, the Municipal Health Commission node was therefore selected as the final authoritative data source in the reputation competition, and the map data of the command platform was subsequently updated to the final location confirmed by medicine.

[0041] Example 2: To quantitatively evaluate the effectiveness of the data resource scheduling and management method of the present invention, the following comparative experiment was designed and executed. This experiment aims to measure and compare the performance of the method of the present invention and a traditional data scheduling method on two core performance indicators: end-to-end latency in data acquisition and accuracy in selecting authoritative data sources. The experimental platform was built in a virtualized environment consisting of three servers to simulate an organizational information system containing multiple data departments. Servers one, two, and three were configured as data node A, data node B, and data node C, respectively. All three servers deployed the same business database and stored copies of the same data items. To simulate the differences in data ownership levels among different departments, their intrinsic reputation scores were determined according to the procedures of the specific implementation method. Node A is preset to be non-equal. Set to 90, representing the highest level of management authority, the node B Set to 70, the node C Set to 50; a stand-alone server is configured as a data demand terminal for initiating data requests and executing arbitration logic, network connections between servers are controlled through a network simulator that programmably injects preset network delays and packet jitter, and the server's running load is applied on demand by a load generation tool.

[0042] The test sets up a test group and a control group, the test group fully deploys the scheduling management method of the application, including broadcasting of data reputation beacons, initial sorting based on weighted reputation scores and authority update timestamps, and final source-seeking decisions based on service health scores; the control group simulates an API gateway calling mode based on static priority, which is configured to make data requests according to the fixed logic of prioritizing node A, then node B in case of failure or timeout, and finally node C; the test is performed under four different working conditions, each working condition is repeated 100 times and the average value is taken as the final record, the working conditions include: working condition one, baseline state, i.e. normal load of each node server and good network state; working condition two, high load state, i.e. only the highest authority node A is subjected to an operation load that makes its CPU usage rate reach 95%; working condition three, high network jitter state, i.e. 50ms average delay and 20ms jitter are injected into the network between nodes; working condition four, combined stress state, i.e. high load is applied to node A and high jitter is applied to the entire network.

[0043] After the test starts, under working condition one, both the test group and the control group select node A as the data source, and there is no significant difference in end-to-end delay; when switching to working condition two, the system response modes of the two groups differ, the control group still tries to connect node A first, triggers a 5000ms timeout because node A service is unresponsive, and then turns to node B and obtains data, resulting in an increase in end-to-end delay to 5125ms; while the demand terminal of the test group, although node A is listed as the preferred authority candidate data source in the initial sorting, receives a low service health score returned by node A in the subsequent health detection, and thus its final source-seeking decision logic selects to initiate a request to node B with a higher score, actively avoiding access to the high-load node, and its delay only increases slightly to 120ms; in working condition three and working condition four, the dynamic reliability weight mechanism intervenes, but since there is no obvious advantage or disadvantage in the channel quality of each node, the test group can still make correct source-seeking judgments, and its delay changes with the network condition, while the control group again causes the request to timeout due to its static strategy under combined stress, for specific data, see Table 1.

[0044] Table 1: Performance comparison table of each group under different working conditions.

[0045]

[0046] The test data shows that, under the high load working condition of node A, the low latency of the test group is due to the health detection step it performs before request initiation, which identifies and avoids node A with insufficient service capacity at the time through the service health score, thereby avoiding the request timeout caused by the fixed priority strategy of the control group.

[0047] Embodiment 3: This embodiment combines Figs. 1 to 3 A department data resource scheduling management method and system are described as follows: Fig. 1 As shown in the figure, the process begins with multiple data nodes holding data replicas, such as data node A, data node B to data node n, which broadcast data reputation beacons after completing their own authoritative updates. The data demand terminal receives the beacons broadcast by each node by listening to the network. Before performing preliminary arbitration, the terminal will start dynamic reliability weight calculation in parallel. This step calculates a dynamic reliability weight by counting the stability of the beacon rhythm to correct the endogenous reputation points, thereby compensating for the uncertainty in the network transmission process. At the same time, a distributed negative feedback loop originating from downstream business applications also influences the arbitration process, i.e. when a business operation is rejected due to data quality issues, the system generates and broadcasts a reputation instruction to temporarily decay the reputation points of the rejected data source. The subsequent preliminary arbitration module combines the endogenous reputation, update time and dynamic weight of the node to determine one or more authoritative candidate data sources. On this basis, the system further performs health detection, sends query intent signaling to these candidate sources and receives their returned service health scores. The final arbitration step combines the preliminary sorting results and the service health score to uniquely determine the most authoritative data source. The data acquisition module only initiates a request to this most authoritative data source and acquires the data item, thereby completing a complete scheduling.

[0048] As shown in the figure, Fig. 2As shown in the figure, the horizontal axis represents the test conditions, including four types: baseline, high load, high network jitter, and combined stress. The vertical axis represents the end-to-end latency expressed on a logarithmic scale in milliseconds (ms). The figure contains two curves. The experimental group, represented by solid dots, represents the system using the method of this invention, while the control group, represented by dashed triangles, represents the system using the traditional static priority strategy. The test results clearly show that under the baseline and high network jitter conditions, there is no significant difference in latency between the experimental and control groups. However, under the conditions of high node load and combined stress, the control group, due to its fixed request logic, initiates requests to high-load nodes and waits for timeouts before retrying, causing its end-to-end latency to rise sharply to over 5000 ms. In contrast, the experimental group, through its health detection mechanism, can actively avoid high-load nodes with insufficient service capacity, and its latency only increases slightly, always remaining within 310 ms. This result verifies the beneficial effects of this invention in ensuring the reliability and timeliness of data acquisition execution.

[0049] like Fig. 3 As shown, when a scheduling task is triggered, the system first checks the beacon cache. If the data reputation beacon in the cache is valid, the system directly uses the cached data to perform final arbitration and sourcing. If the cache is invalid, the system enters a state of listening to network broadcasts to wait for and collect new data reputation beacons. After receiving a valid beacon, the system performs preliminary arbitration and sorting, filtering candidate sources by calculating a weighted reputation score. Next, the system detects the health of the candidate sources, sends query intent signaling to them, and obtains service health scores. Then, it enters the final arbitration and sourcing stage, combining all scores to uniquely determine the final data source. After determining the unique authoritative data source, the system initiates a data acquisition request to it to successfully acquire the data item. In addition, this process also embeds an emergency arbitration mode. When the system receives an emergency signaling, it suspends the regular logic and determines the data source based on emergency rules such as geographical location, directly initiating data acquisition to the emergency data source.

[0050] Example 4: In the first deployment of the departmental data resource scheduling and management method of this invention in a large financial institution, the system administrator needs to set a set of operating parameters for three key mechanisms within the system that match its specific network environment and business risk strategy. These three mechanisms are a dynamic trust weight mechanism for adjusting the impact of channel reliability, a reputation instruction mechanism for suppressing the spread of low-quality data, and an input purification mechanism for addressing metadata heterogeneity. To ensure that the parameter settings have reproducible data basis, the administrator performed a systematic offline calibration and configuration process; regarding the dynamic trust weight calculation formula... Sensitivity coefficient in The administrator collected beacon data broadcast over a continuous 24-hour period from all servers that would serve as data nodes in the organization's production network, and calculated the baseline mean of the beacon arrival interval jitter for each node during this period. Subsequently, the baseline network was reproduced in a test environment using a network simulator, with an additional jitter increment known to represent a moderate degree of network degradation injected. The goal of this increment is to adjust the dynamic trust weight of a node. Reduced to a preset tolerance threshold, here set to 0.9, at which point the coefficient... The value is obtained by solving the equation This approach anchors an abstract sensitivity coefficient to the objective physical characteristics of the financial institution's own network environment.

[0051] The decay parameters in the reputation instruction mechanism are configured in conjunction with the institution's risk management strategy. The decay coefficient of temporary reputation points is set in stages based on the criticality of the business scenario served by the data item. For data items that directly affect core transaction decisions, the decay coefficient is set to a higher value, while for auxiliary analysis data items, the decay coefficient is set to a lower value. The decay limit is uniformly set at 50% of the intrinsic reputation points to prevent continuous reputation attacks targeting specific nodes from causing a complete service interruption. The recovery rule for reputation points is determined to be an exponential decay process. That is, in the absence of new reputation instructions, the reduction value applied to the temporary reputation points is automatically halved every 24 hours until it reaches zero. This procedure achieves a balance between the immediacy, severity, and recoverability of reputation penalties.

[0052] To determine the power spectral density threshold used in the input cleanup mechanism to identify timestamp anomalies, the administrator extracted 10,000 consecutive, correctly measured timestamp readings from each data node under baseline network conditions, forming a standard sample set. A Fast Fourier Transform was performed on each timestamp value stream in this sample set to obtain its power spectral density, and the ratio of its high-frequency energy to the total energy was calculated. This yielded a statistical distribution of this ratio. Ultimately, the threshold was determined to be the mean of this distribution plus four times the standard deviation. This setting ensures that only beacons that significantly deviate from the characteristics of normal data flow in a statistical sense will be judged as having dimensional anomalies and trigger normalization processing. After all the aforementioned parameters have been calibrated and configured, the department's data resource scheduling and management system has been initialized. All its key adaptive and self-regulating mechanisms have obtained quantitative procedures that match its operating environment and management requirements and have data support. The system is then put into online operation.

[0053] Example 5: In the application of the method of the present invention to national critical infrastructure, such as power grid dispatch centers, to cope with possible extreme communication interruption conditions, the data request terminal is configured with an emergency mode triggering procedure based on communication heartbeat monitoring. This procedure is independent of monitoring external emergency broadcast channels. The terminal sets a global beacon monitoring watchdog timer for all known data nodes, and its time window is set to a preset value greater than the longest normal broadcast interval. If the terminal fails to receive a valid data reputation beacon from any data node within this time window, the system determines that a global communication interruption has occurred. Under this communication interruption determination, the system will automatically suspend all beacon monitoring and data request activities based on conventional IP networks and establish a data acquisition module that communicates point-to-point with the national disaster recovery data center via satellite link. This module then initiates a request for critical dispatch data to the disaster recovery data center according to a preset emergency protocol. This failover security mechanism, which does not rely on external signaling, is used to establish data communication under extreme conditions.

[0054] To enable the system's core parameter to generate an endogenous reputation score The data governance process is standardized. The financial institution's data governance committee uses a multi-factor quantitative evaluation scoring procedure to determine the scores. For each cross-departmental shared data item, each data node holding a copy must be scored through a pre-defined evaluation matrix. The evaluation dimensions of this matrix include: distance from the data source (the number of business process steps from the initial data generation event); legal custody responsibility (whether the department to which the node belongs is designated as the authoritative custodian of the data under laws or industry regulations); and data lifecycle role (the role the node plays throughout the entire lifecycle of data generation, processing, use, and archiving). The scores and weights of each evaluation dimension are predefined in the data governance charter, and each node's final intrinsic reputation score is determined by this framework. The score is the sum of its weighted scores across all evaluation dimensions. This procedure transforms the score setting process into a standardized management process that is both systematic and auditable.

[0055] Example 6: In the application of deploying the scheduling management method of the present application in a national supply chain management platform, in order to ensure the robustness of the system and the environmental adaptability of the decision model, a set of standardized pre-model construction and online fault-tolerant mechanism configuration procedures need to be performed before the system goes online. This procedure aims to solve three specific engineering problems: the quantification of service health assessment model, the optimization of business trend warning threshold, and the online filtering of abnormal beacon data. To construct the service health assessment model, the system administrator first collects the historical performance index data set of the platform's multiple data node servers under different running loads. This data set contains a series of multiple groups, each group consisting of the node's real-time central processor load, memory usage, database current connection number, and a corresponding service quality level determined by actual request response time test calibration. This data set is then used as a training set for a multiple linear regression analysis, and the analysis result outputs an evaluation formula with a deterministic coefficient, such as: Service health score = a * CPU load + b * memory usage + c * database current connection number + d * service quality level where the constant and the weight coefficients of each performance index , , are determined by the results of the regression analysis. This process converts an abstract evaluation model into a quantifiable function based on the historical data of the target environment.

[0056] To configure the trigger threshold of the business trend warning mechanism, the administrator retrieves the beacon broadcast frequency historical data of key business domains such as trunk logistics in the past year, and the business analysts mark the frequency abnormal periods caused by real supply chain events to form a time series with true positive labels. The administrator then changes the judgment parameters such as standard deviation multiple and continuous time window number to perform backtesting on the historical data, and calculates the true positive rate and false positive rate for each parameter combination. By drawing and analyzing the receiver operating characteristic curve, the administrator selects the parameter combination corresponding to the point closest to the upper left corner of the curve as the optimal configuration of the warning threshold for this business domain. This procedure balances the sensitivity and specificity of the warning based on statistical optimization of historical business data.

[0057] In order to deal with the data beacon with format damage or content abnormality caused by other system failure or interference in the network, a protocol compliance filter is deployed at the front end of the beacon receiving module of the data demand terminal. The filter performs a structured check and a value range check on each received data packet before sending it to the subsequent arbitration logic. The structured check checks whether the number of fields and the data type of the data packet conform to the predetermined specification, and the value range check judges whether the timestamp, endogenous reputation points and other numerical values fall within the logical valid interval. Any data packet that fails to pass the two checks will be directly discarded and recorded in the exception log. This mechanism provides a pre-compliance check link for the data input end of the system.

[0058] It is obvious for those skilled in the art that the present application is not limited to the details of the above exemplary embodiments, and can be implemented in other specific forms without departing from the spirit or essential characteristics of the present application.

[0059] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application and are not limiting. Although the present application has been described in detail with reference to the preferred embodiments, it should be understood by those skilled in the art that the technical solutions of the present application can be modified or replaced equivalently without departing from the spirit and scope of the technical solutions of the present application.

Claims

1. A department data resource scheduling management method, characterized in that, Includes the following steps: Step a: After each of the multiple data nodes holding a copy of the same data item completes a business logic assertion update, they generate and broadcast a data reputation beacon to the network. The data reputation beacon contains a unique identifier of the data item, a timestamp of the business logic assertion update, and an endogenous reputation score determined based on pre-defined ownership level information. Step b: The data requester terminal receives the data reputation beacon and performs preliminary sorting of multiple data nodes based on the endogenous reputation score and the timestamp of the business logic assertion update to determine one or more candidate data sources for business logic assertion. Step c: The data requester terminal sends a query intent signaling to one or more business logic-identified candidate data sources and receives a service health score returned by the business logic-identified candidate data sources based on their own operating load status. Step d: Based on the preliminary sorting and combined with the service health score, the data request terminal uniquely determines a final business logic identification data source from the candidate data sources. Step e: The data requester terminal only initiates data requests to the data source identified by the final business logic in order to obtain data items; The method further includes: setting an emergency mode trigger on the data requester terminal for monitoring the emergency broadcast channel; when the trigger receives an emergency signaling message containing event keywords and geofence information, suspending the execution of steps b to e and switching to emergency arbitration mode; in emergency arbitration mode, based on the geofence information in the emergency signaling message and the real-time geographical location of each data node, selecting the data node whose geographical location is closest to the geofence center as the final data source, and establishing a beacon cache by the data requester terminal for temporarily storing recently received data reputation beacons; when the same data item needs to be scheduled again, steps b to e are executed first in the beacon cache, and network broadcasting is re-monitored only when the data reputation beacons in the beacon cache do not meet the timeliness requirements.

2. The department data resource scheduling management method of claim 1, wherein, In step d, the final business logic identification data source is uniquely determined by sorting the candidate data sources for business logic identification in descending order based on the service health score, and determining the first candidate data source for business logic identification after sorting as the final business logic identification data source.

3. The departmental data resource scheduling and management method according to claim 1, characterized in that, Before step b, further comprising: the data demander terminal counting the time interval of the arrival of the historical data credit beacon from each data node to calculate a dynamic credit weight representing the stability of the beacon broadcast rhythm of the data node; and in step b, the preliminary sorting is based on the weighted credit score obtained by multiplying the endogenous credit score and the dynamic credit weight, and the calculation of the weighted credit score is: wherein, is the weighted credit score, is the dynamic credit weight, is the endogenous credit score.

4. The departmental data resource scheduling and management method according to claim 1, characterized in that, Also includes: In downstream business applications, set triggers for business veto operations; When the trigger is activated due to a business veto operation, a reputation instruction containing the identifier of the vetoed data source is generated and broadcast; and when the data requester terminal performs the preliminary sorting in step b, it temporarily decays the endogenous reputation score of the vetoed data source according to the received reputation instruction.

5. The departmental data resource scheduling and management method according to claim 1, characterized in that, In step c, the query intent signaling is a pre-request probe signal containing a unique identifier for the data item; the service health score is calculated by a health proxy of the candidate data source identified by the business logic, based on at least one performance indicator of its real-time monitored CPU load, memory usage, and current database connection count, using a preset weighted formula.

6. The departmental data resource scheduling and management method according to claim 1, characterized in that, It also includes: step a, where at least one data demander terminal collects data reputation beacons by classifying them according to the business domain fields contained in the data reputation beacons while receiving them; step b, where the data demander terminal counts the broadcast frequency of the data reputation beacons for each business domain in units of time windows and compares the broadcast frequency with the historical frequency baseline of the business domain; and step c, where a business trend warning signal is generated and sent when the broadcast frequency exceeds the predetermined range of the historical frequency baseline.

7. The departmental data resource scheduling and management method according to claim 1, characterized in that, Before step b, there is also an input purification step: the data requester terminal performs frequency domain feature analysis on the timestamp value stream of the business logic assertion update in the data reputation beacons received from different data nodes; When the power spectral density of the timestamp value stream of a certain data node is found to be abnormal in the high-frequency region, the timestamp value of the data node is normalized before the preliminary sorting in step b.

8. The departmental data resource scheduling and management method according to claim 4, characterized in that, The intrinsic reputation score is temporarily attenuated. Specifically, the data requester terminal maintains a temporary reputation score for each data source locally. When a reputation instruction is received for a specific data source, the temporary reputation score of that data source is reduced under the constraints of a preset attenuation coefficient and an upper limit. The temporary reputation score automatically recovers over time according to preset recovery rules.

9. A departmental data resource scheduling and management system, used to implement the method described in any one of claims 1-8, characterized in that, include: A beacon receiving module is configured to receive a data reputation beacon broadcast by multiple data nodes holding copies of the same data item. The data reputation beacon contains a unique identifier of the data item, a timestamp of a business logic assertion update completed by the data node, and an endogenous reputation score determined based on pre-defined ownership level information. A preliminary arbitration module is configured to perform preliminary sorting of multiple data nodes based on endogenous credit scores and timestamps of business logic assertion updates in order to identify one or more candidate data sources for business logic assertion. A health detection module is configured to send a query intent signaling to one or more business logic identification candidate data sources and receive a service health score returned by the business logic identification candidate data sources based on their own operating load status. A final arbitration module is configured to uniquely determine a final business logic assertion data source from the candidate data sources based on the results of the initial ranking and in conjunction with the service health score. A data acquisition module is configured to initiate data requests only to the data source identified by the final business logic in order to obtain data items.

Images