Inspection and data docking integrated arrangement method and system and computer readable storage medium

Abstract

The application belongs to the technical field of operation and maintenance task arrangement, and discloses a method and system for integrated arrangement of inspection and data docking and a computer readable storage medium. The two types of heterogeneous services of inspection and cross-system data docking are uniformly encapsulated in the same standardized task object. The full-amount service rules and configuration parameters are carried by a vertical four-layer integrated task object model. The physical isolation and controlled transfer mechanism of three types of independent context partitions are matched. On the premise of guaranteeing independent control of inspection business data and data docking business data, the closed-loop linkage of global states such as inspection abnormal data, field mapping quality parameters, permission coding rules and execution permission identification in the same task full life cycle is realized. The data intercommunication and state linkage of the two types of services are realized from the bottom of the task carrier, the data island problem of heterogeneous services is eliminated, and the configuration management cost and operation and maintenance complexity of multi-system distributed deployment are reduced.

Description

Technical Field

[0001] This invention belongs to the field of operation and maintenance task orchestration technology, specifically an integrated orchestration method, system, and computer-readable storage medium for inspection and data docking. Background Technology

[0002] In a digital operations and maintenance (O&M) business system, O&M inspection and cross-system data synchronization are core foundational links for the stable operation of business systems and the unified, comprehensive business data. Currently, most digital O&M platforms adopt a split-architecture approach, independently deploying inspection services, data integration services, and task scheduling services, and connecting various business modules through external workflow interface calls to complete integrated O&M operations. However, the existing digital O&M business system still has the following technical problems: The existing split-operation architecture cannot achieve deep integration between inspection and data docking operations. The two types of business tasks are configured and stored independently, and there is no unified standardized task carrier or dedicated data isolation space. The operation personnel's permission control is scattered, and it is impossible to uniformly encapsulate and manage heterogeneous business tasks. The business interaction process relies only on the transmission of simple instructions through the surface interface, and cannot achieve bidirectional communication of context data.

[0003] At the same time, the existing task scheduling system has problems such as insufficient scheduling adaptability and distributed resource preemption conflicts in engineering scenarios where two types of heterogeneous tasks are deployed together. The supporting general engineering mechanisms such as timeout release, idempotent deduplication, hierarchical retry, and state transition management need to be improved.

[0004] The existing operation and maintenance platform uses fixed thresholds to complete risk assessment, and only refers to abnormal inspection data to identify operational risks. It does not combine field mapping quality and historical execution data for comprehensive assessment, and the risk assessment dimension is relatively simple. In addition, the traditional architecture lacks a full-process data collection and unified archiving mechanism. Summary of the Invention

[0005] The purpose of this invention is to provide an integrated orchestration method, system, and computer-readable storage medium for inspection and data interoperability, so as to solve one or more problems mentioned in the background art.

[0006] To achieve the above objectives, the present invention provides the following technical solution: This invention does not perform operation and maintenance inspections in isolation, nor does it perform cross-system data synchronization in isolation. Instead, it sets up inspection business context partitions, data docking business context partitions, and global common context partitions within a unified task object, enabling abnormal inspection data, field mapping quality, permission encoding, and execution permission identifiers to be linked within the same task lifecycle. Through a weighted gating algorithm based on inspection risk scores and field mapping confidence, it performs hierarchical permission control on cross-system data write actions, blocking formal writes under high-risk conditions, triggering trial runs and secondary approvals under medium-risk conditions, allowing full execution under low-risk conditions, and enabling rollback of temporary resource and configuration changes.

[0007] The integrated orchestration method for inspection and data integration includes the task creation stage, rule generation stage, approval and verification stage, queue control stage, risk assessment stage, execution write-back stage, and evidence aggregation stage. Furthermore, during the task creation phase, the system is configured with a four-layer integrated task object model, consisting of a basic information layer, an access control layer, an inspection business layer, and a data integration business layer, from top to bottom. The four-layer integrated task object model contains seven types of structured fields: basic task information, inspection rule set, data integration specifications, execution strategy, approval information, execution status, and access configuration. The task object is also divided into three independent context partitions. After receiving the inspection target parameters, data docking target parameters, and task execution constraint parameters issued by the user, the system initializes and generates corresponding task objects according to the field specifications of the four-layer integrated task object model. At the same time, it assigns a unique global task ID, idempotent key, and permission code to each task object and sets the initial state of the task to draft state. The system defines the operation permission boundaries for three types of operators—ordinary maintenance personnel, senior administrators, and approvers—using permission codes. Within each task object, three independent context partitions are defined: an inspection business context partition, a data integration business context partition, and a global common context partition. No direct data read / write channel is established between the inspection business context partition and the data integration business context partition. Data that needs to be shared across partitions is uniformly written to the global common context partition for intermediate storage. Data written to the global common context partition is simultaneously marked with three attributes: business type, access permission identifier, and data validity period. Before any partition reads data from the global common context partition, it must verify the match between the access subject's permission code and the data access permission identifier, and simultaneously verify whether the data is within its validity period. Data reading can only be completed after successful verification. Globally shared data such as the comprehensive risk value, execution permission identifier, and global permission code generated throughout the task's lifecycle are transferred in a controlled manner between the inspection business process and the data integration business process through the global common context partition. After completing the initial configuration, the task object enters the status transition queue.

[0008] Furthermore, in the rule generation stage, the system calls the built-in business template to generate an inspection rule set that includes alarm check sub-rules, indicator time window check sub-rules, automated task check sub-rules, data consistency check sub-rules, and anomaly tracing sub-rules. The alarm inspection sub-rule is used to traverse the target system alarm log table and filter out unprocessed high-level alarm data. The indicator time window inspection sub-rule collects the performance indicator data of the target system within the corresponding time period according to the time interval configured by the user. The automated task inspection sub-rule traverses the target system background task list and filters automated tasks that are in running status and associated with data services. The data consistency check sub-rule synchronously reads the baseline data tables of the source system and the target system and completes bidirectional field comparison. The anomaly tracing sub-rule is associated with the other four types of sub-rules and synchronously collects the device module and business link number corresponding to the abnormal data. The system synchronously reads the metadata tables of the source and target systems and uses a weighted algorithm with five calculation factors, including field name semantic similarity, data type compatibility, value range matching, historical mapping success rate, and business importance, to complete the field mapping configuration. The weighted algorithm dynamically adjusts the weight values ​​of each calculation factor according to the field business label. The system presets a first confidence threshold and a second confidence threshold. Based on the confidence value output by the algorithm, the field mapping relationship is divided into three levels: those with a confidence value greater than or equal to the second confidence threshold are classified as automatic pass level; those with a confidence value between the first and second confidence thresholds are classified as manual review level; and those with a confidence value less than the first confidence threshold are classified as rejection level. The system will eventually write the inspection rule set, traceability parameters, field mapping relationship table, and weight assignment parameters into the corresponding task object, and update the running status of the corresponding task object to the pending approval status.

[0009] Furthermore, during the approval and verification phase, the system is configured with a status manager. The status manager has six enumerated types of task running states: draft, pending approval, ready, running, successful execution, and failed execution. The task object completes the status change according to the preset jump logic of the status manager. For tasks in the pending approval state, the system reads the field mapping level and inspection risk data stored in its internal storage to perform hierarchical approval. Tasks with a field mapping level of automatic approval and no high-risk inspection data detected are automatically approved by the system. Tasks with field mapping levels of manual review or with medium-risk inspection data detected are pushed to the terminal of department managers. Tasks with blocking-level risk tags and core business tags are pushed to the terminal of senior managers. After receiving a task to be approved, the department manager retrieves all field mapping entries marked as manual review within the task and checks the name, data type, business meaning, value range, and preset conversion rules of the source and target fields one by one. If the field mapping relationship matches the business requirements and the conversion rules are valid, manually adjust the corresponding field mapping level to automatic approval; if the field mapping relationship has errors or the conversion rules do not meet the requirements, mark the specific reasons for the errors and adjustment suggestions; after all fields have been manually reviewed and processed, submit the overall approval result.

[0010] After the approval process is completed, the system sequentially performs four pre-verification tasks: cross-system connectivity detection, operator permission verification, server hardware resource usage detection, and task resource relationship calculation based on a directed acyclic graph. After all four pre-verification tasks pass the verification, the system updates the task object status to the ready state and generates an execution request encapsulating permission codes, weight parameters, and anomaly tracing numbers.

[0011] Furthermore, in the queue management phase, the system builds a partitioned relational database queue list. The relational database queue list has 14 basic data fields. The system uses the task type field as the classification basis to divide the queue list into inspection task partitions and data docking task partitions. After receiving an execution request in the ready state, the system writes it into the data table of the corresponding queue partition according to the type of the task. At the same time, it initializes the schedulable timestamp to the current timestamp, sets the number of retries to 0, and sets the task status to pending execution. In the distributed architecture, the execution node polls the corresponding queue partition at a fixed time period and executes a row-level lock query statement with skip lock parameters and sorted in descending order of weight coefficient in an independent database transaction. After the execution node successfully preempts the corresponding task, it updates the lock time and execution node identifier field in the data table. The system configures a dedicated timeout duration parameter for the preempted task. After the task timeout duration expires, the row-level lock corresponding to the task is automatically released to release resources. The system uses the idempotent key field as the globally unique deduplication identifier and classifies retry levels according to inspection risk labels and business importance levels. A three-level retry execution mechanism is built through the retry level field. The maximum number of retry times for low-risk tasks is set to 2 with a fixed retry interval. The maximum number of retry times for medium-risk tasks is set to 5 with a gradient increasing retry interval. The automatic retry function is directly disabled for blocking tasks.

[0012] Furthermore, in the risk assessment stage, the execution node calls the inspection rule set bound inside the corresponding task object, executes five types of sub-rules in sequence and generates a standardized structured inspection report, and marks four types of task risk labels in the inspection report: normal, prompt, warning and blockage. The system synchronously starts the weight gating algorithm, collects three types of operating parameters in real time: server load ratio, task priority code, and historical failure count. It dynamically completes the assignment operation of inspection risk weight W1 and field mapping confidence weight W2 by combining server load ratio, task priority code, and historical failure count. The sum of the two weight values ​​is always 1. The algorithm calculates and outputs the comprehensive risk value of the task by combining the weight values, inspection risk score, and field mapping low confidence risk score. The system presets three fixed judgment thresholds: a first risk threshold, a first confidence threshold, and a second confidence threshold. It also defines four types of execution permission identifiers based on the comprehensive risk value, risk level label, and confidence value. When the blocking-level risk label or the comprehensive risk value is greater than the first risk threshold, it is identified as a trial operation only. When a warning-level risk label is set and the confidence value is between the first confidence threshold and the second confidence threshold, it is marked as trial operation and requires secondary approval. When a normal or prompt-level risk label is set and the confidence value is greater than or equal to the second confidence threshold, it is marked as allowed to complete execution. When a task that has completed pre-verification adds blocking-level abnormal data during the waiting execution period, it is marked as rollback only. The rollback only mode is used to revoke temporary configurations and data generated in the pre-verification stage, resource pre-occupancy stage, or trial operation stage, and does not involve the revocation of formal business data that has not yet occurred. Finally, the system stores the inspection report, dynamic weight parameters, comprehensive risk value, and execution permission identifier in a context partition dedicated to the task object.

[0013] Furthermore, during the execution write-back phase, the execution node reads the execution permission identifier configured inside the task object and matches the dedicated job execution mode. In the trial run mode only, the execution node only completes the original data reading and field format conversion operations of the source system and does not execute any data writing instructions. In the trial operation and secondary approval mode, the execution node sequentially completes the entire operation process of data reading, format conversion, trial operation report generation, multi-level approval verification, and formal data writing. In the full execution mode, the execution node can directly execute the complete data docking process of data reading, format conversion, data writing, and result data verification. In rollback-only mode, the execution node calls the built-in transaction rollback interface to atomically undo temporary permission configurations, interface pre-connection caches, resource pre-reservation flags, intermediate table data, trial run temporary cache data, and all temporary configuration changes generated during the pre-verification phase, resource pre-reservation phase, and trial run phase. Since no formal business data writing operation has been initiated during the waiting execution phase, the rollback operation does not involve the revocation of any formal business data. During task execution, the system continuously collects basic data including execution status and runtime, business data including anomaly point numbers, field mapping matching details, approval operation logs, and data layer data including total written data, data primary key number, and data snapshots before and after changes. The system synchronously writes the three types of operational data collected into the task object, the relational database team list, and the permission isolation database, and restricts the operational data that different roles can only access based on their corresponding permission codes.

[0014] Furthermore, in the evidence aggregation stage, after the task completes the work process in the corresponding execution mode, the system integrates the hierarchically collected operation data, anomaly tracing and inspection reports, task execution logs, multi-level approval operation records and permission access logs to generate business execution reports and audit reports. The system assigns the task object status to the value of execution success or execution failure based on the final execution result of the task. For tasks marked as failed, the system calculates the next available schedulable timestamp by combining the task's retry level parameters and maximum retry count parameters, and synchronously updates the schedulable timestamp field in the queue list so that the task returns to the corresponding queue partition to wait for scheduling. If the task still cannot be executed normally after the cumulative number of retries reaches the limit, the system locks the task and generates differentiated alarm messages, which are pushed to the terminals of ordinary maintenance personnel and senior administrators respectively. The system periodically extracts the inspection anomaly dataset and field mapping execution dataset of all completed tasks, and synchronously updates the sub-rule parameters and weighted algorithm weight allocation parameters within the inspection rule set.

[0015] This invention also provides an integrated orchestration system for inspection and data integration, which, based on the above method, includes the following modules: The task modeling module configures a four-layer integrated task object model and a permission configuration unit. It receives business configuration parameters from external operation and maintenance terminals and completes task object initialization creation, global unique identity allocation, operator permission binding, and context partition configuration. The rule approval module has a built-in inspection rule template library, weighted algorithm operation unit and status manager. It calls preset templates to generate five types of inspection sub-rules, collects business system operation data and completes cross-system field mapping matching, completes the mapping relationship level division according to confidence parameters, and reads the associated attributes of task objects to complete the hierarchical approval operation and the pre-verification operation of connectivity, permission and resource levels in sequence. The queue scheduling module configures a partitioned relational database queue list, integrates a transaction lock control unit, an idempotent deduplication unit, and a hierarchical retry unit, receives execution requests from the rule approval module, performs heterogeneous task partitioning and storage according to task type, realizes task preemption of distributed execution nodes through the database row-level locking mechanism, and provides execution timeout unlocking, duplicate task interception, and hierarchical retry scheduling operations. The risk assessment module, which embeds a weighted gating algorithm, retrieves the inspection rule set bound to the task object to collect business system operation data, dynamically allocates the weight parameters corresponding to the inspection risk and field mapping confidence level, calculates the comprehensive risk value, and marks the corresponding execution permission identifier with reference to the first risk threshold, the first confidence threshold, and the second confidence threshold preset by the system. The execution archiving module connects the source business system and the target business system bidirectionally. It calls the corresponding job execution mode according to the execution permission identifier, collects running data in layers and implements permission-isolated storage, integrates running data to generate structured reports, synchronously updates task running status fields, performs secondary scheduling for failed tasks, and iteratively updates the internal configuration parameters of inspection rules and field mapping algorithms.

[0016] The present invention also provides a computer-readable storage medium storing a computer-executable program, which, when executed by a processor, realizes an integrated orchestration process for inspection and data interfacing: Build and initialize the task object model, assign identity and permission parameters, and configure independent context partitions; Generate a set of inspection rules, complete cross-system field mapping through a weighted algorithm, and perform hierarchical approval and pre-verification based on task attributes; For heterogeneous tasks, partition storage is used, and distributed scheduling is completed based on database transactions and row-level locks. Parameters for timeout unlocking, idempotent deduplication, and hierarchical retry are configured. Collect system operation data, calculate the comprehensive risk value using a weighted gating algorithm, and assign an execution license identifier; Match the corresponding job mode to complete data processing, and isolate and store the entire process data; Aggregate runtime data to generate reports, update task status and handle abnormal tasks, and iteratively update built-in inspection rules and field mapping algorithm parameters.

[0017] The beneficial effects of this invention are as follows: 1. This invention encapsulates two heterogeneous business processes—inspection and cross-system data integration—within a single standardized task object. A vertically integrated four-layer task object model carries all business rules and configuration parameters. Combined with a horizontally integrated physical isolation and controlled transfer mechanism for three independent context partitions, it achieves closed-loop linkage of global states such as abnormal inspection data, field mapping quality parameters, permission coding rules, and execution permission identifiers throughout the entire lifecycle of the same task, while ensuring independent management of inspection and data integration business data. This enables data interoperability and state linkage between the two business processes from the bottom layer of the task carrier, eliminating data silos between heterogeneous businesses and reducing the configuration management costs and operational complexity of multi-system deployments.

[0018] 2. This invention constructs a weighted gating algorithm based on a dual dimension of inspection risk score and field mapping confidence, enabling refined hierarchical permission control for cross-system data write operations. The algorithm dynamically allocates weights for the two risk dimensions based on three types of operating parameters: server load status, task priority level, and historical execution failure rate. This comprehensively assesses the overall risk level of the task and matches differentiated execution control strategies: in high-risk states, formal data writing is directly blocked, and only non-intrusive trial operations are performed; in medium-risk states, trial verification and multi-level secondary approval are triggered, and writing is only performed after manual confirmation; in low-risk states, the entire data integration process is allowed; and in the event of sudden anomalies during the waiting execution phase, atomic rollback of temporary resources and configuration changes across the entire chain can be performed. This avoids data security issues caused by blind writing in high-risk scenarios and reduces unnecessary approval process losses in low-risk scenarios.

[0019] 3. This invention is supported by a set of engineering mechanisms, including partitioned queue scheduling, database row-level lock preemption, global idempotent key deduplication, tiered retries, state machine full lifecycle management, tiered approval, access control with permission coding, and multi-dimensional report generation. These mechanisms provide a complete implementation guarantee for the core integrated context linkage and risk gating writing logic. Among them, the partitioned queue list can adapt to the differentiated scheduling characteristics of long-cycle inspection tasks and instantaneous data docking tasks; the database row-level lock and timeout release mechanism can avoid resource preemption conflicts between distributed nodes; idempotent deduplication and tiered retries can ensure the uniqueness and fault tolerance of task execution; state transition management, tiered approval, and permission isolation can achieve full-process compliance and traceability; and the dual reporting mechanism for business and audit can meet the dual needs of daily operation and maintenance and compliance audit. Attached Figure Description

[0020] Figure 1 This is the overall flowchart of the integrated orchestration of the present invention; Figure 2 This is a flowchart illustrating the licensing and calibration process for this invention. Detailed Implementation

[0021] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0022] like Figures 1 to 2 As shown, this embodiment of the invention provides an integrated orchestration method for inspection and data integration, including a task creation stage, a rule generation stage, an approval and verification stage, a queue management stage, a risk assessment stage, an execution write-back stage, and an evidence aggregation stage. In this embodiment of the invention, during the task creation phase, the system is configured with a four-layer integrated task object model, consisting of a basic information layer, a permission control layer, an inspection business layer, and a data docking business layer, from top to bottom. The four-layer integrated task object model contains seven types of structured fields: basic task information, inspection rule set, data docking specification, execution strategy, approval information, execution status, and permission configuration. The task object is further divided into three independent context partitions. The four-layer integrated task object model is a vertical business hierarchy architecture. The four layers from top to bottom correspond to the four management dimensions of basic attributes, permission control, inspection business, and data docking throughout the entire task lifecycle. The three independent context partitions form a horizontal data isolation and interaction architecture, which together with the four-layer business hierarchy form an integrated task carrier structure that "carries business rules in the vertical hierarchy and controls data flow in the horizontal partitions".

[0023] The basic information layer carries basic task information fields, including the global task ID, idempotent key, business type tag, creation time, and basic attribute data of the account of the person who created the task. The access control layer carries access configuration and approval information fields, including task access codes, access boundaries for the three types of operators, accounts of approvers at all levels, and approval operation records. The inspection business layer carries inspection rule sets and execution status fields, including parameters for five types of inspection sub-rules, inspection risk thresholds, task running status identifiers, and inspection process data. The data integration business layer carries data integration specifications and execution strategy fields, including a field mapping table, weighted algorithm parameters, job execution mode configuration, and data write-back rule data.

[0024] The three independent context partitions correspond to the storage of inspection business operation data, data docking business operation data, and global public shared data, respectively. Each partition performs physical isolation storage, forming a data isolation and controlled interoperability system within the task object.

[0025] No direct data read / write channel is established between the inspection business context partition and the data docking business context partition. This eliminates uncontrolled interaction between the two types of heterogeneous business data at the physical link level, ensuring independent management and control of inspection business data and data docking business data. Data that needs to be shared between the two business partitions is uniformly written to the global common context partition for transit storage. All cross-partition data interactions must be completed through the global common context partition, and there are no bypass data transmission paths.

[0026] When data is written to the global public context partition, it is simultaneously accompanied by three types of attribute tags: the business type to which the data belongs, the access permission identifier, and the data validity period identifier. The business type tag is used to identify the business partition from which the data originates, enabling traceability of the data source. The access permission identifier is used to limit the permission level that can read the data, and the access permission control is completed by matching the permission coding system. The data validity period identifier is used to limit the valid reading period of the data. Expired data will automatically become invalid and trigger the background cleanup mechanism.

[0027] When any partition reads data from the global public partition, it first verifies whether the access permission code matches the data access permission identifier, and also verifies whether the data is within its validity period. Only after all verifications pass can the corresponding data content be read. If the verification fails, the read request is directly intercepted and an access exception log is recorded.

[0028] The global public context partition stores five types of globally shared data: a globally unique task identifier, permission code, overall risk label, comprehensive risk value, and execution permission identifier. The inspection business context partition and the data integration business context partition can only write data they generate that needs to be shared, and can only read public data within their own permission scope. Throughout the entire task lifecycle, status data such as risk labels, comprehensive risk values, and execution permission identifiers output by the inspection process are written to the global public partition in real time. The data integration process matches the operation mode by reading the corresponding status data in the public partition, thereby achieving status linkage between the two heterogeneous business processes. The public partition data is updated and overwritten synchronously with the execution progress of each stage.

[0029] After receiving the inspection target parameters, data docking target parameters, and task execution constraint parameters issued by the user, the system initializes and generates corresponding task objects according to the field specifications of the four-layer integrated task object model. At the same time, it assigns a unique global task ID, idempotent key, and permission code to each task object and sets the initial state of the task to draft state. The idempotent key is generated by concatenating three segments in sequence: task business type code, task creation timestamp, and fixed-length random checksum. It is globally unique and corresponds one-to-one with a single task. The permission code is generated based on the combination of the business importance level and data sensitivity level bound to the task. Each permission code corresponds to a preset operation permission range and data access range. The system pre-binds three types of operator accounts with corresponding permission codes, and automatically matches the corresponding permission code when an account initiates an operation request.

[0030] The system defines the operation permission boundaries for three types of operators through permission codes, and the task objects that have completed the initial configuration work enter the status transition queue.

[0031] In this embodiment of the invention, during the rule generation stage, the system calls the built-in business template to generate a set of inspection rules that includes alarm check sub-rules, indicator time window check sub-rules, automated task check sub-rules, data consistency check sub-rules, and anomaly tracing sub-rules. The alarm inspection sub-rule is used to traverse the target system alarm log table and filter out unprocessed high-level alarm data. The indicator time window inspection sub-rule collects the performance indicator data of the target system within the corresponding time period according to the time interval configured by the user. The automated task inspection sub-rule traverses the target system background task list and filters automated tasks that are in running status and associated with data services. When executing the alarm check sub-rule, the system retrieves the alarm log data table within the preset time range of the target system, filters out the preset high-level alarm entries according to the alarm level field, filters out alarm records that have been marked as handled or closed, extracts the alarm number, occurrence time, device, alarm level, and handling status fields of the remaining alarm entries, and generates an alarm detection result set. When executing the indicator window check sub-rule, the system reads the detection start time and end time parameters configured in the task, retrieves four types of core performance indicator data of the target system in the corresponding time period: CPU utilization, memory utilization, disk I / O, and interface response time, calculates the average, peak, and valley values ​​of each indicator at a fixed time granularity, compares them with the preset normal threshold range of the indicators, marks the time period data that exceeds the threshold range as indicator anomalies, and generates an indicator detection result set; When executing the sub-rules for checking automated tasks, the system retrieves a list of all scheduled and real-time tasks in the target system's background, filters automated tasks marked as data synchronization, data transmission, and data verification, checks the current running status, most recent execution result, and next execution time of each task, marks automated tasks that are in the running state and whose execution time overlaps with the current task execution window as conflicting task items, and generates a task detection result set.

[0032] The data consistency check sub-rule synchronously reads the baseline data tables of the source system and the target system and completes bidirectional field comparison. The anomaly tracing sub-rule is associated with the other four types of sub-rules and synchronously collects the device module and business link number corresponding to the abnormal data. The system first filters the list of baseline data tables corresponding to the source system and the target system according to the business scope configured in the task, and pulls the full data snapshot and table structure metadata of each baseline data table; using the primary key field of the data table as the only matching basis, it aligns the data entries of the source end and the target end and generates a primary key matching result set; for the successfully matched data entries, it verifies the numerical content, data format, length precision of the preset core business fields column by column, marks the entries with different field content as data inconsistency anomalies, and records the names of the different fields and their values ​​before and after the difference simultaneously; For data entries where the primary key did not match, two types of one-sided data entries are counted: those that exist in the source but are missing in the target, and those that exist in the target but are missing in the source. These are uniformly marked as data missing anomalies. Finally, all anomalies are summarized to form a data consistency detection result set, which is written into the inspection rule execution log.

[0033] During the execution and detection of the other four types of sub-rules, each time a data entry that meets the anomaly judgment conditions is detected, the execution logic of the anomaly tracing sub-rule is immediately triggered. The anomaly tracing sub-rule extracts the primary key identifier, system code, and associated business node number of the current abnormal data as the starting index for tracing, and traces upstream along the business call chain to the dependent service nodes and underlying device modules, and downstream to the affected business chain and data target nodes. Traverse all associated nodes in the entire chain and collect four types of associated information for each node: device module number, business link node code, dependent service identifier, and call log serial number, to form a complete abnormal source tracing dataset.

[0034] The system synchronously reads the metadata tables of the source and target systems and uses a weighted algorithm with five calculation factors, including field name semantic similarity, data type compatibility, value range matching, historical mapping success rate, and business importance, to complete the field mapping configuration. The weighted algorithm dynamically adjusts the weight values ​​of each calculation factor according to the field business label. The system presets a first confidence threshold and a second confidence threshold. Based on the confidence value output by the algorithm, the field mapping relationship is divided into three levels: those with a confidence value greater than or equal to the second confidence threshold are classified as automatic pass level; those with a confidence value between the first and second confidence thresholds are classified as manual review level; and those with a confidence value less than the first confidence threshold are classified as rejection level. For field mapping relationships with a confidence level value less than the first confidence level threshold and marked as rejection, the system automatically marks them as mapping anomalies, records the names, confidence levels, and reasons for mapping failures of the corresponding source and target fields, and summarizes all mapping anomalies to generate an anomaly list, which is then pushed to the operation terminal of the task creator. After receiving the anomaly list, personnel manually adjust the field matching relationships or supplement the field conversion rules. After the adjustment is completed, the list is resubmitted to the weighted algorithm unit, and the field mapping operation and level division are executed again until there are no rejection level entries in all field mapping relationships of the current task.

[0035] The algorithm takes the source and target field metadata as input data sources and extracts four basic attributes: field name, data type definition, value range enumeration range, and business affiliation tag. For the semantic similarity factor of field name, it performs word segmentation and semantic vector transformation on the source and target field names, calculates the cosine distance between the two semantic vectors and maps it to a similarity score in the range of 0 to 1. For the data type compatibility factor, the preset type compatibility mapping table is traversed to match the compatibility level of the data types of the source and target ends, and the corresponding compatibility score is converted into a range of 0 to 1. For the value range matching factor, the upper and lower limits of the value range and the enumerated value set of the fields at both ends are extracted, the proportion of the overlapping value range to the total range of the value range of the source end is calculated, and the output is a matching score in the range of 0 to 1. For the historical mapping success rate factor, retrieve the historical mapping success counts and total mapping counts of fields with the same name and type from the system's historical mapping log library, and calculate the proportion of the success counts to the total counts as the corresponding score; For the business importance factor, read the business level label bound to the field, match it with the preset importance assignment table, and generate an importance score in the range of 0 to 1; read the business label bound to the current field, match it with the system's preset weight allocation table, and obtain the weight coefficients corresponding to each of the five factors. The sum of the values ​​of the five weight coefficients is 1; multiply the corresponding scores of the five factors by their respective weight coefficients, sum the results of the five products, and output the final field mapping confidence value.

[0036] Cross-system field mapping confidence calculation formula: ; C represents the confidence level of the field mapping, with a value ranging from 0 to 1. The higher the value, the better the matching quality and business adaptability of the field mapping. The weight coefficient representing the semantic similarity factor of field names is determined by the preset weight allocation table of the business tag dynamic matching system bound to the field; This represents the semantic similarity score of the field names, with a value ranging from 0 to 1. It is obtained by performing word segmentation and semantic vector transformation on the source and target field names, and then mapping the result after calculating the cosine distance. The weight coefficient representing the data type compatibility factor is determined by the preset weight allocation table of the business tag dynamic matching system bound to the field; This represents the data type compatibility score, ranging from 0 to 1. It is obtained by traversing the preset type compatibility mapping table, matching the compatibility levels of the source and target data types, and then converting the result. The weight coefficient representing the value range matching degree factor is determined by the preset weight allocation table of the business tag dynamic matching system bound to the field; This represents the score for the range of values ​​matching the target field. The value range is 0 to 1, and it is the proportion of the overlapping range of values ​​between the source and target fields to the total range of values ​​in the source field. The weight coefficient representing the historical mapping success rate factor is determined by the preset weight allocation table of the business tag dynamic matching system bound to the field; This represents the historical mapping success rate score, ranging from 0 to 1. It is the proportion of successful historical mappings of fields with the same name and type in the system's historical mapping log database to the total number of mappings. The weight coefficient representing the business importance factor is determined by the preset weight allocation table of the business tag dynamic matching system bound to the field; This represents the business importance score, ranging from 0 to 1, and is obtained by matching the business level tag bound to the field with a preset importance assignment table.

[0037] The system will eventually write the inspection rule set, traceability parameters, field mapping relationship table, and weight assignment parameters into the corresponding task object, and update the running status of the corresponding task object to the pending approval status.

[0038] In this embodiment of the invention, during the approval and verification stage, the system is configured with a status manager. The status manager has six enumerated types of task running states: draft state, pending approval state, ready state, running state, successful execution, and failed execution. The task object completes the state change according to the preset jump logic of the status manager. After task creation, it initially enters the draft state. After completing the inspection rule configuration and field mapping calculation, it jumps to the pending approval state. After approval and all four pre-verifications are qualified, it jumps to the ready state. After the execution node successfully seizes the task, it jumps to the running state. After the task completes all operations according to the corresponding job mode and passes the verification, it jumps to the execution successful state. If an unrecoverable exception occurs during task execution or the cumulative number of retries reaches the limit, it jumps to the execution failure state. If the approval is rejected, the task reverts to the draft state. If the pre-verifications are unqualified, the task reverts to the pending approval state.

[0039] For tasks in the pending approval state, the system reads the field mapping level and inspection risk data stored in its internal storage to perform hierarchical approval. Tasks with a field mapping level of automatic approval and no high-risk inspection data detected are automatically approved by the system. Tasks with field mapping levels of manual review or with medium-risk inspection data detected are pushed to the terminal of department managers. Tasks with blocking-level risk tags and core business tags are pushed to the terminal of senior managers. After receiving the task to be approved, the terminal of each level of the approver loads and displays the inspection rule parameters associated with the task, field mapping level details, risk tags and corresponding abnormal basis data; the approver selects the approval result on the operation interface, enters the approval opinion text and submits the operation instruction. After receiving the approval instruction, the system updates the task running status to "approved", records the approver's identifier and approval time, and triggers the subsequent pre-verification process; after receiving the approval rejection instruction, the system reverts the task running status to the draft state, and writes the rejection personnel identifier, rejection time, rejection reason text and modification instructions. After a task is returned, the operator can view the rejection information and adjust the task's inspection rules, field mapping configuration, execution parameters, etc. After the adjustment is completed, the approval application is resubmitted. After the system receives the resubmitted application, it clears the original approval process record, resets the task status to pending approval, and re-triggers the hierarchical approval process.

[0040] After the approval process is completed, the system performs four pre-verification checks in sequence: cross-system connectivity detection, operator permission verification, server hardware resource usage detection, and task resource relationship calculation based on a directed acyclic graph. After all four pre-verification checks pass the verification signal, the system updates the task object status to the ready state and generates an execution request encapsulating permission codes, weight parameters, and anomaly tracing numbers.

[0041] If any of the four pre-verification checks fails during the execution of the four checks, the system will terminate the execution of the remaining checks, record the type of the failed check, the reason for the anomaly, and the details of the test data; the task running status will be rolled back to the pending approval status, and a verification anomaly notification will be generated simultaneously, along with details of the anomaly and rectification guidelines, and pushed to the terminal of the person who created the task; after the person completes the rectification of the anomaly, the pre-verification application will be resubmitted, and the system will execute the four pre-verification checks again in sequence.

[0042] When performing cross-system connectivity detection, the system sends connectivity detection messages in a preset format to the source business system and the target business system respectively, receives response messages returned by both systems, verifies the return status code and data format of the response messages, and confirms that the cross-system data transmission link is unobstructed. When performing operator permission verification, the system extracts the operator account and permission code bound to the task, verifies the account's data read and write permissions and task operation permissions in the source system and target system, and confirms that the permission scope covers all operation items of this task. When performing server hardware resource usage detection, the system reads four hardware operation indicators of the target execution node in real time: CPU utilization, memory utilization, disk space remaining, and network bandwidth utilization. These indicators are then compared with preset resource thresholds to confirm that the remaining resources of the execution node can support the operation of this task.

[0043] When performing task resource relationship calculations, the system first extracts four types of resource items currently occupied by the task: computing resource quota, storage resource quota, interface call quota, and database connection quota. Then, it retrieves a list of all tasks in the pending execution state within the current time window and constructs each pending task as an independent node in a directed acyclic graph. If two tasks have the same exclusive resource requirement, then the two corresponding nodes are connected by a directed edge. Perform a topological traversal on the constructed directed acyclic graph, verify whether there are irreconcilable resource conflicts and deadlock risks between the current task and other concurrent tasks, and output the corresponding calculation and verification results after the traversal is completed.

[0044] In this embodiment of the invention, during the queue management phase, the system builds a dedicated relational database queue list. The relational database queue list is configured with 14 basic data fields, including id, global task ID, schedulable timestamp, task lock timestamp, execution node number, current retry count, maximum retry count, idempotent key, task status, execution result, task type, weight coefficient, deadlock prevention flag, and retry level. The system uses the task type field as the classification basis to divide the queue list into an inspection task partition and a data docking task partition. The system receives execution requests that are in a ready state, writes them into the data table of the corresponding queue partition according to the type of the task, initializes the schedulable timestamp to the current timestamp, sets the number of retries to 0, and sets the task status to pending execution. In the distributed architecture, the execution nodes poll the corresponding queue partitions at fixed time intervals and execute row-level lock query statements with skip lock parameters and sorted in descending order of weight coefficients in an independent database transaction. The system reads three types of attribute parameters bound to the task: business importance level tag, inspection risk level tag, and task urgency level tag; for the business importance level, different basic weight scores are assigned to the three levels of core business, important business, and ordinary business. Based on the inspection risk level, different risk weight scores are assigned to four levels: Interruption Level, Warning Level, Alert Level, and Normal Level; based on the urgency of the task, different urgency weight scores are assigned to three levels: Extremely Urgent, Urgent, and Normal Level. The total weight coefficient of the task is obtained by adding the three weight scores together. The higher the value, the higher the scheduling priority. The calculated weight coefficient is written to the weight coefficient field of the corresponding task record in the relational database queue list.

[0045] When a distributed execution node starts up, it registers its own task type adaptation identifier with the scheduling center. The system assigns corresponding queue partition access permissions to each node according to the adaptation identifier. Each execution node periodically sends a task query request to the matching queue partition according to a preset fixed polling cycle. The polling request carries the node's own identifier and current load parameters. The system filters out high-resource-consuming tasks that exceed the node's load capacity. When querying task records, the deadlock prevention identifier field is read synchronously. This identifier is used to mark tasks with resource consumption conflict risks. Task records marked as having deadlock risk are given priority to enter the subsequent locking verification process. The system backend iterates through all locked task records at a fixed scanning cycle, verifies the lock duration and the most recent execution progress update time for each locked task; for tasks that exceed the preset maximum lock duration and have no progress update, the corresponding database row-level lock is forcibly released, the task status is reset to pending execution, the original execution node identifier and lock time field are cleared, and the task is put back into the scheduling queue.

[0046] After the execution node successfully preempts the corresponding task, it updates the lock time and execution node identifier field in the data table. The system configures a dedicated timeout duration parameter for the preempted task. After the task timeout duration expires, the row-level lock corresponding to the task is automatically released to release resources. The status field of the corresponding task in the relational database queue is updated to running status synchronously. At the same time, a status change notification is sent to the bound task object to update the running status of the task object synchronously. The timestamp and trigger node identifier of this status change are recorded and written to the status flow log built into the task object.

[0047] The system reads the type label and risk level parameters of the current task and matches them with the system's preset timeout configuration table; inspection tasks are matched with long-cycle timeouts, and data docking tasks are matched with short-cycle timeouts; among tasks of the same type, the higher the risk level, the shorter the corresponding timeout; the configured timeout value is written into the lock duration field of the corresponding task in the team list.

[0048] The execution node starts an independent database transaction, sets the corresponding transaction isolation level, constructs a structured query statement, filters task records with a status of pending execution and a schedulable timestamp less than or equal to the current timestamp, and sorts them by weight coefficient from high to low. The query statement is appended with an exclusive row-level lock and skip lock parameters. The first unlocked task record after sorting is read and a row-level lock is applied. After locking, the real-time status and idempotent key uniqueness of the current task are verified to confirm that the task has not been executed by other nodes and there are no duplicate scheduling records. If the verification passes, the task preemption is confirmed to be completed. If the verification fails, the current database transaction is rolled back and the next round of task polling operation is continued.

[0049] The system uses an idempotent key field as a globally unique deduplication identifier and classifies retry levels based on inspection risk labels and business importance levels. A three-level retry execution mechanism is built through the retry level field. The maximum number of retries for low-risk tasks is set to 2, with a fixed retry interval of 30 seconds. The maximum number of retries for medium-risk tasks is set to 5, with a gradient increasing retry interval of 20 seconds, 40 seconds, 60 seconds, 90 seconds, and 120 seconds. The automatic retry function is directly disabled for blocking tasks.

[0050] The system extracts the inspection risk label and business importance level label associated with the task and performs a combined judgment. Tasks with an inspection risk label of normal or warning level and a business importance level of ordinary business are classified as Level 1 low-risk retry level. Tasks with an inspection risk label of warning level or a business importance level of important business are classified as Level 2 medium-risk retry level. Tasks with an inspection risk label of blocking level and a business importance level of core business are classified as Level 3 blocking level retry level. After the level classification is completed, the corresponding retry level identifier is written into the retry level field of the corresponding task record in the team list.

[0051] Before a task execution request is written to the queue list, the system extracts the idempotent key field carried in the request and retrieves the task records to be executed and running in the queue list, as well as the completed task records in the historical task archive table. If a valid task record with the same idempotent key is found, the write request for the current task is intercepted, and a duplicate task warning is returned. If no duplicate record is found, the task data write operation is completed. After the execution node successfully preempts the task, it extracts the idempotent key field of the current task a second time and compares it with the list of idempotent keys of tasks already executed by this node. After confirming that there are no duplicate execution records, the subsequent task execution process is started.

[0052] In this embodiment of the invention, during the risk assessment stage, the execution node calls the inspection rule set bound inside the corresponding task object, executes five types of sub-rules in sequence, and generates a standardized structured inspection report. At the same time, the inspection report is marked with four types of task risk labels: normal, prompt, warning, and blocking. The system extracts the anomaly detection result datasets output by each of the five types of inspection sub-rules, matches each detection result with the system's preset risk level determination rules one by one, and assigns a corresponding risk level to each detection result; it iterates through the risk levels corresponding to all detection items, selects the one with the highest level as the overall risk level of this task; it matches the determined overall risk level with the system's preset risk label mapping table, generates a task risk label corresponding to the risk level, and writes the label into the corresponding field of the structured inspection report.

[0053] The system synchronously launches a weighted gating algorithm, collecting three types of operating parameters in real time: server load percentage, task priority coding, and historical failure count. These parameters are then combined to dynamically assign inspection risk weights. 1. Mapping confidence weights to fields The algorithm assigns a value of 2, and the sum of the two weight values ​​is always 1. The algorithm combines the weight values, the inspection risk score, and the field mapping confidence score to calculate and output the comprehensive risk value of the task. The system extracts three parameters for each of the five types of inspection sub-rules: the number of abnormal items, the level of abnormality, and the scope of business impact. It matches these parameters with the system's preset risk score conversion table and outputs the corresponding single-rule risk score for each type of sub-rule. The system then takes the highest risk score among all sub-rules and converts it in conjunction with the total number of abnormal items to generate an overall task inspection risk score within the range of 0 to 1. The score is then written into the corresponding field of the structured inspection report.

[0054] Normalization is performed on the three types of operating parameters collected: server load percentage, task priority encoding, and historical failure count. For the server load percentage parameter, the real-time CPU utilization, memory utilization, and disk I / O utilization of the current execution node are read. The arithmetic mean of the three indicators is taken as the original load value. The original load value is divided by 100 to convert it into a load parameter score in the range of 0 to 1. For the task priority encoding parameter, the priority encoding value bound to the current task is read, and the priority encoding value is divided by the system's preset highest priority level to convert it into a priority parameter score in the range of 0 to 1. For the historical failure count parameter, the historical execution failure count and total execution count of the business type to which the current task belongs in the past 30 days are retrieved, and the ratio of the failure count to the total count is calculated as the original failure rate value, which is output as a historical failure parameter score in the range of 0 to 1. The three normalized parameter scores are substituted into the system's preset weight calculation formula, and a weighted summation operation is performed to obtain the inspection risk weight. The value of 1; field mapping confidence weight The value of 2 is the sum of the numerical value 1 and the inspection risk weight. The difference in values; Read the inspection risk score output from the current task inspection report, and the arithmetic mean of the confidence scores of all fields bound to the task as the average confidence score of the field mapping; then combine the inspection risk score with the inspection risk weight. Multiplying by 1 maps the field to a low-confidence risk value, i.e., 1-Cavg, and then mapping the field to a confidence weight. Multiply the two products together, sum the results, and obtain the overall risk value of the current task.

[0055] Formula for calculating the overall risk value of a task: ; Where: R represents the overall risk value of the task, ranging from 0 to 1, with a higher value indicating a higher overall risk level for the task; W1 represents the inspection risk weight, which is calculated by weighting three types of operating parameters after normalization: server load ratio, task priority code, and number of historical failures. R1 represents the overall inspection risk score for the task, ranging from 0 to 1. It is calculated based on the number of anomalies, anomaly level, and scope of business impact output by the five types of inspection sub-rules. W2 represents the field mapping confidence weight, which is always equal to the difference between 1 and the inspection risk weight W1; Gavg represents the average confidence score of the field mappings, ranging from 0 to 1, and is the arithmetic mean of the confidence scores of all field mappings bound to the current task; The risk score resulting from the 1-Cavg field mapping is higher when the confidence level is lower. The higher the inspection risk, the higher the overall risk; the lower the field mapping confidence, the higher the overall risk; the weighted gating algorithm can dynamically adjust the proportion of inspection risk and mapping risk in the overall risk according to server load, task priority, and historical failure count.

[0056] The system presets three fixed judgment thresholds: a first risk threshold, a first confidence threshold, and a second confidence threshold. It also defines four types of execution permission identifiers based on the comprehensive risk value, risk level label, and confidence value. When the blocking-level risk label or the comprehensive risk value is greater than the first risk threshold, it is identified as a trial operation only. The system first extracts the overall risk label of the task, verifies whether it is at the blocking level, and reads the overall risk value of the task and compares it with the first risk threshold. After completing the first round of judgment, for tasks that have not triggered the blocking level judgment, the system verifies whether the risk label is at the warning level, and reads the average confidence value of the field mapping, and compares it with the first confidence threshold and the second confidence threshold interval respectively. After completing the second round of judgment, for tasks that have not triggered the warning level judgment, check whether the risk label is normal or warning level, and at the same time read the average confidence value of the field mapping and compare it with the second confidence threshold. Finally, verify the tasks in the waiting execution stage, monitor the target system for new alarm data in real time, confirm whether any new blocking-level abnormal data has appeared during the waiting period, and complete the final judgment.

[0057] When a warning-level risk label is set and the confidence value is between the first confidence threshold and the second confidence threshold, it is marked as trial operation and requires secondary approval. When a normal or prompt-level risk label is set and the confidence value is greater than or equal to the second confidence threshold, it is marked as allowed to complete execution. When a task that has completed pre-verification adds blocking-level abnormal data during the waiting execution period, it is marked as rollback only. The rollback only mode is used to revoke temporary configurations and data generated in the pre-verification stage, resource pre-occupancy stage, or trial operation stage, and does not involve the revocation of formal business data that has not yet occurred. For tasks in the ready-to-execute phase, the system continuously calls the alarm check sub-rules and data consistency check sub-rules in the inspection rule set bound to the task according to a preset fixed monitoring cycle to perform incremental anomaly detection on the target system. After each detection, the newly added abnormal data is compared with the risk judgment threshold of the current task. When newly added abnormal data that meets the blocking level judgment criteria is detected, the execution permission flag of the task is immediately updated to rollback only, and the task is removed from the scheduling queue, triggering the corresponding rollback operation process.

[0058] Finally, the system stores the inspection report, dynamic weight parameters, comprehensive risk value, and execution permission identifier in a context partition dedicated to the task object.

[0059] In this embodiment of the invention, during the execution write-back phase, the execution node reads the execution license identifier configured inside the task object and matches the dedicated job execution mode. In the trial run mode only, the execution node only completes the original data reading and field format conversion operations of the source system and does not execute any data writing instructions. After the format conversion operation is completed, the system collects four types of running data: field mapping matching pass rate, number of data format exception entries, total amount of convertible data, and details of exception fields. It generates a test run mode detection report and writes it to the data docking business context partition corresponding to the task. It also clears all temporary cached source data and conversion data in memory and updates the task running status to execution success.

[0060] In the trial operation and secondary approval mode, the execution node sequentially completes the entire operation process of data reading, format conversion, trial operation report generation, multi-level approval verification, and formal data writing. In the full execution mode, the execution node can directly execute the complete data docking process of data reading, format conversion, data writing, and result data verification. In rollback-only mode, the execution node calls the built-in transaction rollback interface to atomically undo temporary permission configurations, interface pre-connection caches, resource pre-reservation flags, intermediate table data, trial run temporary cache data, and all temporary configuration changes generated during the pre-verification phase, resource pre-reservation phase, and trial run phase. Since no formal business data writing operation has been initiated during the waiting execution phase, the rollback operation does not involve the revocation of any formal business data. The execution node initiates a write transaction to the target system, and writes the transformed data to the corresponding data table of the target system in batches according to a preset batch granularity. After each batch of data is written, the primary key integrity and field format of the data in that batch are validated. After all data is written, a preset proportion of the written data is extracted and compared with the original data from the source end for consistency. If all validations pass, the write transaction is committed. If an anomaly is found, all write operations in the current transaction are rolled back, and the details of the abnormal data and the reason for the failure are recorded.

[0061] During the trial operation phase, the execution node only completes the source data reading and field format conversion operations in memory, without writing any persistent data to the target system. After the conversion is completed, four types of operation indicators are counted: field mapping matching accuracy, data format anomaly rate, primary key matching pass rate, and expected total amount of data to be written. Combined with the details of anomaly points, a structured trial operation report is generated. The system reads the risk level and business importance of the current task, matches the corresponding level of approver roles, and pushes the trial operation approval pending notification and report access link to the corresponding personnel's terminals; after the approver views the trial operation report, they submit the approval result and approval opinion; after the system receives the approval instruction, it initializes the target system write transaction and starts the formal data write process. After receiving the approval rejection instruction, the system clears all temporary trial data in memory, generates a rejection operation record and a rejection reason explanation, stores it in the data interface business context partition corresponding to the task, and terminates the execution process of this task.

[0062] The execution node first pulls the temporary data primary key list, configuration change record list, and interface call record list generated throughout the entire task process, arranged in reverse chronological order of the operation occurrence time; the rollback scope covers the temporary permission configuration and interface pre-connection cache generated in the pre-verification stage, the resource pre-occupancy marker generated in the queue scheduling stage, and the temporary memory cache, target end pre-written test data, and intermediate data table temporary entries generated in the trial operation stage, but does not include any written data to formal business data tables.

[0063] The execution node sequentially calls the transaction rollback interfaces of the source system and the target system to undo the corresponding configuration change operations, clear all data entries belonging to the current task in all intermediate data tables and temporary data tables, and release the pre-occupied computing, storage and interface resources; it verifies one by one that all configuration items, permission items and temporary data items have been restored to their original state before the task started, generates a rollback operation record containing rollback object details and resource release results, and writes it to the corresponding context partition of the task.

[0064] During task execution, the system continuously collects basic data including execution status and runtime, business data including anomaly point numbers, field mapping matching details, approval operation logs, and data layer data including total written data, data primary key number, and data snapshots before and after changes. The system synchronously writes the three types of operational data collected into the task object, the relational database team list, and the permission isolation database, and restricts the operational data that different roles can only access based on their corresponding permission codes.

[0065] The dedicated database for permission isolation is divided into independent physical storage partitions according to three preset operator roles, with each partition corresponding to a permission level. During the data writing process, the system writes the three types of running data—basic, business, and data layer—to the storage partitions corresponding to the permission levels based on the permission codes bound to the tasks, and binds a corresponding permission code identifier to each data record. When a data access request is received from an operator, the system extracts the permission code and role type bound to the requesting account, performs a consistency check with the permission code associated with the target data record, and allows the access request and returns the corresponding data content if the check passes. If the check fails, the access request is intercepted and a message indicating that access is denied is returned.

[0066] In this embodiment of the invention, during the evidence aggregation stage, after the task completes the workflow in the corresponding execution mode, the system integrates the hierarchically collected operation data, anomaly tracing and inspection reports, task execution logs, multi-level approval operation records, and access permission logs to generate business execution reports and audit reports. The system assigns the status of the task object to either successful or failed execution based on the final execution result of the task. When generating business execution reports, the system extracts basic task information, inspection and testing results, field mapping details, execution time, data write volume, anomaly locations, and handling results, assembles them into structured reports according to preset business templates, and stores them in the context partition corresponding to the task. When generating audit reports, the system extracts full-process operation logs, approval records, access permission records, data change snapshots, and task status flow records, arranges and assembles them in chronological order to form a full-link audit traceability report, and stores it in the system's dedicated audit database.

[0067] For tasks marked as failed, the system calculates the next available schedulable timestamp by combining the task's retry level parameters and maximum retry count parameters, and synchronously updates the schedulable timestamp field in the queue list so that the task returns to the corresponding queue partition to wait for scheduling. If the task still cannot be executed normally after the cumulative number of retries reaches the limit, the administrator will conduct manual verification and handling. The system will lock the task and generate differentiated alarm messages, which will be pushed to the terminals of ordinary maintenance personnel and senior administrators respectively. The system reads the current cumulative retries and retry level of a task, and matches the corresponding retry interval rules. For Level 1 low-risk tasks, a fixed interval duration parameter is retrieved, added to the current system time, and a timestamp for the next schedulable task is generated. For Level 2 medium-risk tasks, the corresponding gradient's interval duration parameter is matched based on the current cumulative retries, added to the current system time, and a timestamp for the next schedulable task is generated. For Level 3 blocking tasks, no schedulable timestamp is generated; the task status is directly marked as awaiting manual intervention, and the task is not entered into the retry scheduling queue.

[0068] The system periodically extracts the inspection anomaly dataset and field mapping execution dataset of all completed tasks, and synchronously updates the sub-rule parameters and weighted algorithm weight allocation parameters within the inspection rule set.

[0069] Extract the running dataset of all completed tasks according to the preset cycle, and split it into two independent data subsets: inspection anomaly dataset and field mapping execution dataset; for the inspection rule set parameters, respectively count the anomaly detection matching data of five types of sub-rules: alarm check, indicator time window check, automated task check, data consistency check, and anomaly tracing, and calculate the anomaly detection accuracy and false alarm rate of each type of sub-rule. Based on the calculated accuracy and false alarm rates, the three types of parameters—judgment threshold, detection window length, and anomaly level classification boundary—are adjusted accordingly for each type of sub-rule. For the weighted algorithm weight allocation parameters, the matching accuracy of field mapping in each business scenario is statistically analyzed according to the business tag dimension. Based on the matching accuracy value corresponding to the business tag, adjust the weight coefficient values ​​of the five operation factors under that business tag, and keep the sum of the five weight coefficients at 1 after adjustment; write all the adjusted parameters into the system's built-in parameter storage unit, directly overwriting the original historical parameter values, and complete the parameter iterative update.

[0070] This invention also provides an integrated orchestration system for inspection and data integration. Based on the above method, it is characterized by including a task modeling module, a rule approval module, a queue scheduling module, a risk assessment module, and an execution archiving module. The task modeling module configures a four-layer integrated task object model and a permission configuration unit. It receives business configuration parameters from external operation and maintenance terminals and completes task object initialization creation, global unique identity allocation, operator permission binding, and context partition configuration. The rule approval module has a built-in inspection rule template library, weighted algorithm operation unit and status manager. It calls preset templates to generate five types of inspection sub-rules, collects business system operation data and completes cross-system field mapping matching, completes the mapping relationship level division according to confidence parameters, and reads the associated attributes of task objects to complete the hierarchical approval operation and the pre-verification operation of connectivity, permission and resource levels in sequence. The queue scheduling module configures a partitioned relational database queue list, integrates a transaction lock control unit, an idempotent deduplication unit, and a hierarchical retry unit, receives execution requests from the rule approval module, performs heterogeneous task partitioning and storage according to task type, realizes task preemption of distributed execution nodes through the database row-level locking mechanism, and provides execution timeout unlocking, duplicate task interception, and hierarchical retry scheduling operations. The risk assessment module, which embeds a weighted gating algorithm, retrieves the inspection rule set bound to the task object to collect business system operation data, dynamically allocates the weight parameters corresponding to the inspection risk and field mapping confidence level, calculates the comprehensive risk value, and marks the corresponding execution permission identifier with reference to the first risk threshold, the first confidence threshold, and the second confidence threshold preset by the system. The execution archiving module connects the source business system and the target business system bidirectionally. It calls the corresponding job execution mode according to the execution permission identifier, collects running data in layers and implements permission-isolated storage, integrates running data to generate structured reports, synchronously updates task running status fields, performs secondary scheduling for failed tasks, and iteratively updates the internal configuration parameters of inspection rules and field mapping algorithms.

[0071] This invention also provides a computer-readable storage medium storing a computer-executable program. When executed by a processor, the computer-executable program implements an integrated orchestration process for inspection and data interfacing. Build and initialize the task object model, assign identity and permission parameters, and configure three independent context partitions; Generate a set of inspection rules, complete cross-system field mapping through a weighted algorithm, and perform hierarchical approval and pre-verification based on task attributes; For heterogeneous tasks, partition storage is used, and distributed scheduling is completed based on database transactions and row-level locks. Parameters for timeout unlocking, idempotent deduplication, and hierarchical retry are configured. Collect system operation data, calculate the comprehensive risk value of the task using a weighted gating algorithm, and assign execution permission identifiers; Match the corresponding job mode to complete data processing, and isolate and store the entire process data; Aggregate runtime data to generate reports, update task status and handle abnormal tasks, and iteratively update built-in inspection rules and field mapping algorithm parameters.

[0072] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus.

[0073] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. An integrated orchestration method for inspection and data interoperability, characterized in that, The specific steps include the following: A four-layer integrated task object model is constructed from top to bottom, consisting of a basic information layer, an access control layer, an inspection business layer, and a data docking business layer. Each task is assigned a globally unique global task ID, an idempotent key, and an access code. Within the task object, three independent context partitions are divided: inspection business, data docking business, and global common. The task initialization configuration is then completed. Generate an inspection rule set containing multiple types of inspection sub-rules, use a multi-factor weighted algorithm to complete the cross-system field mapping relationship calculation, divide the field mapping relationship level according to the confidence level of the calculation output, and write the inspection rule set and the data corresponding to the field mapping relationship into the task object; The system calls the inspection rule set bound to the task to perform inspection and detection on the target system, outputs the inspection risk score and marks the risk label, combines the inspection risk score with the field mapping low confidence risk value, dynamically allocates the weights of the two types of risk indicators through the weight gating algorithm and calculates the comprehensive risk value of the task, and marks the execution permission identifier by combining the preset first risk threshold, first confidence threshold, second confidence threshold and the risk label. The execution license identifier is read and matched with the job mode, and the corresponding data processing operation is executed. The job modes include a trial operation mode that only completes data reading and format conversion, a mode that executes formal data writing after secondary approval, a mode that fully executes the entire data docking process, and a rollback mode that atomically cancels all temporary changes. The isolation storage and access control of the entire process operation data are realized according to the permission code constraints.

2. The integrated orchestration method for inspection and data interoperability according to claim 1, characterized in that, The four-layer integrated task object model has seven types of structured fields, including basic information, business rules, execution strategies, and approval permissions. When a task is initialized, its initial state is set to draft. The three context partitions store inspection business data, data docking business data, and global public shared data, respectively. The inspection business partition and the data docking business partition do not directly establish data read / write channels. Cross-partition shared data is transferred and stored through the global public partition. When reading data, the permission code and data validity period are verified.

3. The integrated orchestration method for inspection and data interoperability according to claim 2, characterized in that, The inspection rule set includes five sub-rules: alarm inspection, indicator time window inspection, automated task inspection, data consistency inspection, and anomaly tracing. The anomaly tracing sub-rules are linked with the other four sub-rules to collect anomaly-related information. The multi-factor weighted algorithm includes five calculation factors: field name semantics, data type, value range matching degree, historical mapping success rate, and business importance. The factor weights are dynamically adjusted according to the field business labels. The output confidence score, combined with the preset first confidence score threshold and second confidence score threshold, divides the field mapping relationship into three levels: automatic approval, manual review, and rejection.

4. The integrated orchestration method for inspection and data interoperability according to claim 3, characterized in that, After the field mapping relationship level is determined, hierarchical approval is performed based on the field mapping relationship level and the inspection risk score: tasks with a field mapping relationship level of automatic approval and no high-risk data are automatically approved; tasks with a field mapping relationship level of manual review or containing medium-risk data are pushed to department managers for approval; and tasks with a risk label of blocking level and bound to core business are pushed to senior management for approval. After approval, four pre-verification checks are performed in sequence: connectivity detection, operator permission verification, server hardware resource detection, and task resource relationship calculation. After all checks are passed, the status of the task is updated to ready.

5. The integrated orchestration method for inspection and data interoperability according to claim 4, characterized in that, The tasks in the ready state are written to a partitioned relational database queue list. The queue list is divided into inspection task partitions and data docking task partitions according to task type. Distributed execution nodes preempt the tasks through database transactions and exclusive row-level locks. After successful preemption, the lock time and execution node identifier are updated. After the task times out, the row-level lock is automatically released and the scheduling resources are released. Using the idempotent key as a global deduplication identifier, a three-level retry mechanism is established according to risk level. Level 1, low risk, the task is retried a maximum of 2 times with a fixed retry interval. Level 2, medium risk, the task is retried a maximum of 5 times with a gradient increasing retry interval. Level 3, blocking, the task has its automatic retry function disabled.

6. The integrated orchestration method for inspection and data interoperability according to claim 5, characterized in that, The weighted gating algorithm collects three types of operating parameters: server load, task priority, and historical failure count. It dynamically assigns inspection risk weights and low-confidence risk weights to fields. The sum of the two weight values ​​is 1. Combined with the corresponding scores, the comprehensive risk value of the task is calculated. There are four types of execution permission identifiers: if the risk label is blocked or the comprehensive risk value of the task exceeds the first risk threshold, it is a trial run only; if the risk label is warning and the confidence value is between the first and second confidence thresholds, it is a trial run with secondary approval; if the risk label is normal or prompt and the confidence value is not lower than the second confidence threshold, it is allowed to complete the execution. The newly added blocking-level abnormal data during the task waiting execution period is rolled back only. The rollback only mode is used to revoke temporary permission configurations, interface pre-connection caches, resource pre-occupancy flags, intermediate table data and temporary configuration changes generated in the pre-verification stage, resource pre-occupancy stage or trial operation stage, without revoking the data written by formal business.

7. The integrated orchestration method for inspection and data interoperability according to claim 6, characterized in that, In the trial operation mode, only data reading and format conversion are completed, and no data writing operation is performed; in the trial operation and secondary approval mode, a trial operation report is generated, and formal data writing is performed after multi-level approval. The system allows direct execution of the entire data integration process in full execution mode; in the rollback-only mode, the transaction interface is called to atomically undo the previous temporary configurations, cached data, and all temporary changes; the running data is collected in three layers: basic, business, and data layers, and synchronously written to the task object, the team list, and the permission-isolated database.

8. The integrated orchestration method for inspection and data interoperability according to claim 7, characterized in that, After the task is completed, the entire process operation data, inspection reports and log records are integrated to generate two types of reports: business reports and audit reports. The status of the task is marked as successful or failed according to the execution result. For failed tasks, the next schedulable timestamp is calculated based on the corresponding retry level, and the task is returned to the corresponding queue partition to wait for scheduling. If the retry limit is reached, the task is locked and an alarm is pushed. Data of completed tasks is extracted periodically, and the parameters of the inspection rule set and the weight parameters of the multi-factor weighted algorithm are updated synchronously and iteratively.

9. An integrated orchestration system for inspection and data interoperability, based on the method described in any one of claims 1-8, characterized in that, It includes a task modeling module, a rule generation module, a risk assessment module, an execution control module, an approval verification module, a queue scheduling module, and an archiving iteration module; The task modeling module configures the four-layer integrated task object model and permission configuration unit to receive business configuration parameters, complete task initialization creation, global unique identifier allocation, operator permission binding, and configuration of the three independent context partitions. The rule generation module has a built-in inspection rule template library and the multi-factor weighted algorithm operation unit, which are used to generate the inspection rule set composed of multiple types of inspection sub-rules, complete the cross-system field mapping relationship matching, and divide the field mapping relationship level according to the confidence level. The risk assessment module embeds the weighted gating algorithm, which is used to retrieve the target system operation data collected by the inspection rule set, combine the inspection risk score with the low confidence risk value of the field mapping to calculate the comprehensive risk value of the task, and mark the execution permission identifier with reference to the preset first risk threshold, first confidence threshold and second confidence threshold. The execution control module is bidirectionally connected to the source and target business systems. It is used to match the corresponding job mode according to the execution license identifier, collect operation data in layers, and realize isolated storage and access control according to the permission code. The approval and verification module has a built-in status manager, which is used to perform hierarchical approval and four pre-verifications based on task attributes, and to manage the status flow of the task throughout its entire lifecycle. The queue scheduling module is configured with a partitioned database queue list and a transaction row-level lock control unit to realize heterogeneous task partition storage and distributed preemptive scheduling, and is equipped with the idempotent key deduplication and hierarchical retry mechanism. The archiving iteration module is used to integrate the full-process operation data to generate structured reports, handle abnormal tasks, and iteratively update the parameters of the inspection rule set and the multi-factor weighting algorithm.

10. A computer-readable storage medium storing a computer-executable program, wherein when executed by a processor, the computer-executable program performs the following integrated orchestration steps for inspection and data interfacing: Construct a four-layer integrated task object model from top to bottom, consisting of a basic information layer, an access control layer, an inspection business layer, and a data docking business layer. Assign a globally unique global task ID, an idempotent key, and an access code. Configure three independent context partitions and complete task initialization. Generate a set of inspection rules, and use a multi-factor weighted algorithm to perform cross-system field mapping relationship calculations and classify the field mapping relationship levels. By combining the inspection risk score with the low-confidence risk value mapped to the field, the comprehensive risk value of the task is calculated and the execution permission mark is assigned through a weighted gating algorithm. Data processing is completed by matching the corresponding job mode according to the execution license identifier, and the operation data is isolated, stored and accessed according to the permission code.

Images