Monitoring data linkage method between code-free applications

Abstract

The invention provides a monitoring data linkage method between code-free applications, automatic identification and mapping of data structures between the applications are realized by constructing a metadata description model and a field similarity function, and data change detection is performed by adopting an improved locality sensitive hash algorithm. Through a distributed transaction control mechanism, data consistency in cross-application data linkage is ensured, and a multi-stage bloom filter and a distributed transaction feature evaluation model are designed. Furthermore, data synchronization is realized by utilizing a synchronous vector field, and data synchronization batch processing and concurrency adjustment are optimized in combination with a Gaussian divergence theorem. Besides, the invention provides abnormal recovery path optimization based on the minimum action quantity, the reliability of data linkage is ensured, and a dynamic data security protection mechanism is constructed through a wave function, a diffusion equation and nonlinear mapping.

Description

Technical Field

[0001] The present invention belongs to the technical field of codeless applications, and in particular relates to a monitoring data linkage method between codeless applications. Background Art

[0002] As enterprises' digital transformation deepens, building business application systems through codeless development platforms has become a mainstream trend. Such applications have the characteristics of short development cycles and easy maintenance, and are widely used in business scenarios such as order management, inventory management, customer relationship management, human resources management, and financial management. In actual applications, these codeless applications often need to exchange data and link with each other to support cross-departmental business collaboration.

[0003] In the existing technology, the following four solutions are mainly used to realize data linkage between applications:

[0004] 1. Data synchronization method based on message queue

[0005] This solution uses message middleware to transmit data change information, which has the following technical defects:

[0006] A) Message reliability issues:

[0007] The message persistence mechanism is imperfect, which can easily lead to message loss when the system crashes; the message retry mechanism is simple and cannot effectively handle duplicate messages; the strict order of messages cannot be guaranteed, resulting in chaotic data status; there is a lack of message confirmation and receipt mechanism, making it difficult to ensure end-to-end data reliability.

[0008] B). Performance bottleneck:

[0009] System performance significantly degrades when messages accumulate; message processing latency is high, especially in high-concurrency scenarios; message routing and forwarding overhead is high and system resource utilization is low.

[0010] C). Maintenance cost:

[0011] The configuration of message topics and queues is complex; the message format needs to be changed frequently; error message processing and replay are difficult; and the monitoring and alarm mechanisms are imperfect.

[0012] 2. Synchronization solution based on database triggers

[0013] This solution captures data changes through database triggers, which has the following problems:

[0014] A). Performance impact:

[0015] Trigger execution seriously affects database performance; trigger chains cause database response delays; deadlocks are easily formed in concurrent scenarios; and database resources are too high.

[0016] C). Maintenance difficulty:

[0017] Trigger logic is difficult to version control; changing one place requires simultaneous changes in multiple places; cross-database trigger synchronization is complex to implement; and trigger debugging and troubleshooting are difficult.

[0018] D). Functional limitations:

[0019] Can only handle changes at the database level; business logic is difficult to implement in triggers; lacks transaction support mechanism; exception handling capabilities are limited.

[0020] 3.Data connection solution based on API interface

[0021] This solution obtains data by periodically calling the application interface, which has the following defects:

[0022] A). Real-time issues:

[0023] The interface call delay is high; there is a time difference in data synchronization; the polling interval is difficult to control; and the response to sudden data changes is not timely.

[0024] B). Interface management problems:

[0025] Interface version management is complex; interface changes require synchronous modification of the caller; interface documentation is difficult to maintain; and interface testing costs are high.

[0026] C). Performance constraints:

[0027] The interface has limited concurrent capabilities; batch data processing efficiency is low; network bandwidth consumption is large; and server load is high.

[0028] D). Safety hazards:

[0029] The interface authentication mechanism is simple; there is a lack of access control granularity; data transmission is not encrypted; and security audits are incomplete.

[0030] 4. Linkage solutions based on data lineage

[0031] This solution tracks and manages data flow by building a data lineage relationship graph, but has the following technical defects:

[0032] A). Limitations of establishing blood relationship:

[0033] It can only track explicit data transfer paths and cannot identify implicit data associations; the construction of the lineage relationship graph relies on predefined metadata and lacks support for dynamically changing codeless applications; the lineage tracking of data conversion and processing is incomplete, especially in complex business rule scenarios; cross-system lineage relationships are difficult to accurately establish and maintain.

[0034] B). Real-time issues:

[0035] There is a delay in updating the blood relationship graph, which cannot reflect data changes in real time; data synchronization based on blood relationships adopts batch processing mode, which lacks real-time performance; when the blood relationship link is too long, the end-to-end delay increases significantly; circular dependency detection is time-consuming and lacks accuracy.

[0036] C). Performance bottleneck:

[0037] The construction and maintenance of the bloodline graph is expensive, especially in large-scale application scenarios; the performance of bloodline query drops sharply as the complexity of the graph increases; full bloodline analysis leads to high consumption of system resources; there is a lack of incremental update mechanism, and each update requires rebuilding the bloodline relationship.

[0038] D). Application adaptation issues:

[0039] Traditional lineage technology is mainly aimed at data warehouse scenarios and has insufficient support for codeless application features; it lacks the ability to recognize components and data flows unique to codeless platforms; the expression of lineage relationships is too technical, which is not conducive to business personnel's understanding and use; and it cannot automatically adapt to the data structures and interface specifications of different codeless platforms.

[0040] E). Operation and maintenance complexity:

[0041] The maintenance of blood relationship requires the participation of professional technicians; the problem of broken blood relationship is difficult to repair automatically; there is a lack of verification mechanism for the validity of blood relationship; the problem location and fault recovery process are complicated.

[0042] In the code-free application scenario, the above four technical solutions all have obvious shortcomings, which are mainly reflected in the following aspects:

[0043] A). Data structure recognition is difficult:

[0044] The data structure of codeless applications lacks unified standards, the structure changes frequently without a clear notification mechanism, field mapping relationships require manual maintenance, and support for complex data type conversion is limited.

[0045] B). Insufficient data consistency guarantee:

[0046] There is a lack of cross-application transaction support mechanisms, data conflict resolution is simple, data recovery under abnormal circumstances is difficult, and multi-copy data synchronization is inconsistent.

[0047] C). Poor real-time performance:

[0048] There are delays in change detection, the synchronization process consumes a lot of resources, the concurrent processing capability is limited, and the efficiency of large-scale data synchronization is low.

[0049] D). Limited scalability support:

[0050] System performance cannot scale with nodes, the cost of adding new applications is high, configuration changes have a wide impact, and there is a lack of dynamic load balancing capabilities.

[0051] E). High operation and maintenance costs:

[0052] The configuration process is cumbersome, problem location is difficult, there is a lack of automated recovery mechanisms, and monitoring and alarming are incomplete.

[0053] Therefore, there is an urgent need for a data monitoring linkage method specifically for codeless applications. This method needs to overcome the limitations of the above-mentioned technical solutions and provide a more intelligent, efficient and reliable solution. Summary of the invention

[0054] The present invention provides a monitoring data linkage method between codeless applications, aiming to achieve automatic identification of data structure, cross-application consistency assurance, real-time change monitoring and multi-level security protection, thereby improving the efficiency and consistency of data flow between codeless applications. The specific method steps are as follows:

[0055] S1. Data structure identification

[0056] In the process of data linkage between codeless applications, a metadata description model is first established to build a unified representation of application identifiers, data table sets, field sets and association relationships; field attributes are described by multidimensional feature vectors, including field entropy, variance, skewness, kurtosis and uniqueness indicators, to achieve quantification of field distribution characteristics; a similarity function is constructed based on field name similarity, type compatibility and value distribution similarity to achieve intelligent mapping of source application structure to target application structure, ensuring the accuracy and adaptability of data structure identification.

[0057] S2. Data change monitoring

[0058] An improved local sensitive hashing algorithm is used to realize real-time detection of data changes. The algorithm realizes the rapid extraction of data features by constructing a family of hash functions containing random projection vectors and random offsets. An adaptive bucket width mechanism is introduced to dynamically adjust the detection sensitivity according to the frequency of data changes. A combined hash function is designed to generate data fingerprints, and redundant detections are removed through multi-level Bloom filter optimization to improve the efficiency and accuracy of change monitoring.

[0059] S3. Distributed transaction control

[0060] Transaction characteristics are characterized based on multi-dimensional feature space, including complexity, dependency, resource consumption and priority. The state transition operator is introduced to measure transaction risk, and the transaction status is evaluated through the risk accumulation effect; the isolation level decision mechanism based on energy function is adopted to achieve adaptive adjustment of isolation strength; the interaction between transactions is described by defining conflict potential, and the conflict resolution strategy that minimizes potential energy is designed; in the submission protocol, the surface integral of the transaction impact domain and the system stability analysis are combined to make the final decision.

[0061] S4. Data synchronization execution

[0062] Construct a synchronization vector field that describes the data flow characteristics, and determine the optimal batch processing parameters through the Gaussian divergence theorem; use the differential operator to capture the curl characteristics of data changes to achieve accurate incremental synchronization; solve the Laplace equation to obtain the optimal concurrency; based on the dual constraints of the data flow vector position and the synchronization potential function, ensure the efficiency and reliability of the synchronization process.

[0063] S5. Exception handling and recovery

[0064] The entropy function is used to describe the health of the system and quantify the order of the system state. The fault recovery strategy is designed based on the principle of minimum action, and the optimal recovery path is obtained by solving the Lagrange equation. The compensation force function is introduced to handle transaction failures and ensure data consistency.

[0065] S6. Security protection mechanism

[0066] The wave function is used to describe the protection strength of sensitive data, and the protection measures can be adjusted dynamically. The diffusion equation model is used to build a dynamic evolution mechanism for the access control matrix to ensure the accuracy and real-time performance of permission management. Nonlinear mapping is designed to achieve data desensitization, and the mapping parameters are adaptively adjusted according to the sensitivity level to maintain data availability while protecting data security.

[0067] Beneficial effects of the present invention:

[0068] 1. Optimization of automatic identification and mapping of data structures: By constructing a metadata description model and a field similarity function, the present invention effectively realizes the automatic parsing and mapping of data table structures between codeless applications, improves the speed and accuracy of data integration, and reduces manual errors and operating costs in the data processing process.

[0069] 2. Real-time data change monitoring: The improved local sensitive hashing algorithm is used for data change detection, which can quickly respond to data changes and realize real-time data monitoring. The adaptive bucket width and combined hash function are used to improve the sensitivity and efficiency of change detection and reduce the delay and inconsistency risks in data linkage.

[0070] 3. Efficient distributed transaction control: The designed distributed transaction control method, including transaction state management and dynamic adjustment of isolation levels, ensures cross-application data consistency. Through transaction feature evaluation and conflict resolution strategies, it enhances the stability and security of transaction processing and supports data linkage in complex multi-application environments.

[0071] 4. Optimized data synchronization mechanism: The data synchronization mechanism constructed using the vector field, as well as the data block batch processing parameters and concurrency adjustment optimized by the Gaussian divergence theorem, effectively improves the efficiency of data synchronization. The potential energy function of the synchronization vector field realizes dynamic control of data priority, enables rapid synchronization of important data, and optimizes the stability and response time of data flow.

[0072] 5. Exception recovery based on minimum action: The exception recovery path optimized by the minimum action principle ensures that the resource consumption of the recovery process is minimized, which improves the efficiency and accuracy of data recovery. The design of compensation transactions further ensures the consistency of the state after data recovery and enhances the reliability of the system.

[0073] 6. Dynamic security protection mechanism: The dynamic security protection mechanism designed through wave function, diffusion equation and nonlinear mapping method can flexibly adjust the protection strength and access control according to the sensitivity level of the data and the transmission scenario, effectively protecting the security and privacy of the data. BRIEF DESCRIPTION OF THE DRAWINGS

[0074] Figure 1 A flow chart of the steps of the monitoring data linkage method of the present invention is shown. DETAILED DESCRIPTION

[0075] Exemplary embodiments of the present invention are described in detail below with reference to the accompanying drawings.

[0076] Combination Figure 1 , provides a monitoring data linkage method between codeless applications, including the following steps:

[0077] Step 1. Data structure identification

[0078] This step builds a metadata description model and field similarity function to achieve automatic identification of data structures between codeless applications, including data table structure parsing and mapping and field feature extraction. The specific contents are as follows:

[0079] Step 1.1 Establish metadata description model

[0080] First, construct a metadata description model M, which is defined as:

[0081] M=(A,T,F,R)

[0082] in:

[0083] A is the application identifier, which is a unique string used to identify different applications;

[0084] T is a set of data tables, expressed as T = {t1, t2, ..., t n}, where t i A single data table, representing the structure and content of each data table in the application;

[0085] F is a field set, expressed as F = {f1,f2,...,f m}, where f i It is a specific field definition, indicating the specific attributes of each field in the data table;

[0086] R is a set of association relationships, expressed as R = {r1, r2, ..., r k}, where r i It is the association relationship between tables, used to describe the association rules between data tables.

[0087] Step 1.2 Field similarity function construction

[0088] In order to achieve accurate mapping of the data table structure, the field similarity function sim(f1,f2) is defined to calculate the similarity between fields in different applications. The expression of the similarity function is as follows:

[0089] sim(f1,f2)=ω1×sim name (f1,f2)+ω2×sim type (f1,f2)+ω3×sim dist (f1,f2)

[0090] in:

[0091] ω1, ω2, and ω3 are weight coefficients, and their sum is 1, which are used to adjust the influence of each similarity;

[0092] sim name (f1,f2) is the name similarity of fields f1 and f2;

[0093] sim type (f1, f2) is the type similarity of fields f1 and f2;

[0094] sim dist (f1, f2) is the value distribution similarity of fields f1 and f2.

[0095] Specifically, the field name similarity calculation method is as follows:

[0096]

[0097] In the formula, is the word vector representation of fields f1 and f2; Indicates the cosine similarity of two vectors. The value range is [-1,1]. The closer the value is to 1, the more similar the field names are.

[0098] Specifically, the field type similarity calculation method is as follows:

[0099] When field f1 is equal to field f2, sim type (f1,f2)=1;

[0100] When the types of field f1 and field f2 can be directly converted without information loss, that is, lossless conversion, the type similarity is 0.8;

[0101] When the types of field f1 and field f2 are converted, some information may be lost, that is, lossy conversion, and the type similarity is 0.5;

[0102] When there is no reasonable conversion relationship between the types of field f1 and field f2, that is, they are incompatible, the type similarity is 0.

[0103] Furthermore, the conversion rules between types of different fields are shown in the following table:

[0104]

[0105]

[0106] In the table, INTEGER, FLOAT, STRING, BOOLEAN, and DATE are common data types, namely integer, floating-point, character, Boolean, and date.

[0107] Specifically, the field value distribution similarity calculation method is as follows:

[0108]

[0109] in:

[0110] It is the characteristic vector of the field value distribution, including the entropy, variance, skewness, kurtosis and uniqueness index of the field:

[0111] V f =[entropy, variance, skewness, kurtosis, uniqueness]

[0112] In the formula, the calculation of entropy, variance, skewness, and kurtosis are standard methods recognized by statistics and will not be introduced in detail. The uniqueness is calculated as the ratio of the number of unique values ​​in the field to the total number of values ​​in the field;

[0113] Represents the cosine similarity of two feature vectors.

[0114] Step 1.3 Structural analysis and mapping

[0115] After completing the metadata description model, the structure S is applied to the source through the field similarity function s With the target application structure S t The mapping relationship is expressed as:

[0116]

[0117] in:

[0118] The fields to apply to the source;

[0119] The fields to apply to the target;

[0120] Representation field and The similarity between them is calculated as described in step 1.2;

[0121] θ is the similarity threshold, ranging from [0,1], which is used to control the similarity screening criteria.

[0122] Step 2. Data change detection

[0123] After completing the initial data structure matching, the improved Locality Sensitive Hash (LSH) algorithm is used for change detection. By constructing a hash function family and adaptive bucket width, it can quickly detect field value changes.

[0124] Step 2.1 Construct a locality sensitive hashing (LSH) function family

[0125] In order to efficiently detect changes in field data, we first construct a hash function family {h a,b (v)}, where each hash function is defined as:

[0126] h a,b (v) = [(a × v + b) / w]

[0127] in:

[0128] v is the characteristic vector of the field value distribution, calculated as described in step 1.2;

[0129] a is a random projection vector, which follows the normal distribution N(0,1);

[0130] b is a random offset, which follows a uniform distribution U(0,w);

[0131] w is the bucket width parameter, which is used to control the sensitivity of hash detection. The smaller the value, the higher the sensitivity.

[0132] Step 2.2 Adaptive barrel width adjustment

[0133] In order to adapt to the change frequency and sensitivity requirements of different data, an adaptive bucket width mechanism is introduced to dynamically adjust the bucket width w according to the amount of data change. adpt The calculation formula is:

[0134] w adpt =w base ×(1+λ×Δ c / Δ t )

[0135] in:

[0136] w base is the base barrel width;

[0137] λ is the adaptive coefficient, ranging from [0.1, 2], which is used to adjust the sensitivity of change detection;

[0138] Δ c is the amount of change per unit time;

[0139] Δ t is the time window size.

[0140] Step 2.3 Generate a combined hash function

[0141] The constructed hash function family is combined to generate a combined hash function g(v) for unique fingerprint identification of data blocks. The combined hash function is defined as:

[0142] g(v)=(h1(v),h2(v),...,h k (v))

[0143] Here, k is the number of hash functions, and the combined hash functions are used to improve the accuracy of change detection.

[0144] Step 2.4 Data block fingerprint calculation

[0145] After generating the combined hash function, the fingerprint of the data block is calculated based on the combined hash result. The data block fingerprint F(B) is calculated using the following formula:

[0146]

[0147] in:

[0148] B is the data block;

[0149] v i is the i-th record in data block B;

[0150] H() is the hash function SHA-256, which is used to generate a unique hash value for each record;

[0151] XOR operator, used to combine the hash values ​​of all records.

[0152] Step 2.5 Multi-level Bloom filter optimization

[0153] To further improve the accuracy and efficiency of change detection, a multi-level Bloom filter is used to optimize the detection results. i The structure is as follows:

[0154] BF i ={h i1 ,h i2 ,...,h ik}

[0155] where h ij is the hash function of the Bloom filter, and the calculation formula for the misjudgment rate of the Bloom filter is:

[0156] pfp=(1-e -k×n / m ) k

[0157] Among them, k is the number of hash functions; n is the number of elements in the Bloom filter; m is the bit array size of the Bloom filter.

[0158] Step 3. Distributed transaction control

[0159] In the monitoring data linkage between codeless applications, the real-time data linkage of multiple applications requires reliable transaction control to ensure data consistency. To this end, a distributed transaction control method suitable for codeless applications is designed, which includes transaction state management, feature evaluation, risk control and conflict handling to ensure the accuracy and integrity of data linkage between different applications.

[0160] Step 3.1 Define the transaction state space

[0161] In order to support cross-application data consistency in a codeless environment, the transaction state space is defined to ensure that the execution phase of each transaction can be effectively managed in operations such as data update, synchronization, and conflict. The transaction state set is defined as follows:

[0162] Initial state: The transaction begins execution, which may involve data reading or writing by multiple codeless applications.

[0163] Prepared state: The transaction completes data preparation and enters the waiting for submission stage.

[0164] Commit status: The transaction is successfully committed, ensuring that the data linkage between all codeless applications is successfully synchronized.

[0165] Terminated state: The transaction is aborted and the data is restored to the initial state to prevent data inconsistency between codeless applications.

[0166] Step 3.2 Transaction feature construction

[0167] First, we construct a transaction feature space to quantify the key characteristics of data linkage between codeless applications, providing a basis for subsequent risk assessment and isolation level adjustment. The transaction feature space is defined as follows:

[0168] Ψ(T)={v c ,φ d ,φ r ,φ p}

[0169] in:

[0170] φ c It is a complexity feature used to quantify the complexity of transaction operations. The calculation formula is:

[0171]

[0172] In the formula, op i is the number of operations of type i, i∈{insert, update, delete, query}; ω i is the operation weight coefficient, and the sum is 1; n is the total number of operation types, and the value is 4, namely insert, update, delete and query;

[0173] φ d The dependency level feature describes the strength of data dependency between codeless applications, especially the relationship between data tables and fields. The calculation formula is:

[0174] φ d =1-e -λ×|E(G)| / |V(G)|

[0175] Where G is a transaction dependency graph (a read-write dependency graph construction method commonly used in the database field is used, which is a well-known technique in the art and will not be described in detail); λ is a dependency decay coefficient, ranging from (0 to 1); |E(G)| is the number of edges in the dependency graph G, indicating the number of transaction dependencies; |V(G)| is the number of nodes in the dependency graph G, indicating the number of transactions;

[0176] φ r It is a resource consumption feature that represents the occupation of system resources (CPU, memory, and I / O) by transactions during the linkage process. The calculation formula is:

[0177]

[0178] In the formula, u cpu 、u mem 、u io Respectively represent the normalized utilization of CPU, memory, and I / O;

[0179] φ p It is a priority feature used to determine important cross-application data linkage operations. The calculation formula is:

[0180]

[0181] Where p is the service priority value; p0 is the priority threshold.

[0182] Step 3.3 State Migration and Risk Assessment

[0183] During the data linkage process between codeless applications, the system needs to continuously assess the risks of transactions to avoid the impact of inconsistent data on linkage. Risk assessment can be completed by following the steps below:

[0184] The risk change rate, i.e., instantaneous risk density η(t), is calculated by the time partial derivative and divergence of the feature space:

[0185]

[0186] in:

[0187] is the partial derivative of the transaction feature vector Ψ with respect to time t, indicating the rate at which the feature space changes over time;

[0188] Represents the divergence of the transaction feature vector Ψ.

[0189] The cumulative risk is calculated by time integration, taking into account the attenuation effect of historical risk, that is, the risk accumulation effect R(t):

[0190] R(t)=∫0 t η(τ)×e -δ(t-τ) dτ

[0191] in:

[0192] τ is a time variable, representing any moment between time 0 and the current time t;

[0193] η(τ) is the instantaneous risk density at time τ;

[0194] δ is the time decay factor;

[0195] dτ represents a small increment of the integral variable τ.

[0196] Step 3.4 Isolation Level Decision

[0197] The isolation level of data linkage between codeless applications is adjusted based on transaction energy calculation to balance performance and data consistency.

[0198] Define the state energy function E(s) and combine the nonlinear impact of transaction characteristics to evaluate the current isolation requirements. The formula is as follows:

[0199]

[0200] in:

[0201] Ψ i is the i-th characteristic component of the transaction;

[0202] κ is the balance factor, which is used to control the influence of the eigenvector gradient in the energy function;

[0203] is the square of the eigenvector gradient, which is used to measure the rate of change of the feature space.

[0204] Select the appropriate isolation level L(s) based on the state energy value to ensure data consistency between codeless applications. There are three levels:

[0205] When the energy value E(s) is greater than a higher threshold θ h , defined as a high isolation level, under which transactions are executed serially (or equivalently serially) to avoid any concurrent data conflicts.

[0206] When the energy value E(s) is between two thresholds θ l ,θ h The medium isolation level is defined as between 1 and 2. Under this isolation level, a transaction can read the same data multiple times and ensure data consistency, but some concurrent updates are allowed. This is suitable for situations where data consistency requirements are moderate.

[0207] When the energy value E(s) is less than the lower threshold θ l , defined as a low isolation level, allows more concurrent interactions while ensuring that the data read each time is "committed" data.

[0208] Step 3.5 Conflict Detection and Resolution Strategy

[0209] In the multi-data source linkage environment of codeless applications, transaction conflicts will cause data inconsistency. Therefore, a conflict detection and resolution strategy is defined to ensure the reliability of the linkage process. The conflict potential energy function is defined as follows:

[0210] V(T i ,T j )=K / d(T i ,T j ) 2×exp(-‖Ψ i -Ψ j ‖ 2 / 2σ 2 )

[0211] in:

[0212] V(T i ,T j ) is transaction T i , T j The conflict potential between the two transactions. The larger the conflict potential value, the more serious the conflict between the two transactions, and a stricter conflict control strategy is needed.

[0213] K is the conflict coefficient, which is used to control the intensity of the conflict potential energy;

[0214] d(T i ,T j ) is transaction T i , T j The distance in the feature space indicates the degree of closeness in the data linkage process;

[0215] exp() is the natural exponential function;

[0216] σ is an adjustment parameter used to control the impact of transaction feature differences on conflict potential;

[0217] ‖Ψ i -Ψ j ‖ is the eigenvector Ψ i , j The Euclidean distance between them is used to measure the feature difference.

[0218] Finally, the optimal conflict resolution method is selected based on the principle of minimizing conflict potential energy:

[0219] S(T i ,T j )=argmin(V(s)+μ×P(s)),s∈S

[0220] in:

[0221] S(T i ,T j ) represents two conflicting transactions T i ,T j The optimal solution strategy. During the data linkage process of codeless applications, multiple applications may try to access or modify the same data items at the same time, resulting in conflicts. By selecting an appropriate solution strategy, you can ensure data consistency while reducing the impact of conflicts;

[0222] argmin() means selecting the parameter s that minimizes the objective function value;

[0223] S represents the set of all feasible conflict resolution strategies, such as retry, delayed commit, or priority processing of one transaction;

[0224] V(s) represents the conflict potential generated when strategy s is selected;

[0225] P(s) represents the execution cost of selecting strategy s. For example, retrying may consume more system resources, while delayed commit may affect transaction response time. Selecting a low-cost strategy can improve system performance, but may lead to certain compromises in conflict handling.

[0226] μ is the balance factor used to control the weight ratio of conflict potential and cost P(s).

[0227] Step 3.6 Submit decision and confirm submitted status

[0228] In transaction control of data linkage between codeless applications, the submission decision needs to be determined based on characteristics and risk assessment:

[0229] Determine whether the transaction can be submitted based on the scope of the transaction impact and define the transaction impact assessment Θ(T):

[0230]

[0231] in, is the region boundary of loop integration; Ω is the region of area integration; dσ is a tiny boundary element, representing the integration unit of loop integration; dω is a tiny area element, representing the integration unit in area integration.

[0232] The submission decision D(T) is defined as follows:

[0233]

[0234] in:

[0235] sign() is a sign function that returns the sign of the input value (positive, negative, or zero);

[0236] ξ is the stability coefficient, which is used to adjust the impact of the stability of transaction characteristics on the commit decision;

[0237] is the Laplacian operator of the transaction feature vector;

[0238] γ is the divergence coefficient, which is used to control the impact of divergence on transaction submission;

[0239] div(Ψ) is the divergence of the transaction feature vector, indicating the degree of divergence of the feature space;

[0240] If D(T)>0, the commit condition is met, the transaction is marked as committable, and broadcast to all participating nodes of the codeless application to update the data;

[0241] If D(T)<0, the commit condition is not met, the transaction is marked as aborted, and a rollback operation is triggered to ensure that the data is restored to a consistent state;

[0242] If D(T) = 0, the system marks the transaction as "aborted", handles uncertainty conservatively, and ensures data consistency first.

[0243] Step 4. Data synchronization execution

[0244] In order to achieve efficient data synchronization between codeless applications, this paper uses vector field theory to construct a synchronization mechanism for data flow characteristics to ensure the stable flow of data between different applications. The core steps of data synchronization include the following:

[0245] Step 4.1 Constructing the synchronized vector field

[0246] The synchronization vector field F(x, t) describes the flow trend of data between codeless applications. By constructing this field, we can determine how data flows from the source location to the target location and optimize it according to the needs of the application. The vector field formula is as follows:

[0247]

[0248] in:

[0249] F(x,t) is the vector field of data synchronization, describing the flow characteristics of data between codeless applications;

[0250] A(x,t) is the data flow vector, which is used to indicate the direction and strength of data flow;

[0251] Φ(x, t) is the synchronization potential energy function, which is the potential driving force value of data at position x and time t. The larger the value, the higher the urgency of data transmission, and the data at this position will be synchronized first, thus ensuring that important data (that is, data with high potential energy) can be quickly delivered to the target application. The formula is:

[0252]

[0253] Wherein, the initial potential energy value or amplitude of Φ0 determines the overall priority of the system for all data synchronization; a higher Φ0 will accelerate the overall data flow of the system and is suitable for urgent data synchronization tasks, while a lower value is suitable for general data transmission; λ is the attenuation coefficient, and a larger λ value will rapidly attenuate the data potential, thereby limiting the scope of data synchronization and only transmitting data to applications with a closer logical distance; a smaller λ value allows data to be synchronized across a wider range of applications; |x-x0| is the logical distance between the data location x and the reference location x0, reflecting the strength of association between applications. Data that is more closely related to the target application (i.e., |x-x0| is smaller) will be synchronized faster, while data that is far away from the target application or has a weaker association with it (i.e., |x-x0| is larger) will have a lower synchronization priority, thereby ensuring that data with a strong correlation with the target application is transmitted first; cos(ωt) represents the periodic fluctuation of the data synchronization potential over time, ω is the angular frequency, and the size of ω can be adjusted to control the frequency of data synchronization. If ω is higher, data synchronization is triggered multiple times in a shorter period of time, which is suitable for high-frequency and real-time synchronization requirements; if ω is lower, it is suitable for scheduled data update tasks such as daily and weekly.

[0254] Step 4.2 Calculate data block boundaries and batch parameters

[0255] During data synchronization, optimizing the size of data blocks can improve efficiency. Using the Gaussian divergence theorem, the total flow on the data block boundary can be calculated to determine the appropriate batch size. The calculation formula is as follows:

[0256]

[0257] in:

[0258] B(t) is the batch size of data blocks;

[0259] S is the boundary of the data block, which is used to describe the boundary of data transmission;

[0260] V is the volume of the data block, which represents the amount of data processed in one batch.

[0261] When B(t) is greater than a preset threshold, the size of the data block is increased to efficiently transmit a large amount of data;

[0262] When B(t) is less than or equal to a preset threshold, the data block is reduced to optimize the transmission of small batches of data.

[0263] Step 4.3 Incremental synchronization

[0264] Incremental synchronization is used to detect the amount of data changes and perform necessary synchronization operations to reduce unnecessary full synchronization. The incremental change formula is:

[0265]

[0266] in:

[0267] Δ(B t ,B t-1 ) represents the increment of data block B from time t-1 to time t;

[0268] It represents the curl difference of data stream F from time t-1 to time t.

[0269] When the increment is greater than the preset threshold, it means that the data has changed significantly and incremental synchronization is required; when the increment is less than or equal to the preset threshold, synchronization is not performed to save resources.

[0270] Step 4.4 Concurrency Optimization

[0271] To ensure the reasonable allocation of system resources during data synchronization, use the concurrency optimization formula to adjust the concurrency intensity of data synchronization:

[0272]

[0273] in:

[0274] is the Laplace operator, which is used to calculate the spatial variation trend of concurrency C;

[0275] C is the concurrency, which indicates the intensity of concurrent execution in data synchronization;

[0276] ρ(r) is the resource density distribution function, which reflects the distribution of resources in space r. This formula is a recognized standard formula in mathematics and physics, and is usually used to describe the spatial distribution of scalar fields (such as density) and their changing trends. Therefore, it is directly quoted in the present invention and will not be elaborated in detail.

[0277] When ρ(r) is greater than the preset threshold, the concurrency C is increased to speed up the data synchronization rate;

[0278] When ρ(r) is less than or equal to the preset threshold, the concurrency is reduced to avoid resource overload.

[0279] Step 5. Exception handling and recovery

[0280] In the data linkage process of codeless applications, when data transmission delays or exceptions occur, exception handling and recovery mechanisms are needed to ensure data consistency and flow stability. This step proposes a set of exception recovery methods based on minimum action and compensation transactions. The specific process is as follows:

[0281] Step 5.1 Health Assessment

[0282] When an anomaly is detected, the health evaluation formula is used to quantify the current data linkage status and determine the urgency and priority of recovery. The health evaluation formula is as follows:

[0283]

[0284] in:

[0285] H(s) is the health evaluation value, which indicates the overall status of the current codeless application data linkage process. The lower the health value, the more serious the abnormal situation and the more urgent the recovery demand.

[0286] k B It is a balancing factor used to adjust the weights of different states in the health assessment to ensure that the assessment results are adapted to the overall execution status of the codeless application.

[0287] p i The probability of each state indicates the frequency of occurrence of each state (“normal”, “delayed”, “failed”). The state probability can be obtained through real-time monitoring data;

[0288] s is a state vector, which contains information about different states during the data linkage process between codeless applications, including "normal", "delayed", and "failed".

[0289] The calculated health value H(s) can be used to determine the overall status of data flow between codeless applications. When the health value is lower than the preset threshold, it indicates that data recovery needs to be started immediately.

[0290] Step 5.2: Definition of recovery time interval

[0291] After the recovery mechanism is triggered, the recovery time interval [t1, t2] is defined to control the time range of the recovery operation. This time interval limits the execution scope of the recovery operation and ensures that the recovery is completed within a reasonable time. Among them, t1 is the moment when the anomaly is detected, indicating the start time of the recovery operation; t2 is the estimated time to complete the recovery, which is dynamically set according to the severity of the data anomaly.

[0292] Defining a recovery time interval can prevent the recovery time from being too long and ensure that data is restored to a consistent state within a reasonable time.

[0293] Step 5.3 Recovery path selection based on minimum effort

[0294] After defining the time interval, the minimum action formula is applied to select the recovery path to ensure that the resource consumption is minimized during the recovery process. The minimum action formula is:

[0295]

[0296] in:

[0297] δ is the variation symbol, which means that the action on the recovery path is varied, that is, the change in the path with the smallest action is sought;

[0298] q state variable, indicating the specific recovery status after an exception occurs during data linkage, including the current data transmission or execution status;

[0299] is the time rate of change of the state, indicating the speed of data recovery;

[0300] t is time, indicating the time when the recovery operation is performed;

[0301] t1 and t2 are the start and end times of the recovery operation;

[0302] It is the Lagrangian quantity, which represents the dynamic changes in the data linkage process between codeless applications. It contains the energy of state change and reflects the speed and position offset of state change.

[0303] Integrating the Lagrangian in the recovery interval represents the action of the entire recovery process; by calculating the minimum action, the recovery path with the lowest energy consumption is preferentially selected to ensure efficient resource utilization.

[0304] Step 5.4 Compensation transaction design and execution

[0305] In the last step of recovery, the compensation force function Q(t) is applied to ensure that the data state is restored to a normal state and maintain data consistency through an automatic compensation mechanism. The compensation force function is:

[0306]

[0307] in:

[0308] Q(t) is the compensation force function, which indicates the compensation strength during the recovery process and is used to correct the data state to ensure that the data after recovery is consistent with the expected state;

[0309] is the partial derivative of the Lagrangian with respect to velocity, reflecting the influence of velocity change on the compensation process; It is the partial derivative of the Lagrangian with respect to the position, reflecting the influence of the state position on the compensation process.

[0310] The compensation force function Q(t) controls the compensation strength during the recovery process to ensure that the data is consistent after recovery. For example, when an error occurs in a transaction operation, the compensation force function will automatically roll back the data based on the calculation to maintain data consistency between codeless applications.

[0311] Step 6. Security protection mechanism

[0312] Finally, in the data linkage between codeless applications, in order to ensure the security and privacy of data, a dynamically adjusted security protection mechanism is designed through wave functions, diffusion equations and nonlinear mapping methods to meet the needs of permission management, sensitive data protection and data desensitization.

[0313] Step 6.1 Define the protection strength of sensitive data

[0314] The protection strength of sensitive data is defined by the wave function, and the protection strength is dynamically adjusted according to the importance of the data to adapt to the data transmission security requirements of different scenarios. The formula is as follows:

[0315] Ψ(r,t)=Ae i(kr-ωt)

[0316] in:

[0317] Ψ(r,t) is the protection strength wave function, which represents the protection strength of the data at position r and time t;

[0318] A is the amplitude of the protection strength, which determines the overall protection level of the data. The larger the amplitude, the higher the protection strength.

[0319] k is the spatial wave number, which controls the variation of protection intensity at different locations;

[0320] ω is the time frequency, which controls the dynamic adjustment of protection intensity over time.

[0321] By adjusting the A, k, and ω parameters, the protection strength can be flexibly set according to the sensitivity level of the data and the application scenario. For example, the protection strength of highly sensitive data will be dynamically enhanced as the transmission time passes or the spatial location changes.

[0322] Step 6.2 Diffusion Equation Model of Access Control Matrix

[0323] The access control matrix is ​​established using the diffusion equation model, so that access rights can be dynamically adjusted according to real-time needs and access frequency to improve the real-time and accuracy of permission management. The formula is as follows:

[0324]

[0325] in:

[0326] M is the access control matrix, which represents the access rights of different subjects to data;

[0327] is the time change rate of the matrix, reflecting the real-time adjustment of permissions;

[0328] D is the diffusion coefficient, which controls the speed and scope of permission changes. A high diffusion coefficient can be used for frequently accessed data to quickly respond to permission requests.

[0329] is the Laplacian operator of the matrix, which is used for spatial authority diffusion;

[0330] f(M,t) is a control function that adjusts the values ​​of matrix elements according to time to meet real-time permission requirements.

[0331] This diffusion equation ensures that access rights can be adjusted dynamically according to access needs and time. For sensitive data with high frequency of access, it can accelerate the diffusion of rights and improve response speed; for data with low frequency of access, it maintains a low diffusion rate and reduces changes in rights.

[0332] Step 6.3 Nonlinear mapping of data desensitization

[0333] Nonlinear mapping is used to desensitize sensitive data to protect data privacy while ensuring basic data availability. The parameters of the desensitization mapping will be automatically adjusted according to the sensitivity level of the data. The mapping formula is as follows:

[0334] T(x)=tanh(αx+β)×e -γ|x|

[0335] in:

[0336] T(x) desensitization mapping function, used to convert the original data x into desensitized data;

[0337] α and β are parameters that control the degree of nonlinearity of the mapping. The higher the value, the greater the difference between the desensitized data and the original data;

[0338] Tanh(αx+β) is a nonlinear transformation that compresses data into a smaller range, which is suitable for protecting sensitive data;

[0339] γ is the attenuation coefficient, which controls the attenuation strength of data desensitization. When it is larger, it is suitable for highly sensitive data.

[0340] By adjusting α, β, and γ, data desensitization of different sensitivity levels can be achieved. For example, for highly sensitive data, the α and γ values ​​can be increased to enhance the desensitization effect, while for low-sensitivity data, these parameters can be reduced to ensure partial availability of the data.

[0341] Through the above-mentioned accompanying drawings, the present invention provides an innovative monitoring data linkage method between codeless applications, which aims to improve the intelligence and accuracy of data linkage through accurate automatic data structure identification, change monitoring, and distributed transaction control. In particular, by utilizing multi-level Bloom filters, vector field theory, and exception recovery path optimization based on minimum action, the method not only improves the efficiency and consistency of data synchronization, but also enhances the security and privacy protection during data processing.

[0342] In addition, the implementation of the present invention can significantly reduce the need for manual intervention in a code-free environment, effectively support data sharing and collaboration across applications, and provide solid technical support for multi-scenario data interaction. It has high practical value and broad application prospects.

[0343] It is worth noting that the protection scope of the present invention is not limited to the specific implementations described in the above embodiments. Equivalent variations or modifications made by those skilled in the art based on the enlightenment of the present invention should be included in the protection scope of the present invention as long as they do not deviate from the spirit of the technical solution of the present invention.

Claims

1. A monitoring data linkage method between codeless applications, characterized in that: The following steps are involved: Step 1. Data structure identification: Establish a metadata description model between codeless applications. The model includes application identifiers, data table sets, field sets, and association relationship sets. Multidimensional feature vectors are used to describe field attributes to quantify field distribution characteristics. A similarity function is constructed based on field name similarity, type compatibility, and value distribution similarity to achieve mapping from the source application structure to the target application structure. Step 2. Data change monitoring: Use the local sensitive hashing algorithm to detect data changes in real time, build a hash function family through random projection vectors and random offsets, and realize the rapid extraction of data features; and introduce an adaptive bucket width mechanism to dynamically adjust the detection sensitivity according to the frequency of data changes; Step 3. Distributed transaction control: Characterize transaction characteristics based on multidimensional feature space, measure transaction risks through state transition operators, and adopt isolation level decision mechanism based on energy function to achieve dynamic adjustment of isolation strength; define conflict potential to describe the interaction between transactions, and design conflict resolution strategies to minimize potential energy; Step 4. Data synchronization execution: Construct a synchronization vector field that describes the data flow characteristics, determine the optimal batch processing parameters and concurrency through the Gaussian divergence theorem, and achieve data synchronization between codeless applications; Step 5. Abnormal handling and recovery: Use the entropy function to evaluate the health and quantify the order of the state, design the fault recovery strategy based on the principle of least action, and obtain the optimal recovery path through the Lagrange equation; Step 6. Security protection mechanism: The protection strength of sensitive data is described by wave function, the dynamic evolution mechanism of the access control matrix is ​​constructed using the diffusion equation model, and nonlinear mapping is designed to achieve data desensitization, and the mapping parameters are adjusted according to the sensitivity level.

2. The method according to claim 1, wherein: The data change monitoring step further includes using a multi-stage Bloom filter to optimize and remove redundant data during the detection process.

3. The method according to claim 1, wherein: The distributed transaction control step further includes calculating the risk change rate using the time partial derivative and divergence of the transaction characteristics, and calculating the cumulative risk by time integration, taking into account the attenuation effect of historical risks.

4. The method according to claim 1, wherein: The data synchronization execution step utilizes a differential operator to capture the curl characteristics of data changes to achieve accurate incremental synchronization.

5. The method according to claim 1, wherein: The synchronization vector field in the data synchronization execution step utilizes a synchronization potential function, which is dynamically adjusted based on the location and time of the data to preferentially synchronize high priority or urgent data.

6. The method according to claim 1, wherein: The security protection mechanism step includes dynamically adjusting wave function parameters according to the protection strength of sensitive data, including adjusting the wave number and frequency to adapt to different protection requirements.

Images