Data optimization method and system based on cross-module association modeling

Abstract

The application discloses a data optimization method and system based on cross-module association modeling, and the method comprises the following steps: acquiring historical working data of a plurality of business modules of a project system; establishing a data dependency logical model between any two business modules according to the historical working data; in response to new working data generated by any two business modules, judging whether the new working data has a data conflict situation; and when the data conflict situation exists, optimizing and correcting the new working data based on the data dependency logical model to obtain optimized working data. It can be seen that the application can realize dynamic modeling and intelligent correction of data dependency between business modules, improve system data consistency and working efficiency, and reduce the risk of business errors caused by data conflicts.

Description

Technical Field

[0001] This invention relates to the field of data processing technology, and in particular to a data optimization method and system based on cross-module association modeling. Background Technology

[0002] With the rapid increase in the complexity of enterprise project management systems and the frequency of data interaction between business modules, project management and IT departments are placing increasing emphasis on data consistency. A key technical issue is how to promptly identify and intelligently correct data conflicts between business modules to reduce the risk of business errors. Existing technologies typically use fixed dependency rules or manual review to check data consistency and handle conflicts based on preset templates to maintain basic system operation. However, existing solutions lack dynamic modeling of data dependency logic between any two business modules and intelligent judgment and optimization mechanisms for new data conflicts. This makes it difficult to achieve real-time correlation analysis and automatic correction of data dependencies between business modules. Commonly used static rules or manual intervention methods are unsuitable for complex multi-module business scenarios, resulting in insufficient system data consistency. Data conflicts can easily lead to business errors or low work efficiency, limiting the data reliability and overall operational performance of the project system. Therefore, existing technologies have shortcomings that urgently need to be addressed. Summary of the Invention

[0003] The technical problem to be solved by this invention is to provide a data optimization method and system based on cross-module association modeling, which can realize dynamic modeling and intelligent conflict correction of data dependencies between business modules, improve system data consistency and work efficiency, and reduce the risk of business errors caused by data conflicts.

[0004] To address the aforementioned technical problems, the first aspect of this invention discloses a data optimization method based on cross-module association modeling, the method comprising: Obtain historical work data from multiple business modules of the project system; Based on the historical work data, establish a data dependency logic model between any two of the business modules; In response to new work data generated by any two of the aforementioned business modules, determine whether there is a data conflict in the new work data; In the event of data conflicts, the new working data is optimized and corrected based on the data dependency logic model to obtain optimized working data.

[0005] As an optional implementation, in the first aspect of the present invention, the historical work data or the new work data includes at least one of process data, business data and status change data.

[0006] As an optional implementation, in the first aspect of the present invention, establishing a data dependency logic model between any two business modules based on the historical work data includes: For any two business modules, the data belonging to the same time period in the historical work data corresponding to the two business modules are input into a preset association feature model for training until convergence, so as to obtain the time data dependency model corresponding to the two business modules. Data belonging to the same user type from the historical work data corresponding to the two business models are input into a preset association feature model for training until convergence, so as to obtain the user data dependency model corresponding to the two business modules. Data belonging to the same business type from the historical work data corresponding to the two business models are input into a preset association feature model for training until convergence, so as to obtain the business data dependency model corresponding to the two business modules. The time data dependency model, the user data dependency model, and the business data dependency model are determined as the data dependency logic models corresponding to the two business modules.

[0007] As an optional implementation, in the first aspect of the present invention, determining the time data dependency model, the user data dependency model, and the business data dependency model as the data dependency logic models corresponding to the two business modules includes: For any two models among the time data dependency model, the user data dependency model, and the business data dependency model, a vectorized base model is established between the two models. Input the historical working data corresponding to any of the aforementioned business modules into the two models to obtain the predicted correlation data output separately; The historical working data and the predicted correlation data of the two models are used as training data and input into the vectorized base model for training until convergence, thus obtaining the vectorized model between the two models. The time data dependency model, the user data dependency model, and the business data dependency model, along with the corresponding vectorized model, are determined as the data dependency logic models corresponding to the two business modules.

[0008] As an optional implementation, in a first aspect of the invention, the vectorization model is used to simultaneously output vector representations with semantically identical features to two models based on input historical working data; the loss function of the vectorization base model during training is the sum of a first loss function and a second loss function; the first loss function is a first degree of difference between the first vector prediction data and the predicted associated data of one of the two models; the first vector prediction data is the output prediction data of the first output vector representation of the vectorization base model after processing the historical working data, inputted into one of the two models; the second loss function is a second degree of difference between the second vector prediction data and the predicted associated data of the other of the two models; the second vector prediction feature is the output prediction data of the second output vector representation of the vectorization base model after processing the historical working data, inputted into the other of the two models.

[0009] As an optional implementation, in the first aspect of the present invention, determining whether the new working data has a data conflict includes: The new working data of either of the two business modules is input into the time data dependency model, the user data dependency model and the business data dependency model corresponding to the two business modules to obtain the output predicted time-related data, predicted user-related data and predicted business-related data; Calculate the feature similarity corresponding to the predicted time-related data, predicted user-related data, and predicted business-related data; Determine whether the data feature degree is less than a preset similarity threshold. If so, determine that the new working data has a data conflict.

[0010] As an optional implementation, in the first aspect of the present invention, calculating the feature similarity corresponding to the predicted time-related data, predicted user-related data, and predicted business-related data includes: Calculate the first data similarity between the predicted time-related data and the new working data of the other module in the two business modules; Calculate a second data similarity between the predicted user association data and the new working data of the other module in the two business modules; Calculate the third data similarity between the predicted business-related data and the new working data of the other module in the two business modules; The feature similarity is obtained by calculating the weighted sum of the first data similarity, the second data similarity, and the third data similarity.

[0011] As an optional implementation, in the first aspect of the present invention, optimizing and correcting the new working data based on the data dependency logic model to obtain optimized working data includes: Calculate the feature similarity for each of the new working data, and identify the new working data with lower feature similarity as the data to be optimized; The prediction dependency model corresponding to the highest of the first data similarity, the second data similarity, and the third data similarity for the data to be optimized is determined as the target dependency prediction model; the target dependency prediction model is the time data dependency model, the user data dependency model, or the business data dependency model; For each data portion in the data to be optimized, identify associated data in another set of new working data that is related to the data type of that data portion; The associated data is input into the target dependency prediction model to obtain the output prediction data portion; Calculate the data difference between the predicted data portion and the data portion; When the data difference is less than a preset difference threshold, the data portion is replaced with the predicted data portion.

[0012] A second aspect of this invention discloses a data optimization system based on cross-module association modeling, the system comprising: The acquisition module is used to acquire historical work data from multiple business modules of the project system. A module is established to create a data dependency logic model between any two of the business modules based on the historical work data. The judgment module is used to determine whether there is a data conflict in the new working data generated by any two of the business modules. The optimization module is used to optimize and correct the new working data based on the data dependency logic model when data conflicts exist, so as to obtain optimized working data.

[0013] As an optional implementation, in a second aspect of the present invention, the historical work data or the new work data includes at least one of process data, business data, and status change data.

[0014] As an optional implementation, in the second aspect of the present invention, the specific method by which the establishing module establishes a data dependency logic model between any two business modules based on the historical working data includes: For any two business modules, the data belonging to the same time period in the historical work data corresponding to the two business modules are input into a preset association feature model for training until convergence, so as to obtain the time data dependency model corresponding to the two business modules. Data belonging to the same user type from the historical work data corresponding to the two business models are input into a preset association feature model for training until convergence, so as to obtain the user data dependency model corresponding to the two business modules. Data belonging to the same business type from the historical work data corresponding to the two business models are input into a preset association feature model for training until convergence, so as to obtain the business data dependency model corresponding to the two business modules. The time data dependency model, the user data dependency model, and the business data dependency model are determined as the data dependency logic models corresponding to the two business modules.

[0015] As an optional implementation, in the second aspect of the present invention, the specific method by which the establishing module determines the time data dependency model, the user data dependency model, and the business data dependency model as the data dependency logic model corresponding to the two business modules includes: For any two models among the time data dependency model, the user data dependency model, and the business data dependency model, a vectorized base model is established between the two models. Input the historical working data corresponding to any of the aforementioned business modules into the two models to obtain the predicted correlation data output separately; The historical working data and the predicted correlation data of the two models are used as training data and input into the vectorized base model for training until convergence, thus obtaining the vectorized model between the two models. The time data dependency model, the user data dependency model, and the business data dependency model, along with the corresponding vectorized model, are determined as the data dependency logic models corresponding to the two business modules.

[0016] As an optional implementation, in a second aspect of the invention, the vectorization model is used to simultaneously output vector representations with semantically identical features to two models based on input historical working data; the loss function of the vectorization base model during training is the sum of a first loss function and a second loss function; the first loss function is a first degree of difference between the first vector prediction data and the predicted associated data of one of the two models; the first vector prediction data is the output prediction data of the first output vector representation of the vectorization base model after processing the historical working data, inputted into one of the two models; the second loss function is a second degree of difference between the second vector prediction data and the predicted associated data of the other of the two models; the second vector prediction feature is the output prediction data of the second output vector representation of the vectorization base model after processing the historical working data, inputted into the other of the two models.

[0017] As an optional implementation, in the second aspect of the present invention, the specific method by which the determining module determines whether the new working data has a data conflict includes: The new working data of either of the two business modules is input into the time data dependency model, the user data dependency model and the business data dependency model corresponding to the two business modules to obtain the output predicted time-related data, predicted user-related data and predicted business-related data; Calculate the feature similarity corresponding to the predicted time-related data, predicted user-related data, and predicted business-related data; Determine whether the data feature degree is less than a preset similarity threshold. If so, determine that the new working data has a data conflict.

[0018] As an optional implementation, in a second aspect of the invention, the specific method by which the determining module calculates the feature similarity corresponding to the predicted time-related data, the predicted user-related data, and the predicted business-related data includes: Calculate the first data similarity between the predicted time-related data and the new working data of the other module in the two business modules; Calculate a second data similarity between the predicted user association data and the new working data of the other module in the two business modules; Calculate the third data similarity between the predicted business-related data and the new working data of the other module in the two business modules; The feature similarity is obtained by calculating the weighted sum of the first data similarity, the second data similarity, and the third data similarity.

[0019] As an optional implementation, in a second aspect of the invention, the optimization module optimizes and corrects the new working data based on the data dependency logic model to obtain optimized working data, including: Calculate the feature similarity for each of the new working data, and identify the new working data with lower feature similarity as the data to be optimized; The prediction dependency model corresponding to the highest of the first data similarity, the second data similarity, and the third data similarity for the data to be optimized is determined as the target dependency prediction model; the target dependency prediction model is the time data dependency model, the user data dependency model, or the business data dependency model; For each data portion in the data to be optimized, identify associated data in another set of new working data that is related to the data type of that data portion; The associated data is input into the target dependency prediction model to obtain the output prediction data portion; Calculate the data difference between the predicted data portion and the data portion; When the data difference is less than a preset difference threshold, the data portion is replaced with the predicted data portion.

[0020] A third aspect of this invention discloses another data optimization system based on cross-module association modeling, the system comprising: Memory containing executable program code; A processor coupled to the memory; The processor calls the executable program code stored in the memory to execute some or all of the steps in the data optimization method based on cross-module association modeling disclosed in the first aspect of the present invention.

[0021] The fourth aspect of the present invention discloses a computer storage medium storing computer instructions, which, when invoked, are used to execute some or all of the steps in the data optimization method based on cross-module association modeling disclosed in the first aspect of the present invention.

[0022] Compared with the prior art, the embodiments of the present invention have the following beneficial effects: This invention establishes a data dependency logic model between any two business modules by acquiring historical work data from multiple business modules of a project system. When new work data is generated, it determines data conflicts and optimizes and corrects the data based on the dependency logic model to obtain optimized work data. This enables dynamic modeling and intelligent conflict correction of data dependencies between business modules, improves system data consistency and work efficiency, and reduces the risk of business errors caused by data conflicts. Attached Figure Description

[0023] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0024] Figure 1 This is a flowchart illustrating a data optimization method based on cross-module association modeling disclosed in an embodiment of the present invention.

[0025] Figure 2 This is a schematic diagram of the structure of a data optimization system based on cross-module association modeling disclosed in an embodiment of the present invention.

[0026] Figure 3 This is a schematic diagram of another data optimization system based on cross-module association modeling disclosed in an embodiment of the present invention. Detailed Implementation

[0027] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0028] The terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this invention are used to distinguish different objects, not to describe a specific order. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, apparatus, product, or device that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products, or devices.

[0029] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of the invention. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0030] This invention discloses a data optimization method and system based on cross-module correlation modeling. By acquiring historical work data from multiple business modules of a project system, a data dependency logic model is established between any two business modules. When new work data is generated, data conflicts are identified, and optimization and correction are performed based on the dependency logic model to obtain optimized work data. This enables dynamic modeling and intelligent conflict correction of data dependencies between business modules, improving system data consistency and work efficiency, and reducing the risk of business errors caused by data conflicts. Detailed explanations follow.

[0031] Example 1 Please see Figure 1 , Figure 1 This is a flowchart illustrating a data optimization method based on cross-module association modeling disclosed in an embodiment of the present invention. Figure 1 The described data optimization method based on cross-module correlation modeling can be applied to data processing systems / data processing equipment / data processing servers (including local processing servers or cloud processing servers). Figure 1 As shown, this data optimization method based on cross-module association modeling can include the following operations: 101. Obtain historical work data from multiple business modules of the project system.

[0032] Optionally, the system in this project can be an Enterprise Resource Planning (ERP) system, a Customer Relationship Management (CRM) system, an Automated Office Automation (OA) system, a research and development project management pipeline, or a distributed microservice architecture system; this invention does not limit the scope of the system.

[0033] Optionally, this business module can be an order management module, an inventory control module, a financial settlement module, a user permission verification module, or a production planning and scheduling module; this invention does not impose any limitations.

[0034] Optionally, historical work data or new work data includes at least one of process data, business data, and status change data.

[0035] Optionally, the process data can be the current node information of the business approval flow, the process branch selection mark, or the priority weight of task execution; this invention does not limit this.

[0036] Optionally, the business data can be the transaction amount, product inventory balance, user identification ID, or geographical coordinates; this invention does not impose any limitations.

[0037] Optionally, the state change data can be a rollback identifier of a database transaction, a change enumeration value of the system lifecycle from "initialization" to "activation", or a state bitmap in binary form; this invention does not impose any limitations.

[0038] Optionally, the historical working data can be structured SQL records stored in a database, unstructured log files, distributed tracing data, historical messages in a message queue, or system snapshot images; this invention does not impose any limitations.

[0039] 102. Based on historical work data, establish a data dependency logic model between any two business modules.

[0040] Optionally, the data dependency logic model can be a directed acyclic graph (DAG) structure, a time series prediction model built on a long short-term memory network (LSTM), an association feature matrix based on a graph neural network (GCN), or a nonlinear mapping model built on a multilayer perceptron. This invention does not limit the model.

[0041] 103. In response to new work data generated by any two business modules, determine whether there is a data conflict in the new work data.

[0042] Optionally, the new working data can be API call requests generated in real time, form transactions submitted by the front end, asynchronously triggered status update instructions, or automatically executed tasks generated by the system at regular intervals; this invention does not impose any limitations.

[0043] Optionally, the data conflict situation can be a concurrent conflict caused by multiple modules modifying the same global variable at the same time, an inconsistency in state machine transition logic, mutual exclusion of business field logic, or an abnormal state that violates data integrity constraints. This invention does not limit the scope of the conflict.

[0044] 104. In the event of data conflicts, optimize and correct the new working data based on the data dependency logic model to obtain optimized working data.

[0045] As can be seen, the above-described embodiments of the invention establish a data dependency logic model between any two business modules by acquiring historical work data from multiple business modules of the project system. When new work data is generated, the data conflict situation is judged and optimized and corrected based on the dependency logic model to obtain optimized work data. This enables dynamic modeling and intelligent conflict correction of data dependencies between business modules, improves system data consistency and work efficiency, and reduces the risk of business errors caused by data conflicts.

[0046] As an optional embodiment, the step above, establishing a data dependency logic model between any two business modules based on historical working data, includes: For any two business modules, the data belonging to the same time period in the historical work data corresponding to the two business modules are input into the preset association feature model for training until convergence, so as to obtain the time data dependency model corresponding to the two business modules. Data belonging to the same user type from the historical work data corresponding to the two business models are input into a preset association feature model for training until convergence, so as to obtain the user data dependency model corresponding to the two business modules. Data belonging to the same business type from the historical work data corresponding to the two business models are input into a preset association feature model for training until convergence, so as to obtain the business data dependency model corresponding to the two business modules. The time data dependency model, user data dependency model, and business data dependency model are identified as the data dependency logic models corresponding to the two business modules.

[0047] Optionally, the associated feature model can be a self-attention mechanism model based on the Transformer architecture, a temporal convolutional network (TCN), a bidirectional recurrent neural network (Bi-RNN), or a feature extractor with a gating mechanism; this invention does not limit the model.

[0048] Optionally, the time data dependency model can be a mathematical model reflecting the probability distribution of business response delay, a task arrival model based on Poisson distribution, or a time-domain feature vector set that captures periodic patterns; this invention does not impose any limitations.

[0049] Optionally, the user type for this operation can be a system administrator, a regular salesperson, a guest user, an automated script robot, or a permission group with a specific role; this invention does not impose any limitations.

[0050] Optionally, the user data dependency model can be a collaborative filtering model based on user behavior profiles, a user permission topology graph, or a probabilistic graphical model that reflects the operating habits of a specific role; this invention does not impose any limitations.

[0051] Optionally, the business type can be core financial settlement business, non-core information query business, high-frequency data writing business, or long-cycle asynchronous processing business; this invention does not impose any limitations.

[0052] As can be seen, through the above optional embodiments, by training the historical work data of any two business modules according to time period, operation user type and business type respectively, a time data dependency model, a user data dependency model and a business data dependency model are obtained and determined as the data dependency logic model. This enables accurate modeling of the dependency relationship of multi-dimensional historical data, improves the comprehensiveness and accuracy of the dependency logic, and reduces the risk of missing dependency relationships due to single-dimensional analysis.

[0053] As an optional embodiment, the step above, determining the time data dependency model, user data dependency model, and business data dependency model as the data dependency logic model corresponding to the two business modules, includes: For any two models among the time data dependency model, user data dependency model, and business data dependency model, establish a vectorized base model between the two models. Input the historical work data corresponding to any business module into the two models to obtain the predicted correlation data output separately; Historical working data and the prediction correlation data of the two models are used as training data and input into the vectorized base model for training until convergence, thus obtaining the vectorized model between the two models. The time data dependency model, user data dependency model, and business data dependency model, along with their corresponding vectorized models, are identified as the data dependency logic models for these two business modules.

[0054] Optionally, the vectorization base model can be an autoencoder based on the AutoEncoder structure, an embedding layer model evolved from Word2Vec, a Siamese network with feature alignment function, or a representation learning network based on contrastive learning. This invention does not limit the specific model.

[0055] As can be seen, by establishing a vectorized basic model among the time, user, and business data dependency models, and integrating all dependency models with the vectorized model to form a data dependency logic model, the integration and consistency of the relationship between models are improved, and the risk of dependency logic inconsistency caused by model independence is reduced.

[0056] As an optional embodiment, in the above steps, the vectorization model is used to simultaneously output vector representations with semantic features that are as similar as possible to the two models based on the input historical working data; the loss function of the vectorization base model during training is the sum of the first loss function and the second loss function; the first loss function is the first difference between the first vector prediction data and the predicted associated data of one of the two models; the first vector prediction data is the output prediction data of the first output vector representation of the vectorization base model after processing the historical working data and then inputting it into one of the two models; the second loss function is the second difference between the second vector prediction data and the predicted associated data of the other of the two models; the second vector prediction feature is the output prediction data of the second output vector representation of the vectorization base model after processing the historical working data and then inputting it into the other of the two models.

[0057] Optionally, the first or second degree of difference can be mean squared error (MSE), cross-entropy loss, cosine distance loss, or KL divergence, and this invention does not limit it.

[0058] As can be seen, through the above optional embodiments, by limiting the joint loss function and training a vectorized model using historical working data, a unified vector semantic representation of multi-dependency models can be achieved, which helps to improve the fusion and consistency of the correlation between models.

[0059] As an optional embodiment, the step of determining whether there is a data conflict in the new working data includes: The new working data of either of the two business modules is input into the time data dependency model, user data dependency model and business data dependency model corresponding to the two business modules to obtain the output predicted time-related data, predicted user-related data and predicted business-related data. Calculate the feature similarity corresponding to predicted time-related data, predicted user-related data, and predicted business-related data; Determine if the data feature degree is less than the preset similarity threshold. If so, determine that there is a data conflict in the new working data.

[0060] Optionally, the feature similarity can be numerical correlation calculated based on Pearson correlation coefficient, inverse ratio of Manhattan distance, structural similarity index SSIM, or feature point overlap extracted based on principal component analysis; this invention does not limit the specific features.

[0061] Optionally, the similarity threshold can be 0.75, 0.85, or a dynamic adaptive discrimination boundary value obtained based on the statistics of historical conflict samples; this invention does not limit the value.

[0062] As can be seen, through the above optional embodiments, by inputting new working data of any business module into the corresponding time, user and business data dependency model to obtain predicted related data, calculating the feature similarity of the predicted related data to judge the data conflict situation, realizing accurate conflict detection based on historical dependency prediction, improving the intelligence and timeliness of conflict judgment, and reducing the risk of data error propagation caused by undetected conflicts.

[0063] As an optional embodiment, the step of calculating the feature similarity corresponding to the predicted time-related data, predicted user-related data, and predicted business-related data in the above steps includes: Calculate the first data similarity between the predicted time-related data and the new working data of another module in the two business modules; Calculate the second data similarity between the predicted user association data and the new working data of the other module in the two business modules; Calculate the third data similarity between the predictive business-related data and the new working data of another module in the two business modules; The feature similarity is obtained by calculating the weighted average of the first, second, and third data similarities.

[0064] As can be seen, through the above optional embodiments, feature similarity is obtained by calculating the multi-dimensional similarity between the predicted associated data and the new working data of another module, and the new working data with lower feature similarity is identified as the data to be optimized. This achieves accurate positioning of conflicting data based on multi-similarity comparison, improves the accuracy of optimization target selection, and reduces the risk of incorrect optimization objects caused by similarity calculation deviation.

[0065] As an optional embodiment, the step above, optimizing and correcting the new working data based on the data dependency logic model to obtain optimized working data, includes: Calculate the feature similarity for each new working data point, and identify the new working data with lower feature similarity as the data to be optimized; The prediction dependency model corresponding to the highest of the first, second, and third data similarities of the data to be optimized is determined as the target dependency prediction model; optionally, the target dependency prediction model is a time data dependency model, a user data dependency model, or a business data dependency model. For each data segment in the data to be optimized, identify the associated data in another new working data segment that is related to the data type of that data segment; The associated data is input into the target dependency prediction model to obtain the output prediction data portion; Calculate the data difference between the predicted data portion and the actual data portion; When the data difference is less than the preset difference threshold, the data portion is replaced with the predicted data portion.

[0066] Optionally, the data to be optimized may be a transaction request that is judged to have a high degree of logical deviation, an abnormal data frame with a format error, or a state change instruction that violates the order of business operations; this invention does not limit this.

[0067] Optionally, the data portion can be a specific field in a data message, a single record in a transaction sequence, or a single weight in a state transition matrix; this invention does not impose any limitations.

[0068] Optionally, the replacement can be a forced overwrite operation on the original data, an asynchronous correction based on logical locks, marking it as "corrected" in the log and re-entering it into the database, or triggering the reload mechanism of the front-end system. This invention does not limit the scope of the replacement.

[0069] As can be seen, through the above optional embodiments, by determining the target dependency prediction model corresponding to the highest similarity of the data to be optimized, the part of the data to be optimized is used to find related data in another new working data and input into the target model to obtain the predicted data part, and the data is replaced when the difference is less than the threshold, so as to achieve accurate data part optimization and correction based on the target dependency model, improve the consistency and quality of working data, and reduce the risk of business logic conflicts caused by improper correction.

[0070] In one specific implementation plan, the above technical solution is achieved through a comprehensive project pre-construction service management system (hereinafter referred to as the system) and its supporting "Qicheng" platform underlying architecture. The system is deployed on a power grid company's cloud server and adopts a front-end and back-end separation technical architecture to ensure digital management of the entire project lifecycle.

[0071] The specific implementation process is as follows: First, the system acquires historical operational data from multiple business modules within the project system. This system covers several core business modules, including project registration, feasibility studies, meeting management, contract management, procurement management, and monthly investment reports. Through the high-performance database connection pool AlibabaDruid integrated into the backend, the system utilizes the MyBatis persistence layer framework to extract historical operational records, business-related data, and reported information for each module throughout the entire process of "project registration—feasibility studies—meeting decisions—contract execution—procurement implementation" from the domestic relational database PolarDB and the Redis cache used for processing unstructured data.

[0072] Secondly, based on this historical work data, a data dependency logic model is established between any two of these business modules. This model is built upon the unified master data standard and metadata management system established by the system, aiming to solve the complex business relationship challenges under the parallel operation of multiple modules. This model can identify, for example, how the master index data of the "Project Filing Module" affects the project indicators of the "Procurement Management Module," and the data dependency relationship between the "Contract Management Module" and the "Supplier Center." This process involves periodically performing data analysis using the Quartz task scheduling framework within the Spring Boot framework, and using the E-Charts visualization tool to graphically display the logical dependencies between modules.

[0073] Subsequently, in response to any two new work data generated by this business module, it determines whether there is a data conflict. When a user enters new data through a front-end interface built on Vue 2.6 and ElementUI, the request is forwarded to the back-end via an Nginx reverse proxy. This system utilizes Spring Boot's built-in validation rule engine to compare newly generated data (such as newly entered contract numbers or project budgets) in real time to see if they conflict with preset rules in the dependency logic model or existing master data. Especially in multi-system integration and high-concurrency access scenarios, Spring Security ensures secure data access during the compliance validation process.

[0074] Finally, in the event of data conflicts, the new working data is optimized and corrected based on this data dependency logic model to obtain optimized working data. The system utilizes intelligent assistance at the execution layer to invoke logic for automatically filling in form fields (such as project metrics and customer information), and automatically corrects redundant or inconsistent data according to the conflict resolution strategy in the data dependency logic model. For example, if inconsistencies are found between the data in the "Procurement Results" and "Contract Application" stages, the system will correct them based on the master data standard, achieving "one-time entry, global sharing," thereby eliminating human error across systems. Finally, the optimized working data is synchronized in real-time to the "Qicheng" platform and related business subsystems via a RESTful API interface.

[0075] In addition, the system supports real-time display of key indicators and data correction status at each stage of the project from filing to procurement via an 8-inch capacitive touch LCD monitor or a large-screen visual interface, ensuring high reliability and data consistency of the system in a 24-hour operating environment.

[0076] Example 2 Please see Figure 2 , Figure 2 This is a schematic diagram of the structure of a data optimization system based on cross-module association modeling disclosed in an embodiment of the present invention. Figure 2 The described data optimization system based on cross-module correlation modeling can be applied to data processing systems / data processing equipment / data processing servers (including local processing servers or cloud processing servers). For example... Figure 2 As shown, this data optimization system based on cross-module correlation modeling may include: Module 201 is used to obtain historical work data from multiple business modules of the project system.

[0077] Establish module 202, which is used to establish a data dependency logic model between any two business modules based on historical work data.

[0078] The judgment module 203 is used to determine whether there is a data conflict in the new working data generated by any two business modules.

[0079] The optimization module 204 is used to optimize and correct new working data based on the data dependency logic model when data conflicts exist, so as to obtain optimized working data.

[0080] As can be seen, the above-described embodiments of the invention establish a data dependency logic model between any two business modules by acquiring historical work data from multiple business modules of the project system. When new work data is generated, the data conflict situation is judged and optimized and corrected based on the dependency logic model to obtain optimized work data. This enables dynamic modeling and intelligent conflict correction of data dependencies between business modules, improves system data consistency and work efficiency, and reduces the risk of business errors caused by data conflicts.

[0081] As an optional embodiment, historical work data or new work data includes at least one of process data, business data, and status change data.

[0082] As can be seen, the above optional embodiments define the data content of the working data to comprehensively characterize the working data features in the project system, assist in the dynamic modeling of data dependencies between business modules and intelligent conflict correction, and improve system data consistency and work efficiency.

[0083] As an optional implementation, the specific method by which the module establishes a data dependency logic model between any two business modules based on historical working data includes: For any two business modules, the data belonging to the same time period in the historical work data corresponding to the two business modules are input into the preset association feature model for training until convergence, so as to obtain the time data dependency model corresponding to the two business modules. Data belonging to the same user type from the historical work data corresponding to the two business models are input into a preset association feature model for training until convergence, so as to obtain the user data dependency model corresponding to the two business modules. Data belonging to the same business type from the historical work data corresponding to the two business models are input into a preset association feature model for training until convergence, so as to obtain the business data dependency model corresponding to the two business modules. The time data dependency model, user data dependency model, and business data dependency model are identified as the data dependency logic models corresponding to the two business modules.

[0084] As can be seen, through the above optional embodiments, by training the historical work data of any two business modules according to time period, operation user type and business type respectively, a time data dependency model, a user data dependency model and a business data dependency model are obtained and determined as the data dependency logic model. This enables accurate modeling of the dependency relationship of multi-dimensional historical data, improves the comprehensiveness and accuracy of the dependency logic, and reduces the risk of missing dependency relationships due to single-dimensional analysis.

[0085] As an optional implementation, the method by which the establishment module determines the time data dependency model, user data dependency model, and business data dependency model as the data dependency logic model corresponding to the two business modules includes: For any two models among the time data dependency model, user data dependency model, and business data dependency model, establish a vectorized base model between the two models. Input the historical work data corresponding to any business module into the two models to obtain the predicted correlation data output separately; Historical working data and the prediction correlation data of the two models are used as training data and input into the vectorized base model for training until convergence, thus obtaining the vectorized model between the two models. The time data dependency model, user data dependency model, and business data dependency model, along with their corresponding vectorized models, are identified as the data dependency logic models for these two business modules.

[0086] As can be seen, by establishing a vectorized basic model among the time, user, and business data dependency models, and integrating all dependency models with the vectorized model to form a data dependency logic model, the integration and consistency of the relationship between models are improved, and the risk of dependency logic inconsistency caused by model independence is reduced.

[0087] As an optional embodiment, the vectorization model is used to simultaneously output vector representations with semantically identical features to the two models based on the input historical working data; the loss function of the vectorization base model during training is the sum of a first loss function and a second loss function; the first loss function is a first degree of difference between the first vector prediction data and the predicted associated data of one of the two models; the first vector prediction data is the output prediction data of the first output vector representation of the vectorization base model after processing the historical working data, after being input into one of the two models; the second loss function is a second degree of difference between the second vector prediction data and the predicted associated data of the other of the two models; the second vector prediction feature is the output prediction data of the second output vector representation of the vectorization base model after processing the historical working data, after being input into the other of the two models.

[0088] As can be seen, through the above optional embodiments, by limiting the joint loss function and training a vectorized model using historical working data, a unified vector semantic representation of multi-dependency models can be achieved, which helps to improve the fusion and consistency of the correlation between models.

[0089] As an optional embodiment, the specific method by which the determination module determines whether new working data has a data conflict includes: The new working data of either of the two business modules is input into the time data dependency model, user data dependency model and business data dependency model corresponding to the two business modules to obtain the output predicted time-related data, predicted user-related data and predicted business-related data. Calculate the feature similarity corresponding to predicted time-related data, predicted user-related data, and predicted business-related data; Determine if the data feature degree is less than the preset similarity threshold. If so, determine that there is a data conflict in the new working data.

[0090] As can be seen, through the above optional embodiments, by inputting new working data of any business module into the corresponding time, user and business data dependency model to obtain predicted related data, calculating the feature similarity of the predicted related data to judge the data conflict situation, realizing accurate conflict detection based on historical dependency prediction, improving the intelligence and timeliness of conflict judgment, and reducing the risk of data error propagation caused by undetected conflicts.

[0091] As an optional embodiment, the specific method by which the determination module calculates the feature similarity corresponding to the predicted time-related data, predicted user-related data, and predicted business-related data includes: Calculate the first data similarity between the predicted time-related data and the new working data of another module in the two business modules; Calculate the second data similarity between the predicted user association data and the new working data of the other module in the two business modules; Calculate the third data similarity between the predictive business-related data and the new working data of another module in the two business modules; The feature similarity is obtained by calculating the weighted average of the first, second, and third data similarities.

[0092] As can be seen, through the above optional embodiments, feature similarity is obtained by calculating the multi-dimensional similarity between the predicted associated data and the new working data of another module, and the new working data with lower feature similarity is identified as the data to be optimized. This achieves accurate positioning of conflicting data based on multi-similarity comparison, improves the accuracy of optimization target selection, and reduces the risk of incorrect optimization objects caused by similarity calculation deviation.

[0093] As an optional embodiment, the optimization module optimizes and corrects the new working data based on the data dependency logic model to obtain a specific method for optimizing the working data, including: Calculate the feature similarity for each new working data point, and identify the new working data with lower feature similarity as the data to be optimized; The prediction dependency model corresponding to the highest of the first, second, and third data similarities of the data to be optimized is determined as the target dependency prediction model; optionally, the target dependency prediction model is a time data dependency model, a user data dependency model, or a business data dependency model. For each data segment in the data to be optimized, identify the associated data in another new working data segment that is related to the data type of that data segment; The associated data is input into the target dependency prediction model to obtain the output prediction data portion; Calculate the data difference between the predicted data portion and the actual data portion; When the data difference is less than the preset difference threshold, the data portion is replaced with the predicted data portion.

[0094] As can be seen, through the above optional embodiments, by determining the target dependency prediction model corresponding to the highest similarity of the data to be optimized, the part of the data to be optimized is used to find related data in another new working data and input into the target model to obtain the predicted data part, and the data is replaced when the difference is less than the threshold, so as to achieve accurate data part optimization and correction based on the target dependency model, improve the consistency and quality of working data, and reduce the risk of business logic conflicts caused by improper correction.

[0095] Example 3 Please see Figure 3 , Figure 3 This is another data optimization system based on cross-module association modeling disclosed in the embodiments of the present invention. Figure 3 The described data optimization system based on cross-module correlation modeling is applied in data processing systems / data processing equipment / data processing servers (wherein, the server includes local processing servers or cloud processing servers). For example... Figure 3 As shown, this data optimization system based on cross-module correlation modeling may include: Memory 301 storing executable program code; Processor 302 coupled to memory 301; The processor 302 calls the executable program code stored in the memory 301 to execute the steps of the data optimization method based on cross-module association modeling described in Embodiment 1.

[0096] Example 4 This invention discloses a computer read storage medium that stores a computer program for electronic data interchange, wherein the computer program causes a computer to execute the steps of the data optimization method based on cross-module association modeling described in Embodiment 1.

[0097] Example 5 This invention discloses a computer program product, which includes a non-transitory computer-readable storage medium storing a computer program, and the computer program is operable to cause a computer to perform the steps of the data optimization method based on cross-module association modeling described in Embodiment 1.

[0098] The foregoing has described specific embodiments of this specification; other embodiments are within the scope of the appended claims. In some cases, the actions or steps described in the claims may be performed in a different order than those shown in the embodiments and may still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily have to follow the specific or sequential order shown to achieve the desired result. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0099] The systems, devices, modules, or units described in the above embodiments can be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, a computer can be, for example, a personal computer, laptop computer, cellular phone, camera phone, smartphone, personal digital assistant, media player, navigation device, email device, game console, tablet computer, wearable device, or any combination of these devices.

[0100] For ease of description, the above devices are described in terms of function, divided into various units. Of course, in implementing this specification, the functions of each unit can be implemented in one or more software and / or hardware components.

[0101] Those skilled in the art will understand that the embodiments of this specification can be provided as methods, systems, or computer program products. Therefore, the embodiments of this specification can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the embodiments of this specification can take the form of a computer program product implemented on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0102] This specification is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this specification. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create a machine for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0103] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0104] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0105] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0106] Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0107] Computer-readable media include both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0108] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0109] This specification can be described in the general context of computer-executable instructions that are executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform a specific task or implement a specific abstract data type. This specification can also be practiced in distributed computing environments, where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0110] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to interchangeably. Each embodiment focuses on describing the differences from other embodiments. In particular, the system embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments.

[0111] Finally, it should be noted that the data optimization method and system based on cross-module association modeling disclosed in the embodiments of the present invention are merely preferred embodiments of the present invention and are only used to illustrate the technical solutions of the present invention, not to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A data optimization method based on cross-module association modeling, characterized in that, The method includes: Obtain historical work data from multiple business modules of the project system; Based on the historical work data, establish a data dependency logic model between any two of the business modules; In response to new work data generated by any two of the aforementioned business modules, determine whether there is a data conflict in the new work data; In the event of data conflicts, the new working data is optimized and corrected based on the data dependency logic model to obtain optimized working data.

2. The data optimization method based on cross-module association modeling according to claim 1, characterized in that, The historical work data or the new work data includes at least one of process data, business data, and status change data.

3. The data optimization method based on cross-module association modeling according to claim 1, characterized in that, The step of establishing a data dependency logic model between any two business modules based on the historical work data includes: For any two business modules, the data belonging to the same time period in the historical work data corresponding to the two business modules are input into a preset association feature model for training until convergence, so as to obtain the time data dependency model corresponding to the two business modules. Data belonging to the same user type from the historical work data corresponding to the two business models are input into a preset association feature model for training until convergence, so as to obtain the user data dependency model corresponding to the two business modules. Data belonging to the same business type from the historical work data corresponding to the two business models are input into a preset association feature model for training until convergence, so as to obtain the business data dependency model corresponding to the two business modules. The time data dependency model, the user data dependency model, and the business data dependency model are determined as the data dependency logic models corresponding to the two business modules.

4. The data optimization method based on cross-module association modeling according to claim 3, characterized in that, The step of determining the time data dependency model, the user data dependency model, and the business data dependency model as the data dependency logic models corresponding to the two business modules includes: For any two models among the time data dependency model, the user data dependency model, and the business data dependency model, a vectorized base model is established between the two models. Input the historical working data corresponding to any of the aforementioned business modules into the two models to obtain the predicted correlation data output separately; The historical working data and the predicted correlation data of the two models are used as training data and input into the vectorized base model for training until convergence, thus obtaining the vectorized model between the two models. The time data dependency model, the user data dependency model, and the business data dependency model, along with the corresponding vectorized model, are determined as the data dependency logic models corresponding to the two business modules.

5. The data optimization method based on cross-module association modeling according to claim 4, characterized in that, The vectorization model is used to simultaneously output vector representations with semantically similar features to the two models based on the input historical working data; the loss function of the vectorization base model during training is the sum of a first loss function and a second loss function; the first loss function is the first difference between the first vector prediction data and the predicted associated data of one of the two models; the first vector prediction data is the output prediction data of the first output vector representation of the vectorization base model after processing the historical working data and inputting it into one of the two models; the second loss function is the second difference between the second vector prediction data and the predicted associated data of the other of the two models; the second vector prediction feature is the output prediction data of the second output vector representation of the vectorization base model after processing the historical working data and inputting it into the other of the two models.

6. The data optimization method based on cross-module association modeling according to claim 5, characterized in that, The determination of whether the new working data has data conflicts includes: The new working data of either of the two business modules is input into the time data dependency model, the user data dependency model and the business data dependency model corresponding to the two business modules to obtain the output predicted time-related data, predicted user-related data and predicted business-related data; Calculate the feature similarity corresponding to the predicted time-related data, predicted user-related data, and predicted business-related data; Determine whether the data feature degree is less than a preset similarity threshold. If so, determine that the new working data has a data conflict.

7. The data optimization method based on cross-module association modeling according to claim 6, characterized in that, The calculation of the feature similarity corresponding to the predicted time-related data, predicted user-related data, and predicted business-related data includes: Calculate the first data similarity between the predicted time-related data and the new working data of the other module in the two business modules; Calculate a second data similarity between the predicted user association data and the new working data of the other module in the two business modules; Calculate the third data similarity between the predicted business-related data and the new working data of the other module in the two business modules; The feature similarity is obtained by calculating the weighted sum of the first data similarity, the second data similarity, and the third data similarity.

8. The data optimization method based on cross-module association modeling according to claim 7, characterized in that, The optimization and correction of the new working data based on the data dependency logic model to obtain optimized working data includes: Calculate the feature similarity for each of the new working data, and identify the new working data with lower feature similarity as the data to be optimized; The prediction dependency model corresponding to the highest of the first data similarity, the second data similarity, and the third data similarity for the data to be optimized is determined as the target dependency prediction model; the target dependency prediction model is the time data dependency model, the user data dependency model, or the business data dependency model; For each data portion in the data to be optimized, identify associated data in another set of new working data that is related to the data type of that data portion; The associated data is input into the target dependency prediction model to obtain the output prediction data portion; Calculate the data difference between the predicted data portion and the data portion; When the data difference is less than a preset difference threshold, the data portion is replaced with the predicted data portion.

9. A data optimization system based on cross-module association modeling, characterized in that, The system includes: The acquisition module is used to acquire historical work data from multiple business modules of the project system. A module is established to create a data dependency logic model between any two of the business modules based on the historical work data. The judgment module is used to determine whether there is a data conflict in the new working data generated by any two of the business modules. The optimization module is used to optimize and correct the new working data based on the data dependency logic model when data conflicts exist, so as to obtain optimized working data.

10. A data optimization system based on cross-module association modeling, characterized in that, The system includes: Memory containing executable program code; A processor coupled to the memory; The processor calls the executable program code stored in the memory to execute the data optimization method based on cross-module association modeling as described in any one of claims 1-8.

Images