AI-based enterprise business data processing system and method

Abstract

The invention relates to the field of enterprise business management, in particular to an AI-based enterprise business data processing system and method, and the method comprises the steps: 1, obtaining source system data, and storing the obtained data in one or more target systems; step 2, constructing a data synchronization strategy, and monitoring and managing data synchronization between the target system databases based on the data synchronization strategy: generating the data synchronization strategy based on the data synchronization cost; after the importance of the data and the data synchronization task is quantified, the data and the data synchronization task are combined with the data synchronization cost, and a data synchronization strategy is optimized; an adjustment rule is added in the optimized data synchronization strategy, and the importance of data or tasks is dynamically adjusted; and step 3, sending the data of the one or more target systems to the cloud platform for storage.

Description

Technical Field

[0001] The present invention relates to the field of enterprise business management, and in particular to an enterprise business data processing system and method based on AI. Background Art

[0002] In today's digital age, enterprise business data, as the core asset of enterprise operation and development, covers a wide range. Enterprise business data usually includes basic data of the enterprise (such as the company's registered address, registered capital, business scope, organizational structure, etc.), asset information (such as detailed information on various types of assets such as fixed assets and current assets of the enterprise), business operation information (including customer information and customer behavior information), financial data (such as the company's income, expenditure, profit, assets and liabilities, etc.), supply chain data, etc.

[0003] In enterprise business data processing, common methods include data collection and integration, data storage and management, and data analysis and mining. Common enterprise business systems include enterprise resource planning (ERP) systems, customer relationship management (CRM) systems, and big data processing platforms such as Hadoop and Flink.

[0004] In the process of enterprise business processing, data synchronization between different systems is an urgent problem to be solved. Since enterprises usually use multiple different business systems to support their respective business functions, these systems are often developed independently and use different data storage solutions. Summary of the invention

[0005] The present invention realizes data synchronization between a source system (CRM) and different target systems (ERP, Hadoop) and data synchronization between different target systems by constructing a data synchronization strategy, and ensures data consistency during the data synchronization process.

[0006] The technical solution proposed by the present invention is: an enterprise business data processing method based on AI, the method comprising:

[0007] Step 1: Obtain source system data and store the acquired data in one or more target systems;

[0008] Step 2: Build a data synchronization strategy to monitor and manage data synchronization between target system databases based on the data synchronization strategy, including:

[0009] Generate a data synchronization strategy based on data synchronization costs;

[0010] After quantifying the importance of data and data synchronization tasks, we combine this with the data synchronization cost to optimize the data synchronization strategy.

[0011] Add adjustment rules to the optimized data synchronization strategy to dynamically adjust the importance of data or tasks;

[0012] Step 3: Send the data of one or more target systems to the cloud platform for storage.

[0013] Preferably, the source system includes a customer relationship management system (CRM) and a business intelligence (BI) system, and the target system includes an enterprise resource planning (ERP) system and a big data processing (Hadoop) system;

[0014] Before obtaining the source system data and storing the obtained data in the target system database, it also includes:

[0015] Determine whether the target system database is connected to the source system;

[0016] Determine the consistency of data synchronized from the source system to different target systems;

[0017] The step of determining whether the target system database is connected to the source system includes the following steps:

[0018] Modify one or more fields in the source system and check whether the corresponding fields in the target system have changed. If yes, proceed to step 2. Otherwise, proceed to the following steps:

[0019] Modify one or more fields in the target system and check whether the corresponding fields in the source system are automatically synchronized. If yes, proceed to step 2. Otherwise, proceed to the following steps:

[0020] Sample and export the target system and source system data for comparison to determine whether they are consistent. If yes, proceed to step 2. Otherwise, proceed to the following steps:

[0021] Check the API call log of the target system to confirm whether there is an exchange record in the source system. If yes, proceed to step 2. Otherwise, automatically connect the target system database with the source system. Specifically, the following steps are included:

[0022] Determine the data fields that need to be synchronized;

[0023] Identify sensitive fields in synchronized data fields, encrypt or hash sensitive fields, and create independent indexes for high-frequency query fields;

[0024] Call the API interface of the target system to connect with the source system, or use ETL tools to extract data from the source system and load it into the ERP system after cleaning;

[0025] According to the preset monitoring frequency, regularly audit the access rights of the API interface and remove redundant account information.

[0026] Preferably, determining the consistency of data synchronized from the source system to different target systems includes:

[0027] Acquire multiple data to be verified from different target systems, and divide them into blocks according to the same rule to obtain multiple corresponding data blocks 1 and data blocks 2;

[0028] Calculate and obtain the hash value 1 of each data block 1 and the hash value 2 of each data block 2;

[0029] Construct Merkle tree one based on hash value one, and construct Merkle tree two based on hash value two;

[0030] Obtain the root hash value of Merkle tree 1 and the root hash value of Merkle tree 2 and compare them. If there is a difference, find the corresponding data block 1 and data block 2, and use data block 1 as the basis to overwrite the corresponding data block 2.

[0031] If data block 2 cannot be automatically overwritten, the difference data block 1 and data block 2 are pushed to the host computer for manual processing.

[0032] Preferably, generating a data synchronization strategy based on data synchronization cost includes:

[0033] Build a dynamic adjustment model to dynamically adjust the data volume ratio between full data and incremental data. Specifically:

[0034] Constructing the state matrix ,in, Indicates the data change rate, Indicates the full data synchronization cost. represents the incremental data synchronization cost, Indicates the average time taken for historical synchronization;

[0035] Constructing the weight matrix , the first row elements of the weight matrix represent the decision vectors corresponding to the full data synchronization; the second row elements of the weight matrix represent the decision vectors corresponding to the incremental data synchronization;

[0036] Generate synchronization strategy ;in and Indicates the proportion of data synchronized in full and incremen- tally in sync.

[0037] Optimize the weight matrix by gradient descent to minimize the synchronization cost , where the gradient calculation process is:

[0038] ;

[0039] The updated weight matrix is ;

[0040] when When the full data synchronization is triggered; otherwise, Proportional synchronous incremental data; where, Indicates the full synchronization threshold.

[0041] Preferably, after quantifying the importance of data and data synchronization tasks, the data synchronization strategy is optimized in combination with the data synchronization cost, including:

[0042] Extracting data importance and data synchronization task importance from metadata of a target system, wherein the data importance is the weight of the impact of the extracted data on the system business based on the preset business rules of the target system, and the data synchronization task importance is the priority of the task defined based on the type of synchronization task, and the task type includes real-time report task and offline analysis task;

[0043] Specifically: Based on the rating of the customer corresponding to the data, the importance of the data is associated with the rating, and the importance of the data is quantified using the customer rating. ; Based on the priority of data synchronization tasks, the importance of data synchronization tasks is associated with the priority, and the importance of data synchronization tasks is quantified by the priority ;

[0044] Will and Add to In the new state matrix ;

[0045] Building a new synchronization cost ,in, represents the penalty coefficient, Indicates the delay time when synchronizing incremental data;

[0046] Update the weight matrix by gradient descent to minimize ; The gradient calculation process is: ;in, , ;

[0047] Optimized synchronization strategy: ;

[0048] when When the data is fully synchronized, ;in, represents the task importance threshold of one;

[0049] when When , incremental data synchronization delay is allowed. Indicates the task importance threshold 2.

[0050] Preferably, an adjustment rule is added to the optimized data synchronization strategy to dynamically adjust the importance of data or tasks, including:

[0051] Based on customer behavior, transaction data and system business rules, quantify customer value and formulate dynamic adjustment rules for customer ratings, which specifically include the following steps:

[0052] Obtain customer behavior, transaction data, and system business rule data from the target system database to form a customer data set;

[0053] Extract key features from customer data sets to form customer key feature sets ,Real-time key features include customers’ cumulative transaction amount, maximum transaction amount, activity, and repurchase rate;

[0054] After normalizing the customer's key feature set, input it into the time series model ARIMA to predict the customer's consumption amount in the next 6 months;

[0055] Build a customer value scoring model:

[0056] ,in, They represent the scoring weight coefficients respectively; Indicates the maximum transaction amount of the customer; Indicates the cumulative transaction amount; Represents the repurchase rate; Indicates the predicted value of the customer's consumption in the next 6 months; represents the attenuation coefficient, ;

[0057] Set customer value rating category range , When When the customer is judged as a low-value customer;

[0058] Based on the type of data synchronization task, timeliness requirements, and the value of associated data, a dynamic adjustment rule for the task priority is formulated, which specifically includes the following steps:

[0059] According to the purpose of data, the types of data synchronization tasks are divided into real-time risk control, real-time reporting, batch analysis and historical archiving;

[0060] Assign task weights to different types of data synchronization tasks ;

[0061] Build a task importance scoring model:

[0062] ;in, represents the task weight vector, ; Indicates that the task has been delayed. represents the adjustment coefficient; Indicates the importance of data. ;

[0063] Set the task importance rating classification interval , When , it is judged as the highest priority task; When , the synchronization task is judged as a low priority task;

[0064] Set adjustment rules:

[0065] When a customer's daily transaction volume is more than five times the average of the previous six months, the customer value score is upgraded; at the same time, the priority of the task associated with the customer is upgraded;

[0066] When the customer has no transaction data for 3 consecutive months, and Continuous decline will trigger a downgrade of the customer value score and reduce the priority of tasks associated with the customer;

[0067] Specifically: transmit customer transaction data through Kafka, call the customer value scoring model through Flink, calculate the customer value score, and write the customer value score to Redis for real-time query by the system;

[0068] Periodically train the ARIMA time series model through Spark to update the Hive table.

[0069] Preferably, the step of adding adjustment rules to the optimized data synchronization strategy to dynamically adjust the importance of data or tasks also includes:

[0070] Add a temporary importance adjustment mechanism to temporarily increase or decrease the importance of data or tasks. Specifically:

[0071] Adding a temporary adjustment table in a target system database, wherein the temporary adjustment table includes an adjustment record ID, an adjustment object type, a customer ID, a synchronization task ID, importance before adjustment, importance after adjustment, effective time, expiration time, operator ID, and adjustment reason;

[0072] The adjustment mechanism is: the current effective importance value is ;in, A quantitative value indicating the importance of the current effect. represents the adjusted importance, represents the output of the task importance scoring model or the customer value scoring model, Indicates the current time. Indicates the effective time of the adjustment. represents the adjusted expiration time;

[0073] Use the priority override middleware to override the importance before adjustment with the adjusted importance;

[0074] According to the preset scanning frequency, the overdue adjustments are automatically scanned and recorded, and the temporary importance adjustment mechanism is automatically stopped after the notification information is triggered.

[0075] Preferably, the step of overwriting the importance before adjustment with the adjusted importance through the priority overwriting middleware further includes: integrating the customer importance score, the task importance score, the data importance and the temporary adjustment record to generate a comprehensive importance, and generating a final adjusted importance based on the comprehensive importance and the adjusted importance in the temporary adjustment table, wherein the specific steps are as follows:

[0076] Obtain customer importance scores based on the customer value scoring model ;

[0077] Obtaining task importance scores based on the task importance scoring model ;

[0078] Obtaining data importance and the quantitative value of adjusted importance ;

[0079] Will , , and Perform normalization processing;

[0080] Overall importance ;

[0081] in, represents the resource weight coefficient; where, express , Normalized value.

[0082] An AI-based enterprise business data processing system includes: a processor and a memory and a communication module connected to the processor, and the system is used to execute the AI-based enterprise business data processing method.

[0083] A computer-readable storage medium stores a computer program, and the computer program is executed by a processor to implement the AI-based enterprise business data processing method.

[0084] Beneficial effects of the present invention:

[0085] 1. The present invention first determines whether the customer data of the source system (CRM system) is connected with the target system (ERP system), and achieves safe connection if it is not connected, ensuring that the customer data is in different data.

[0086] 2. After solving the problem of connecting customer data with the target system, the present invention designs a data synchronization strategy to keep different data storage solutions consistent. For example, while synchronizing from the source system to the database of the ERP system, some data or analysis is stored in Hadoop. The data synchronization strategy is used to ensure data consistency between the ERP system database and the Hadoop storage solution.

[0087] 3. The present invention dynamically adjusts the model, adds adjustment rules in the data synchronization process, adjusts the importance of data or tasks, and dynamically adjusts the ratio of full data to incremental data in the data synchronization process according to the importance of data and tasks. BRIEF DESCRIPTION OF THE DRAWINGS

[0088] Figure 1 This is a flow chart of an enterprise business data processing method based on AI of the present invention. DETAILED DESCRIPTION

[0089] The following description is used to disclose the present invention so that those skilled in the art can implement the present invention. The preferred embodiments described below are only examples, and those skilled in the art can think of other obvious variations. The basic principles of the present invention defined in the following description can be applied to other embodiments, variations, improvements, equivalents, and other technical solutions that do not deviate from the spirit and scope of the present invention.

[0090] It is to be understood that the term "one" should be understood as "at least one" or "one or more", that is, in one embodiment, the number of an element may be one, while in another embodiment, the number of the element may be multiple, and the term "one" should not be understood as a limitation on the quantity.

[0091] Embodiment 1:

[0092] refer to Figure 1 The technical solution provided by the present invention is: an enterprise business data processing method based on AI, comprising the following steps:

[0093] Step 1: Acquire source system data and store the acquired data in one or more target systems; the source system includes a customer relationship management system (CRM) and a business intelligence (BI) system, and the target system includes an enterprise resource planning (ERP) system and a big data processing (Hadoop) system;

[0094] Step 2: Build a data synchronization strategy to monitor and manage data synchronization between target system databases based on the data synchronization strategy, including the following sub-steps:

[0095] Step 2.1: Generate a data synchronization strategy based on the data synchronization cost. This includes the following sub-steps:

[0096] Build a dynamic adjustment model to dynamically adjust the data volume ratio between full data and incremental data. Specifically:

[0097] Constructing the state matrix ,in, Indicates the data change rate, Indicates the full data synchronization cost, represents the incremental data synchronization cost, Indicates the average time taken for historical synchronization.

[0098] The data change rate is the percentage of newly added or modified records in a unit of time to the total number of records. The number of changed records can be counted from the database log. The full data synchronization cost is the time, bandwidth, and computing resources (CPU and memory usage) consumed during full synchronization. The incremental data synchronization cost is the resource consumption (CPU and memory usage) of the CDC tool. The average time consumption of historical synchronization is the average time consumption of the past N synchronization tasks.

[0099] Constructing the weight matrix , the first row of elements of the weight matrix represents the decision vector corresponding to the full data synchronization; the second row of elements of the weight matrix represents the decision vector corresponding to the incremental data synchronization;

[0100] Generate synchronization strategy ;in and Indicates the proportion of data synchronized in full and incremen- tally in sync.

[0101] Optimize the weight matrix by gradient descent to minimize the synchronization cost , where the gradient calculation process is:

[0102] ;

[0103] The updated weight matrix is ;

[0104] when When the full data synchronization is triggered; otherwise, Proportional synchronous incremental data; where, Indicates the full synchronization threshold.

[0105] For example, in this embodiment, the state vector in the current state is ;

[0106] The weight matrix is , ;

[0107] After Softmax normalization, we get , ;

[0108] The synchronization strategy is to use full synchronization for 52% of the synchronized data volume and incremental synchronization for 48%.

[0109] Step 2.2: After quantifying the importance of data and data synchronization tasks, combine them with the data synchronization cost and optimize the data synchronization strategy. This includes the following sub-steps:

[0110] Extracting data importance and data synchronization task importance from metadata of a target system, wherein the data importance is the weight of the impact of the extracted data on the system business based on the preset business rules of the target system, and the data synchronization task importance is the priority of the task defined based on the type of synchronization task, and the task type includes real-time report task and offline analysis task;

[0111] Specifically: Based on the rating of the customer corresponding to the data, the importance of the data is associated with the rating, and the importance of the data is quantified using the customer rating. ; Based on the priority of data synchronization tasks, the importance of data synchronization tasks is associated with the priority, and the importance of data synchronization tasks is quantified by the priority ;

[0112] Will and Add to In the new state matrix ;

[0113] Building a new synchronization cost ,in, represents the penalty coefficient, Indicates the delay time when synchronizing incremental data;

[0114] Update the weight matrix by gradient descent to minimize ; The gradient calculation process is: ;

[0115] in, , ;

[0116] Optimized synchronization strategy: ;

[0117] when When the data is fully synchronized, ;in, represents the task importance threshold of one;

[0118] when When , incremental data synchronization delay is allowed. Indicates the task importance threshold 2.

[0119] For example, in this embodiment, the two target systems are the ERP system and the Hadoop system; after the data of the CRP system (source system) is connected to the ERP system, the data needs to be synchronized to Hadoop for backup.

[0120] Now we need to synchronize the order data of VIP customers. , the type of synchronization task is order data, and the priority of the order data task is set to 3, that is, ;

[0121] when , , , hour,

[0122] ;

[0123] Optimal solution: When hour, ;when hour, Because the data of VIP customers is of high importance, the system still chooses full synchronization even if the cost of full synchronization is slightly higher (to avoid the risk of a 75-unit penalty).

[0124] Step 2.3: Add adjustment rules to the optimized data synchronization strategy to dynamically adjust the importance of data or tasks. This includes the following sub-steps:

[0125] Based on customer behavior, transaction data and system business rules, quantify customer value and formulate dynamic adjustment rules for customer ratings, which specifically include the following steps:

[0126] Obtain customer behavior, transaction data, and system business rule data from the target system database to form a customer data set;

[0127] Extract key features from customer data sets to form customer key feature sets ,Real-time key features include customers’ cumulative transaction amount, maximum transaction amount, activity, and repurchase rate;

[0128] After normalizing the customer's key feature set, input it into the time series model ARIMA to predict the customer's consumption amount in the next 6 months;

[0129] Build a customer value scoring model:

[0130] ,in, They represent the scoring weight coefficients respectively; Indicates the maximum transaction amount of the customer; Indicates the cumulative transaction amount; Represents the repurchase rate; Indicates the predicted value of the customer's consumption in the next 6 months; represents the attenuation coefficient, ;

[0131] Set customer value rating category range , When When the customer is judged as a low-value customer;

[0132] Based on the type of data synchronization task, timeliness requirements, and the value of associated data, a dynamic adjustment rule for the task priority is formulated, which specifically includes the following steps:

[0133] According to the purpose of data, the types of data synchronization tasks are divided into real-time risk control, real-time reporting, batch analysis and historical archiving;

[0134] Assign task weights to different types of data synchronization tasks ;

[0135] Build a task importance scoring model:

[0136] ;in, represents the task weight vector, ; Indicates that the task has been delayed. represents the adjustment coefficient; Indicates the importance of data. ;

[0137] Set the task importance rating classification interval , When , it is judged as the highest priority task; When , the synchronization task is judged as a low priority task;

[0138] Set adjustment rules:

[0139] When a customer's daily transaction volume is more than five times the average of the previous six months, the customer value score is upgraded; at the same time, the priority of the task associated with the customer is upgraded;

[0140] When the customer has no transaction data for 3 consecutive months, and Continuous decline will trigger a downgrade of the customer value score and reduce the priority of tasks associated with the customer;

[0141] Specifically: transmit customer transaction data through Kafka, call the customer value scoring model through Flink, calculate the customer value score, and write the customer value score to Redis for real-time query by the system;

[0142] Periodically train the ARIMA time series model through Spark to update the Hive table.

[0143] Step 3: Store the data of one or more target systems on the cloud platform.

[0144] Before proceeding to step 1, the following steps are also included:

[0145] Determine whether the target system database is connected to the source system and determine the consistency of data synchronized from the source system to different target systems.

[0146] Among them, judging whether the target system database is connected with the source system includes the following steps:

[0147] Modify one or more fields in the source system and check whether the corresponding fields in the target system have changed. If yes, proceed to step 2. Otherwise, proceed to the following steps:

[0148] Modify one or more fields in the target system and check whether the corresponding fields in the source system are automatically synchronized. If yes, proceed to step 2. Otherwise, proceed to the following steps:

[0149] Sample and export the target system and source system data for comparison to determine whether they are consistent. If yes, proceed to step 2. Otherwise, proceed to the following steps:

[0150] Check the API call log of the target system to confirm whether there is an exchange record in the source system. If yes, proceed to step 2. Otherwise, automatically connect the target system database with the source system. Specifically, the following steps are included:

[0151] Determine the data fields that need to be synchronized;

[0152] Identify sensitive fields in synchronized data fields, encrypt or hash sensitive fields, and create independent indexes for high-frequency query fields;

[0153] Call the API interface of the target system to connect with the source system, or use ETL tools to extract data from the source system and load it into the ERP system after cleaning;

[0154] According to the preset monitoring frequency, regularly audit the access rights of the API interface and remove redundant account information.

[0155] Among them, judging the consistency of data synchronized from the source system to different target systems includes the following steps:

[0156] Acquire multiple data to be verified from different target systems, and divide them into blocks according to the same rule to obtain multiple corresponding data blocks 1 and data blocks 2;

[0157] Calculate and obtain the hash value 1 of each data block 1 and the hash value 2 of each data block 2;

[0158] Construct Merkle tree one based on hash value one, and construct Merkle tree two based on hash value two;

[0159] Obtain the root hash value of Merkle tree 1 and the root hash value of Merkle tree 2 and compare them. If there is a difference, find the corresponding data block 1 and data block 2, and use data block 1 as the basis to overwrite the corresponding data block 2.

[0160] If data block 2 cannot be automatically overwritten, the difference data block 1 and data block 2 are pushed to the host computer for manual processing.

[0161] Embodiment 2:

[0162] This embodiment adds adjustment rules based on the first embodiment when dynamically adjusting the importance of data or tasks. Specifically, it includes the following steps:

[0163] Add a temporary importance adjustment mechanism to temporarily increase or decrease the importance of data or tasks. Specifically:

[0164] Adding a temporary adjustment table in a target system database, wherein the temporary adjustment table includes an adjustment record ID, an adjustment object type, a customer ID, a synchronization task ID, importance before adjustment, importance after adjustment, effective time, expiration time, operator ID, and adjustment reason;

[0165] The adjustment mechanism is: the current effective importance value is ;in, A quantitative value indicating the importance of the current effect. represents the adjusted importance, represents the output of the task importance scoring model or the customer value scoring model, Indicates the current time. Indicates the effective time of the adjustment. represents the adjusted expiration time;

[0166] Use the priority override middleware to override the importance before adjustment with the adjusted importance;

[0167] According to the preset scanning frequency, the overdue adjustments are automatically scanned and recorded, and the temporary importance adjustment mechanism is automatically stopped after the notification information is triggered.

[0168] In this embodiment, the temporary importance adjustment mechanism works in conjunction with the dynamic adjustment model, allowing the business personnel of the enterprise to quickly respond to changes in duration and temporarily adjust the importance of data or tasks.

[0169] Embodiment three:

[0170] Based on the second embodiment, this embodiment first integrates the customer importance score, task importance score, data importance and temporary adjustment record to generate a comprehensive importance, and then compares it with the adjusted importance in the temporary adjustment table to generate the final adjusted importance. This achieves the effect of managing customer importance score, task importance score, data importance and temporary adjustment record through priority coverage middleware. Specifically, the following steps are included:

[0171] Obtain customer importance scores based on the customer value scoring model ;

[0172] Obtaining task importance scores based on the task importance scoring model ;

[0173] Obtaining data importance and the quantitative value of adjusted importance ;

[0174] Will , , and Perform normalization processing;

[0175] Overall importance ;

[0176] in, represents the resource weight coefficient; where, express , The normalized value,

[0177] if , then use Forced coverage , otherwise use .

[0178] For example, take the slicing task of 5G network as an example:

[0179] The input variables are , , , in effect ; The values ​​are 0.3, 0.3, 0.2, and 0.2 respectively;

[0180] Overall importance ;

[0181] because ,use Forced coverage , , and the final adjusted importance is 0.9.

[0182] If the adjusted importance is the importance of the data, use the adjusted importance of the data instead of the importance of the data quantified based on customer ratings , forming a new state matrix, and finally obtaining the data ratio of full synchronization and the data ratio of incremental synchronization. According to the data ratio of full synchronization and the data ratio of incremental synchronization, the data synchronization process is controlled.

[0183] The present invention also provides an AI-based enterprise business data processing system, comprising: a processor and a memory and a communication module connected to the processor, and the system is used to execute the AI-based enterprise business data processing method.

[0184] The present invention also provides a computer-readable storage medium, which stores a computer program, and the computer program is executed by a processor to implement the AI-based enterprise business data processing method.

[0185] In the embodiments disclosed in the present invention, the process described above with reference to the flowchart can be implemented as a computer software program. The embodiments disclosed in the present invention include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program contains a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication part, and / or installed from a removable medium. When the computer program is executed by the central processing unit (CPU), the above functions defined in the method of the present application are executed. It should be noted that the computer-readable medium mentioned above in the present application can be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to, an electrical connection with one or more wire segments, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present application, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In the present application, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, which carries a computer-readable program code. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device. The program code embodied on the computer readable medium may be transmitted using any appropriate medium, including but not limited to wireless, electrical wire, optical cable, RF, etc., or any suitable combination of the foregoing.

[0186] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present invention. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a part of a code, and the module, a program segment or a part of a code contains one or more executable instructions for implementing the specified logical functions. It should also be noted that in some alternative implementations, the functions marked in the boxes can also occur in a different order than the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of boxes in the block diagram and / or flowchart, can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0187] Those skilled in the art should understand that the embodiments of the present invention described above and shown in the accompanying drawings are only examples and do not limit the present invention. The objectives of the present invention have been fully and effectively achieved. The functions and structural principles of the present invention have been demonstrated and explained in the embodiments. Without departing from the principles, the implementation methods of the present invention may be subject to any changes or modifications.

Claims

1. An enterprise business data processing method based on AI, characterized in that: The method comprises: Step 1: Obtain source system data and store the acquired data in one or more target systems; Step 2: Build a data synchronization strategy to monitor and manage data synchronization between target system databases based on the data synchronization strategy, including: Generate a data synchronization strategy based on data synchronization costs; After quantifying the importance of data and data synchronization tasks, we combine this with the data synchronization cost to optimize the data synchronization strategy. Add adjustment rules to the optimized data synchronization strategy to dynamically adjust the importance of data or tasks; Step 3: Store the data of one or more target systems on the cloud platform.

2. The method for processing enterprise business data based on AI according to claim 1, characterized in that: The source system includes a customer relationship management system (CRM) and a business intelligence (BI) system, and the target system includes an enterprise resource planning (ERP) system and a big data processing (Hadoop) system. Before obtaining the source system data and storing the obtained data in the target system database, it also includes: Determine whether the target system database is connected to the source system; Determine the consistency of data synchronized from the source system to different target systems; The step of determining whether the target system database is connected to the source system includes the following steps: Modify one or more fields in the source system and check whether the corresponding fields in the target system have changed. If yes, proceed to step 2. Otherwise, proceed to the following steps: Modify one or more fields in the target system and check whether the corresponding fields in the source system are automatically synchronized. If yes, proceed to step 2. Otherwise, proceed to the following steps: Sample and export the target system and source system data for comparison to determine whether they are consistent. If yes, proceed to step 2. Otherwise, proceed to the following steps: Check the API call log of the target system to confirm whether there is an exchange record in the source system. If yes, proceed to step 2. Otherwise, automatically connect the target system database with the source system. Specifically, the following steps are included: Determine the data fields that need to be synchronized; Identify sensitive fields in synchronized data fields, encrypt or hash sensitive fields, and create independent indexes for high-frequency query fields; Call the API interface of the target system to connect with the source system, or use ETL tools to extract data from the source system and load it into the ERP system after cleaning; According to the preset monitoring frequency, regularly audit the access rights of the API interface and remove redundant account information.

3. The enterprise business data processing method based on AI according to claim 2 is characterized in that: Determining the consistency of data synchronized from the source system to different target systems includes: Acquire multiple data to be verified from different target systems, and divide them into blocks according to the same rule to obtain multiple corresponding data blocks 1 and data blocks 2; Calculate and obtain the hash value 1 of each data block 1 and the hash value 2 of each data block 2; Construct Merkle tree one based on hash value one, and construct Merkle tree two based on hash value two; Obtain the root hash value of Merkle tree 1 and the root hash value of Merkle tree 2 and compare them. If there is a difference, find the corresponding data block 1 and data block 2, and use data block 1 as the basis to overwrite the corresponding data block 2. If data block 2 cannot be automatically overwritten, the difference data block 1 and data block 2 are pushed to the host computer for manual processing.

4. The method for processing enterprise business data based on AI according to claim 3, characterized in that: The generating of the data synchronization strategy based on the data synchronization cost includes: Build a dynamic adjustment model to dynamically adjust the data volume ratio between full data and incremental data. Specifically: Constructing the state matrix ,in, Indicates the data change rate, Indicates the full data synchronization cost, represents the incremental data synchronization cost, Indicates the average time taken for historical synchronization; Constructing the weight matrix , the first row elements of the weight matrix represent the decision vectors corresponding to the full data synchronization; the second row elements of the weight matrix represent the decision vectors corresponding to the incremental data synchronization; Generate synchronization strategy ;in and Indicates the proportion of data synchronized in full and incremen- tally in sync. Optimize the weight matrix by gradient descent to minimize the synchronization cost , where the gradient calculation process is: ； The updated weight matrix is ; when When the full data synchronization is triggered; otherwise, Proportional synchronous incremental data; where, Indicates the full synchronization threshold.

5. The method for processing enterprise business data based on AI according to claim 4, characterized in that: After the importance of data and data synchronization tasks is quantified, the data synchronization strategy is optimized in combination with the data synchronization cost, including: Extracting data importance and data synchronization task importance from metadata of a target system, wherein the data importance is the weight of the impact of the extracted data on the system business based on the preset business rules of the target system, and the data synchronization task importance is the priority of the task defined based on the type of synchronization task, and the task type includes real-time report task and offline analysis task; Specifically: Based on the rating of the customer corresponding to the data, the importance of the data is associated with the rating, and the importance of the data is quantified using the customer rating. ; Based on the priority of data synchronization tasks, the importance of data synchronization tasks is associated with the priority, and the importance of data synchronization tasks is quantified by the priority ; Will and Add to In the new state matrix ; Building a new synchronization cost ,in, represents the penalty coefficient, Indicates the delay time when synchronizing incremental data; Update the weight matrix by gradient descent to minimize ; The gradient calculation process is: ;in, , ; Optimized synchronization strategy: ; when When the data is fully synchronized, ;in, represents the task importance threshold of one; when When , incremental data synchronization delay is allowed. Indicates the task importance threshold 2.

6. The method for processing enterprise business data based on AI according to claim 5, characterized in that: Add adjustment rules to the optimized data synchronization strategy to dynamically adjust the importance of data or tasks, including: Based on customer behavior, transaction data and system business rules, quantify customer value and formulate dynamic adjustment rules for customer ratings, which specifically include the following steps: Obtain customer behavior, transaction data, and system business rule data from the target system database to form a customer data set; Extract key features from customer data sets to form customer key feature sets ,Real-time key features include customers’ cumulative transaction amount, maximum transaction amount, activity, and repurchase rate; After normalizing the customer's key feature set, input it into the time series model ARIMA to predict the customer's consumption amount in the next 6 months; Build a customer value scoring model: ,in, Respectively represent the scoring weight coefficient; Indicates the maximum transaction amount of the customer; Indicates the cumulative transaction amount; Represents the repurchase rate; Indicates the predicted value of the customer's consumption in the next 6 months; represents the attenuation coefficient, ; Set customer value rating category range , When When the customer is judged as a low-value customer; Based on the type of data synchronization task, timeliness requirements, and the value of associated data, a dynamic adjustment rule for the task priority is formulated, which specifically includes the following steps: According to the purpose of data, the types of data synchronization tasks are divided into real-time risk control, real-time reporting, batch analysis and historical archiving; Assign task weights to different types of data synchronization tasks ; Build a task importance scoring model: ;in, represents the task weight vector, ; Indicates that the task has been delayed. represents the adjustment coefficient; Indicates the importance of data. ; Set the task importance rating classification interval , When , it is judged as the highest priority task; When , the synchronization task is judged as a low priority task; Set adjustment rules: When a customer's daily transaction volume is more than five times the average of the previous six months, the customer value score is upgraded; at the same time, the priority of the task associated with the customer is upgraded; When the customer has no transaction data for 3 consecutive months, and Continuous decline will trigger a downgrade of the customer value score and reduce the priority of tasks associated with the customer; Specifically: transmit customer transaction data through Kafka, call the customer value scoring model through Flink, calculate the customer value score, and write the customer value score to Redis for real-time query by the system; Periodically train the ARIMA time series model through Spark to update the Hive table.

7. The method for processing enterprise business data based on AI according to claim 6, characterized in that: The step of adding adjustment rules to the optimized data synchronization strategy to dynamically adjust the importance of data or tasks also includes: Add a temporary importance adjustment mechanism to temporarily increase or decrease the importance of data or tasks. Specifically: Adding a temporary adjustment table in a target system database, wherein the temporary adjustment table includes an adjustment record ID, an adjustment object type, a customer ID, a synchronization task ID, importance before adjustment, importance after adjustment, effective time, expiration time, operator ID, and adjustment reason; The adjustment mechanism is: the current effective importance value is ;in, A quantitative value indicating the importance of the current effect. represents the adjusted importance, represents the output of the task importance scoring model or the customer value scoring model, Indicates the current time. Indicates the effective time of the adjustment. represents the adjusted expiration time; Use the priority override middleware to override the importance before adjustment with the adjusted importance; According to the preset scanning frequency, the overdue adjustments are automatically scanned and recorded, and the temporary importance adjustment mechanism is automatically stopped after the notification information is triggered.

8. The method for processing enterprise business data based on AI according to claim 7, characterized in that: The method of using the priority overwriting middleware to overwrite the importance before adjustment with the adjusted importance also includes: integrating the customer importance score, the task importance score, the data importance and the temporary adjustment record to generate a comprehensive importance, and generating a final adjusted importance based on the comprehensive importance and the adjusted importance in the temporary adjustment table. The specific steps are as follows: Obtain customer importance scores based on the customer value scoring model ; Obtaining task importance scores based on the task importance scoring model ; Obtaining data importance and the quantitative value of adjusted importance ; Will , , and Perform normalization processing; Overall importance ; in, represents the resource weight coefficient; where, express , The normalized value, if , then use Forced coverage , otherwise use .

9. An enterprise business data processing system based on AI, comprising: A processor and a memory and a communication module connected to the processor, characterized in that the system is used to execute an AI-based enterprise business data processing method as described in any one of claims 1-8 above.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and the computer program is executed by a processor to implement an AI-based enterprise business data processing method as described in any one of claims 1 to 8.

Images