Data storage optimization method and platform for business integration

Abstract

The invention provides a data storage optimization method and platform for business integration, and relates to the technical field of data processing, and the method comprises the steps: carrying out the department architecture extraction of a target enterprise, and obtaining a plurality of business function departments; calling service data to obtain a plurality of pieces of historical service data; service interaction analysis is carried out, and a data integration topology network is constructed and generated; according to the data flow direction, segmenting the plurality of historical business data into a plurality of groups of business data subsets; constructing a plurality of business cooperation judgment models, and mapping and synchronizing the business cooperation judgment models to a plurality of business data uploading nodes; and performing data flow direction prediction on the dynamically uploaded service data, and performing intelligent data routing based on a prediction result to complete data storage updating of the K middleware. The technical problem that in the prior art, data management and storage of different business departments in an enterprise are generally static and isolated, and intelligent and dynamic adjustment and optimization capabilities are lacked, so that the efficiency of the whole business process is low is solved.

Description

Technical Field

[0001] The present invention relates to the technical field of data processing, and particularly to a data storage optimization method and platform for business integration. Background Art

[0002] Enterprises need to process a huge amount of data in the big data era, and often involve different data sources and multi-department collaboration. How to optimize data storage and management and improve the efficiency of data analysis is one of the core issues urgently to be solved in the big data field. Specifically, the data of different business departments is usually stored in their respective independent systems, forming data islands, which leads to difficulties in cross-department data sharing and collaboration, poor information circulation, and affects the efficiency of the overall business process. When dealing with cross-department data integration, the existing technologies usually rely on static rules or manual configuration, lacking in-depth understanding of the internal relationship of data and intelligent analysis capabilities, which makes it difficult to cope with complex and changeable business scenarios, unable to make real-time intelligent judgments and decisions, and affects the flexibility and accuracy of business collaboration. Summary of the Invention

[0003] This application provides a data storage optimization method and platform for business integration, aiming to solve the technical problem that the existing technologies for data management and storage of different business departments within an enterprise are usually static and isolated, lacking the capabilities of intelligence and dynamic optimization, resulting in low efficiency of the overall business process.

[0004] In the first aspect disclosed in this application, a data storage optimization method for business integration is provided. The method includes: extracting the department architecture of the target enterprise to obtain multiple business functional departments; invoking business data of the multiple business functional departments to obtain multiple historical business data; performing business interaction analysis on the multiple historical business data for the multiple business functional departments to construct and generate a data integration topology network, where the data integration topology network includes multiple business data upload nodes mapped to the multiple business functional departments and K middleware, and the K middleware is used for pre-storing business integration data; dividing the multiple historical business data into multiple groups of business data subsets according to the data flow direction of the multiple historical business data, where each group of business data subsets in the multiple groups of business data subsets includes a local data subset and multiple cross-department data subsets; constructing multiple business collaboration judgment models based on the multiple groups of business data subsets, and synchronously mapping the multiple business collaboration judgment models to the multiple business data upload nodes to complete the function enhancement of the data integration topology network; in the data integration topology network, the multiple business data upload nodes predict the data flow direction of the business data dynamically uploaded by the multiple business functional departments through the multiple business collaboration judgment models, and perform intelligent data routing based on the prediction results to complete the data storage update of the K middleware.

[0005] The second aspect disclosed in this application provides a data storage optimization platform for business integration. The platform is used for the above-mentioned data storage optimization method for business integration. The platform includes a departmental architecture extraction module for extracting the departmental architecture of a target enterprise to obtain multiple business functional departments; a business data invocation module for invoking business data from the multiple business functional departments to obtain multiple historical business data; a business interaction analysis module for performing business interaction analysis of the multiple business functional departments based on the multiple historical business data to construct and generate a data integration topology network. Among them, the data integration topology network includes multiple business data upload nodes mapped to the multiple business functional departments and K middleware for pre-storing business integration data; a historical business data segmentation module for segmenting the multiple historical business data into multiple groups of business data subsets according to the data flow direction of the multiple historical business data. Among the multiple groups of business data subsets, each group of business data subsets includes a local data subset and multiple cross-departmental data subsets; a topology network function enhancement module for constructing multiple business collaboration judgment models based on the multiple groups of business data subsets and synchronously mapping the multiple business collaboration judgment models to the multiple business data upload nodes to complete the function enhancement of the data integration topology network; an intelligent data routing module for, in the data integration topology network, the multiple business data upload nodes predicting the data flow direction of the business data dynamically uploaded by the multiple business functional departments through the multiple business collaboration judgment models and performing intelligent data routing based on the prediction results to complete the data storage update of the K middleware.

[0006] One or more technical solutions provided in this application have at least the following technical effects or advantages: By extracting the departmental structure of the target enterprise and invoking the historical business data of each department, it is possible to comprehensively understand the business flow within the enterprise, and then effectively identify the interaction relationships and data dependencies among departments, laying a foundation for subsequent data integration. Based on the historical business data, a data integration topology network is constructed, which effectively reflects the interaction relationships among various business functional departments within the enterprise. By mapping multiple business data upload nodes and K middleware in the network, pre-storage and management of business data are achieved. Such a topological structure not only optimizes the distribution of data storage, but also improves the speed and efficiency of data access, ensuring the efficient flow of data among departments. By analyzing the data flow direction of historical business data, the data is segmented into multiple groups of business data subsets, and on this basis, multiple business collaboration judgment models are constructed, which can deeply analyze the explicit and implicit information of business data and identify the relevance between data. This model-based design enables the system to intelligently judge whether business data needs to be uploaded or flow between different departments, thereby enhancing the collaboration ability and decision-making support among departments. At the data upload node, the business collaboration judgment model predicts the flow direction of the real-time uploaded data, and based on the prediction result, intelligent data routing is performed, ensuring that the data can be transmitted to the appropriate middleware for processing or storage in a timely and accurate manner, thereby optimizing the resource utilization rate of the enterprise.

[0007] The above description is only an overview of the technical solution of this application. In order to be able to more clearly understand the technical means of this application, it can be implemented in accordance with the content of the specification. And in order to make the above and other purposes, features and advantages of this application more obvious and understandable, the following specifically gives the specific implementation manners of this application. Brief Description of the Drawings

[0008] Figure 1 It is a schematic flowchart of a data storage optimization method for business integration provided by an embodiment of this application; Figure 2 It is a schematic structural diagram of a data storage optimization platform for business integration provided by an embodiment of this application.

[0009] Description of the Reference Numerals: Departmental Structure Extraction Module 10, Business Data Invocation Module 20, Business Interaction Analysis Module 30, Historical Business Data Segmentation Module 40, Topological Network Function Enhancement Module 50, Intelligent Data Routing Module 60. Detailed Description of the Preferred Embodiments

[0010] By providing a data storage optimization method and platform for business integration in the embodiments of this application, the technical problem that the data management and storage of different business departments within an enterprise in the prior art are usually static and isolated, lacking the ability of intelligent and dynamic optimization, resulting in low efficiency of the overall business process, is solved.

[0011] After introducing the basic principle of the present application, various non-limiting implementation manners of the present application will be specifically introduced below in conjunction with the accompanying drawings of the specification. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0012] Embodiment 1, as Figure 1 shown, the embodiment of the present application provides a method for optimizing data storage for business integration, and the method includes: Extract the department structure of the target enterprise to obtain multiple business function departments.

[0013] The department structure is the structure of the internal organizational departments of an enterprise and their relationships, including information such as the names, functions, and levels of the departments. Obtain the organizational structure information of the enterprise through the internal system of the target enterprise, including the organizational structure diagram, department responsibility descriptions, etc. According to the organizational structure diagram and department responsibility descriptions, identify each department in the enterprise and their relationships with each other, and determine which departments are directly involved in the core business activities of the enterprise to obtain multiple business function departments. Business function departments are departments within an enterprise divided according to their main business functions, specifically departments directly involved in business processes and core operations, such as R & D departments, technology departments, etc.

[0014] Invoke business data for the multiple business function departments to obtain multiple historical business data.

[0015] Determine the extraction time range of the historical data, such as data for the past year, quarter, or month. The extraction time range is determined according to specific requirements. Obtain the data sources of each business function department, such as the business databases within the department, and extract data from the business databases of each business function department according to the extraction time range to obtain multiple historical business data, providing an accurate data basis for subsequent data analysis and business collaboration.

[0016] Based on the multiple historical business data, perform business interaction analysis of the multiple business function departments, and construct and generate a data integration topology network. Among them, the data integration topology network includes multiple business data upload nodes mapped to the multiple business function departments and K middleware, and the K middleware is used for pre-storing business integration data.

[0017] According to multiple historical business data, analyze the flow paths of data between different business function departments, including identifying which data is generated by which department and how they are transmitted and shared between departments. By analyzing the frequency, direction, and pattern of data flow, identify the business interaction relationships between different departments. For example, the sales department may regularly share sales data with the finance department and inventory data with the warehousing department.

[0018] According to the results of business interaction analysis, a business data upload node is created for each business function department. These nodes represent the positions of each department in the data integration network and are the main interfaces for data inflow and outflow. According to the historical data flow paths and the interaction relationships between departments, the various business data upload nodes are connected to obtain a business interaction topology network. In the business interaction topology network, K middleware are deployed, where K is a positive integer. The role of the middleware is to serve as a temporary storage point for data pre-storage and help coordinate the flow of data between different business function departments.

[0019] According to the data flow directions of the multiple historical business data, the multiple historical business data are segmented into multiple groups of business data subsets. Among them, in each group of business data subsets, each group of business data subsets includes a local data subset and multiple cross-department data subsets.

[0020] By analyzing the multiple historical business data, identify the data flow directions between each business function department, including analyzing the starting point (generating department) of the data, the flowing path (cross-department data transmission situation), and the ending point (receiving department) of the data to determine the data flow direction.

[0021] According to the data flow direction, the multiple historical business data are segmented into multiple business data subsets. Each data subset contains data with similar attributes under specific conditions. Each group of business data subsets consists of a local data subset and multiple cross-department data subsets. Among them, the local data subset is the data generated within a specific business function department and mainly used by that department. These data do not need or rarely need to be shared with other departments; the cross-department data subset is the data generated in one department but needs to be transmitted and shared among multiple departments. These data involve inter-departmental collaboration and business processes.

[0022] Based on the multiple groups of business data subsets, construct multiple business collaboration judgment models, and map and synchronize the multiple business collaboration judgment models to the multiple business data upload nodes to complete the function enhancement of the data integration topology network.

[0023] Based on multiple groups of business data subsets, construct multiple business collaboration judgment models. These models are responsible for automatically classifying and sorting data to ensure that the data is stored according to predetermined rules and business requirements. The specific construction process of the models will be detailed in the subsequent steps. Map the constructed multiple business collaboration judgment models to the corresponding business data upload nodes. Each node is responsible for processing a specific business data subset and performing data processing tasks according to the judgment results of the model. Moreover, the model can be synchronized with the business data upload node in real time. By deploying the business collaboration judgment model in the data integration topology network, new data processing functions are introduced, improving the efficiency and intelligence level of the network.

[0024] In the data integration topology network, the multiple service data upload nodes use the multiple service collaboration judgment models to predict the data flow direction of the service data dynamically uploaded by the multiple service function departments, and perform intelligent data routing based on the prediction results to complete the data storage update of the K middleware.

[0025] During their daily operations, multiple service function departments generate new real-time service data, which are uploaded by their respective service function departments to the corresponding multiple service data upload nodes. The service data upload nodes use the constructed service collaboration judgment models to predict the flow direction of the dynamically uploaded real-time service data. The model predicts the data flow path based on the learned features and patterns, and the output prediction results include collaborative data and non-collaborative data.

[0026] If it is collaborative data, obtain its real-time transfer department, locate the real-time middleware in the K middleware according to the real-time transfer department, and route the real-time service data to the real-time middleware; if it is non-collaborative data, perform formatting processing on the real-time service data to obtain the first formatted data, and locally store the first formatted data at the first service data upload node.

[0027] Through the above steps, the data can automatically flow to the correct processing nodes or storage locations according to its type and the requirements of the service departments, realizing efficient data management and transmission, thereby improving the intelligent level of the overall business operation of the enterprise.

[0028] Furthermore, based on the multiple historical service data, perform business interaction analysis of the multiple service function departments to construct and generate a data integration topology network. The method further includes: By performing data flow analysis on the multiple historical service data, obtain multiple transfer department sets; configure the multiple service data upload nodes according to the multiple service function departments; perform communication connections of the multiple service data upload nodes according to the multiple transfer department sets to obtain a business interaction topology network, where the business interaction topology network includes K communication connection paths; in the business interaction topology network, deploy middleware on the K communication connection paths to obtain the data integration topology network.

[0029] Analyze multiple historical service data, trace the data flow path among the service function departments, and determine the complete data flow path by identifying the data generation department, transmission department, and final receiving department.

[0030] Analyze multiple historical business data, track the flow path of data among various business functional departments, determine the complete flow path of data by identifying the data generation department, transmission department, and final receiving department, and identify the dependency relationships among departments based on the data flow path. For example, sales data first flows from the sales department to the finance department and then to the logistics department, and this dependency relationship reflects the intensity of business connections among departments. Based on the results of data flow analysis, construct multiple sets of transfer departments. A set of transfer departments contains multiple business functional departments participating in data transfer, and these departments act together in a specific business process.

[0031] Perform node configuration. Specifically, first, determine the number of business data upload nodes to be configured and their positions in the network according to the number of multiple business functional departments and the results of data flow analysis. Then, determine the main business requirements and data transmission requirements of each business functional department. For example, the sales department needs to frequently upload sales records, and the finance department needs to process a large amount of financial statement data. Based on this, configure the computing resources, storage space, and network bandwidth required for the nodes to ensure that they can meet the business requirements. Finally, obtain multiple configured business data upload nodes.

[0032] Determine the connection relationships among departments according to multiple sets of transfer departments, and based on this, establish communication connections between each business data upload node and other nodes. This includes establishing connections between nodes at the physical and logical levels to ensure that data can flow smoothly between nodes. After all connection paths are established, integrate duplicate paths to obtain a business interaction topology network. The business interaction topology network includes K communication connection paths, where K is the number of communication connection paths in the business interaction topology network, and K is a positive integer.

[0033] Middleware is a component deployed in the network for optimizing data transmission, processing, and storage. Middleware can relieve node load, improve data transmission speed, and enhance the overall function of the network. Deploy middleware on the K communication connection paths to optimize the connection paths between nodes and prevent single-path overload. After the middleware deployment is completed, a complete data integration topology network is formed. This network can not only handle the transmission of business data but also process and store data through middleware.

[0034] Furthermore, by performing data flow analysis on the multiple historical business data, multiple sets of transfer departments are obtained. The method further includes: Track the data flow paths of the multiple historical business data to obtain multiple out-degree business data; use the multiple business functional departments as the analysis starting point, traverse the multiple out-degree business data, and obtain multiple discrete department call sequences; perform aggregation of business functional departments on the multiple discrete department call sequences to obtain the multiple sets of transfer departments.

[0035] Analyze multiple historical business data, focusing on the external sharing of department data, and track how the data generated by each department flows to other departments. Out-degree business data refers to the data generated by one department and transmitted to other departments. Filter out multiple out-degree business data for all cross-department data sharing. These data are the basis for business integration between different departments within the enterprise and reflect the data dependencies and collaboration relationships between departments.

[0036] Take each business function department as the analysis starting point. The analysis starting point department refers to the source department of data flow, that is, the department that initially generates and shares data. Starting from each analysis starting point department, traverse the out-degree business data one by one and track the flow of data in other departments. According to the order of data flow, generate a discrete department call sequence corresponding to each out-degree data. These call sequences show the flow trajectory of data within the enterprise and reveal the existing data dependencies between departments.

[0037] Aggregate the business function departments in multiple discrete department call sequences to form multiple sets of transfer departments. A set of transfer departments refers to an aggregate formed by multiple departments through data flow and interaction in a certain type of business process.

[0038] Furthermore, based on the multiple subsets of business data, construct multiple business collaboration judgment models. The method further includes: Perform mapped data calls on the multiple out-degree business data and multiple sets of transfer departments based on the first business function department to obtain the first out-degree business data and the first set of transfer departments. Among them, the first business function department is any one of the multiple business function departments; split the first out-degree business data according to the first set of transfer departments to obtain K cross-department data subsets. Among them, the first set of transfer departments includes K transfer departments; extract the first historical business data mapped to the first business function department from the multiple historical business data, and obtain the first local data subset by removing the first out-degree business data from the first historical business data; use the first local data subset and the K cross-department data subsets as the first set of business data subsets to construct the first business collaboration judgment model; and so on, construct the multiple business collaboration judgment models based on the multiple sets of business data subsets.

[0039] First, randomly extract an analysis object from multiple business function departments as the first business function department, and perform mapped data calls on the multiple out-degree business data and multiple sets of transfer departments to obtain the first out-degree business data and the first set of transfer departments directly related to the first business function department.

[0040] The first circulation department set is analyzed to identify the specific K circulation departments contained in the set. These departments are the key nodes where data needs to be transmitted across departments during the business integration process. According to the K circulation departments, the first out-degree business data is split into K cross-department data subsets, and each cross-department data subset has a mapping relationship with the corresponding circulation department.

[0041] From multiple historical business data, the first historical business data corresponding to the first business function department is extracted, and the first out-degree business data is removed from the first historical business data to distinguish which data is shared externally and which data is only used within the department. After removing the out-degree business data, the remaining data is the first local data subset of the department. These data are only used within the first business function department and do not involve cross-departmental business interactions.

[0042] Based on the multi-layer perceptron, a standard collaboration judgment model is constructed. The input layer of the model receives the features of K cross-departmental data subsets, the hidden layer is responsible for processing these features, and the output layer gives the judgment results. After model training, the business collaboration judgment layer is obtained. The data formatting layer is constructed based on the first local data subset. The business collaboration judgment layer and the data formatting layer are cascaded to obtain the first business collaboration judgment model. This model imitates the data interaction habits of multiple business departments in business collaboration, and judges whether the subsequent business uploaded data should be used for business collaboration of business departments.

[0043] In this way, multiple business data subsets are traversed to build multiple business collaboration judgment models.

[0044] Furthermore, the first local data subset and K cross-departmental data subsets are used as a first group of business data subsets to construct a first business collaboration judgment model, and the method further includes: Perform feature vector extraction on the K cross-departmental data subsets to obtain K business collaboration feature sets; construct a standard collaboration judgment model based on a multi-layer perceptron, and use the K business collaboration feature sets and K cross-departmental data subsets to perform model differentiation of the standard collaboration judgment model to obtain K business collaboration judgment branches; connect the K business collaboration judgment branches in parallel to obtain a business collaboration judgment layer; divide the first local data subset based on data reception time to obtain multiple updated data subsets; perform format feature analysis on the multiple updated data subsets to obtain a standard data format; construct a data formatting layer based on the standard data format; cascade the business collaboration judgment layer and the data formatting layer to complete the construction of the first business collaboration judgment model.

[0045] First, explicit information (such as dates, times, amounts, etc.) and implicit information (such as transaction patterns, behavioral trends, etc.) are extracted from each cross - departmental data subset. These information reflect the specific performance and potential rules of the data in cross - departmental operations. Through the integration of explicit and implicit information, feature vectors are generated, and K business collaboration feature sets are obtained through integration. These feature sets contain the core features of the data, providing a basis for subsequent model training and judgment.

[0046] Based on the extracted explicit and implicit information features, a standard collaboration judgment model is constructed using a multi - layer perceptron (MLP). The MLP can predict the flow and processing method of business data according to the learned data features. According to specific business requirements and the characteristics of cross - departmental data subsets, the standard model is differentiated to generate K business collaboration judgment branches dedicated to processing specific types of data. Each branch focuses on processing different categories of cross - departmental data, ensuring that the model can make targeted judgments and predictions.

[0047] The K business collaboration judgment branches are connected in parallel to form a business collaboration judgment layer. This layer can process multiple data subsets in parallel, integrate the judgment results of each branch, and provide comprehensive business collaboration support.

[0048] Extract the timestamp of the data reception time from the first local data subset. The timestamp is the specific time point of data reception. According to business requirements, define the segmentation rules for the data reception time. For example, segment according to time windows such as days and weeks. The first local data subset is segmented into multiple updated data subsets according to the defined time window. Each updated data subset contains the data received within a specific time period.

[0049] Conduct format feature analysis on each updated data subset to identify the key attributes of the data format, including data types (such as text, numbers, dates), the names of data fields, the lengths of fields, the arrangement order of data, etc. Based on the analysis results, formulate a unified standard data format. This standard data format can cover the common features in all updated data subsets. For example, unify the date format to YYYY - MM - DD to ensure the operability and universality of the format.

[0050] Design a data formatting layer, encode the rules of the standard data format into the formatting layer, and divide the data formatting layer into several modules. Each module processes different aspects of data formatting. For example, different modules are responsible for data type conversion, field standardization, date format unification, etc. This data formatting layer can be strengthened to ensure that subtle differences in the data are carefully captured during the formatting process, further enhancing the model's processing ability.

[0051] Clarify the hierarchical relationship between the business collaboration judgment layer and the data formatting layer. Data is first judged by the business collaboration judgment layer. If the business collaboration judgment result is a non-empty set, the real-time transfer department is output. On the contrary, if the business collaboration judgment result is an empty set, the data formatting layer is activated for formatting processing, and then local storage is performed. Cascading the business collaboration judgment layer and the data formatting layer according to this logic ensures that data passes through each layer for processing in the correct order.

[0052] The finally obtained first business collaboration judgment model realizes in-depth understanding and precise processing of enterprise business data through the iterative association learning mechanism. The core of the model is to simulate data collaboration and flow between different business departments through the extraction and association learning of explicit and implicit information. In the model, a multi-layer perceptron is used to predict the flow direction and make decisions based on data features. Through model differentiation, multiple judgment branches are generated for different business requirements and data types to achieve more refined processing.

[0053] Furthermore, extracting feature vectors from the K cross-department data subsets to obtain K business collaboration feature sets, the method further includes: Extracting explicit information from the K cross-department data subsets to obtain K explicit information sets; mining implicit information from the K cross-department data subsets to obtain K implicit information sets, where the K explicit information sets and the K implicit information sets both use the K transfer department identifiers; extracting feature vectors based on the K explicit information sets and the K implicit information sets to obtain the K business collaboration feature sets.

[0054] For each cross-department data subset, first extract explicit information in the cross-department data, such as timestamps, transaction amounts, product IDs, etc., and encode them as explicit information. Learn and extract these features through an encoder to form a preliminary explicit feature vector. Use information association to identify and capture the detailed connections between explicit information, which reflect the relevance of data in different business processes. For example, the association between sales data and inventory data in a certain time period.

[0055] Use a BP neural network to mine implicit information from the cross-department data subset. Implicit information is derived from in-depth analysis of the data, such as transaction patterns, user behavior trends, etc. These information are often not easily observable directly but are crucial for business collaboration judgment. Through an iterative method, use information association to capture the relevance between implicit information and the relevance between implicit information and explicit information. Such an iterative learning process can better understand the relevance of data in different business scenarios.

[0056] Among them, the K explicit information sets and the K implicit information sets both adopt the corresponding K transfer department identifiers, and the feature vectors of the K explicit information sets and the K implicit information sets are integrated to obtain the K business collaboration feature sets.

[0057] Furthermore, the method further includes: When the first business function department generates first real-time business data, the first business function department uploads the first real-time business data to the first business data upload node in the data integration topology network; at the first business data upload node, data flow prediction is synchronously performed through the K business collaboration judgment branches in the business collaboration judgment layer of the first business collaboration judgment model; if the business collaboration judgment result is a non-empty set, the real-time transfer department is output; the first business data upload node locates the real-time middleware among the K middleware according to the real-time transfer department, and routes the first real-time business data to the real-time middleware.

[0058] When the first business function department performs its regular business operations, it generates first real-time business data, and the specific data content depends on the nature of the department's business. The first business function department transmits the generated first real-time business data to the corresponding first business data upload node through the network.

[0059] When the first business data upload node receives the first real-time business data, it activates the business collaboration judgment layer in the first business collaboration judgment model and invokes the K business collaboration judgment branches in the business collaboration judgment layer. The K business collaboration judgment branches synchronously judge and predict the data. Each branch model predicts the data flow in the network according to its designed and trained characteristics. For example, some data may be determined to need to be transmitted to the finance department, while other data may be transmitted to the customer service department. By synthesizing the prediction results of the K branches, the final business collaboration judgment result is obtained.

[0060] If the business collaboration judgment result is a non-empty set, it means that the data needs to be transmitted to one or more transfer departments for processing. The non-empty set judgment result includes a set of transfer departments, which are the next recipients of the business data. According to the output of the model, the real-time transfer department receiving the data is identified.

[0061] According to the real-time transfer department, the middleware associated with these departments is located in the network. Each middleware is associated with a transfer department and is responsible for temporarily storing, processing, or forwarding business data. The first real-time business data is transmitted from the upload node to the located real-time middleware for temporary storage, format conversion, data aggregation, or further processing.

[0062] Furthermore, the method further includes: If the business collaboration judgment result is an empty set, activate the data formatting layer of the first business collaboration judgment model; perform formatting processing on the first real-time business data through the data formatting layer to obtain first-formatted data; locally store the first-formatted data at the first business data upload node.

[0063] If the business collaboration judgment result is an empty set, it means that the current data does not need to be transmitted to other transfer departments for processing. In this case, activate the data formatting layer to standardize the data and locally store it instead of continuing to transmit it.

[0064] Select applicable formatting rules according to the type and content of the first real-time business data. These rules define how to process each part of the data, including date format, value range, field sorting, etc. The data formatting layer processes the first real-time business data one by one according to the formatting rules to ensure the uniformity and standardization of the data format. After the processing operation, first-formatted data is generated. Locally store the first-formatted data at the first business data upload node.

[0065] In summary, the data storage optimization method for business integration provided by the embodiments of the present application has the following technical effects: By extracting the department structure of the target enterprise and calling the historical business data of each department, it is possible to comprehensively understand the business flow within the enterprise, and then effectively identify the interaction relationships and data dependencies between departments, laying a foundation for subsequent data integration; based on the historical business data, construct a data integration topology network, which effectively reflects the interaction relationships between various business functional departments within the enterprise. By mapping multiple business data upload nodes and K middleware in the network, pre-storage and management of business data are realized. Such a topology structure not only optimizes the distribution of data storage, but also improves the speed and efficiency of data access, ensuring the efficient flow of data between departments; by analyzing the data flow of historical business data, divide the data into multiple groups of business data subsets, and on this basis, construct multiple business collaboration judgment models, which can deeply analyze the explicit and implicit information of business data and identify the correlation between data. This model-based design enables the system to intelligently judge whether business data needs to be uploaded or flow between different departments, thus enhancing the collaboration ability and decision support between departments; at the data upload node, predict the flow direction of the real-time uploaded data through the business collaboration judgment model, and perform intelligent data routing based on the prediction result, ensuring that the data can be transmitted to the appropriate middleware for processing or storage in a timely and accurate manner, thereby optimizing the resource utilization rate of the enterprise.

[0066] Embodiment 2, based on the same inventive concept as the data storage optimization method for business integration in the foregoing embodiment, as Figure 2As shown in the figure, the embodiment of the present application provides a data storage optimization platform for business integration, and the platform includes: A department architecture extraction module 10, which is used to extract the department architecture of the target enterprise to obtain multiple business function departments; a business data call module 20, which is used to call business data of the multiple business function departments to obtain multiple historical business data; a business interaction analysis module 30, which is used to perform business interaction analysis of the multiple business function departments based on the multiple historical business data to construct and generate a data integration topology network, wherein the data integration topology network includes multiple business data upload nodes mapped to the multiple business function departments and K middleware, and the K middleware is used for pre-storing business integration data; a historical business data segmentation module 40, which is used to segment the multiple historical business data into multiple groups of business data subsets according to the data flow direction of the multiple historical business data, wherein in the multiple groups of business data subsets, each group of business data subsets includes a local data subset and multiple cross-department data subsets; a topology network function enhancement module 50, which is used to construct multiple business collaboration judgment models based on the multiple groups of business data subsets and map and synchronize the multiple business collaboration judgment models to the multiple business data upload nodes to complete the function enhancement of the data integration topology network; an intelligent data routing module 60, which is used to predict the data flow direction of the business data dynamically uploaded by the multiple business function departments by the multiple business data upload nodes through the multiple business collaboration judgment models in the data integration topology network and perform intelligent data routing based on the prediction result to complete the data storage update of the K middleware.

[0067] Furthermore, the platform further includes a data integration topology network construction module to perform the following operation steps: By performing data flow analysis on the multiple historical business data, multiple sets of transfer departments are obtained; the multiple business data upload nodes are configured according to the multiple business function departments; the communication connections of the multiple business data upload nodes are performed according to the multiple sets of transfer departments to obtain a business interaction topology network, wherein the business interaction topology network includes K communication connection paths; in the business interaction topology network, middleware deployment is performed on the K communication connection paths to obtain the data integration topology network.

[0068] Furthermore, the platform further includes multiple sets of transfer department acquisition modules to perform the following operation steps: Track the data flow paths of the multiple historical business data to obtain multiple out-degree business data; use the multiple business function departments as the analysis starting point, traverse the multiple out-degree business data to obtain multiple discrete department call sequences; perform business function department aggregation on the multiple discrete department call sequences to obtain the multiple transfer department sets.

[0069] Furthermore, the platform further includes a business collaboration judgment model construction module to perform the following operation steps: Perform mapped data calls on the multiple out-degree business data and the multiple transfer department sets based on the first business function department to obtain the first out-degree business data and the first transfer department set, where the first business function department is any one of the multiple business function departments; split the first out-degree business data according to the first transfer department set to obtain K cross-department data subsets, where the first transfer department set includes K transfer departments; extract the first historical business data mapped to the first business function department from the multiple historical business data, and obtain the first local data subset by removing the first out-degree business data from the first historical business data; use the first local data subset and the K cross-department data subsets as the first group of business data subsets to construct the first business collaboration judgment model; and so on, construct the multiple business collaboration judgment models based on the multiple groups of business data subsets.

[0070] Furthermore, the platform further includes a first business collaboration judgment model construction module to perform the following operation steps: Extract feature vectors from the K cross-department data subsets to obtain K business collaboration feature sets; construct a standard collaboration judgment model based on a multi-layer perceptron, use the K business collaboration feature sets and the K cross-department data subsets to perform model differentiation of the standard collaboration judgment model to obtain K business collaboration judgment branches; parallelize the K business collaboration judgment branches to obtain a business collaboration judgment layer; segment the first local data subset based on the data reception time to obtain multiple updated data subsets; perform format feature analysis on the multiple updated data subsets to obtain a standard data format; construct a data formatting layer based on the standard data format; cascade the business collaboration judgment layer and the data formatting layer to complete the construction of the first business collaboration judgment model.

[0071] Furthermore, the platform further includes K business collaboration feature set acquisition modules to perform the following operation steps: Perform explicit information extraction on the K cross-department data subsets to obtain K explicit information sets; perform implicit information mining on the K cross-department data subsets to obtain K implicit information sets, where the K explicit information sets and the K implicit information sets both use the K transfer department identifiers; perform feature vector extraction based on the K explicit information sets and the K implicit information sets to obtain the K business collaboration feature sets.

[0072] Furthermore, the platform further includes a first real-time service data routing module to perform the following operation steps: When the first business function department generates first real-time service data, the first business function department uploads the first real-time service data to the first service data upload node in the data integration topology network; at the first service data upload node, perform data flow prediction synchronously through the K business collaboration judgment branches of the business collaboration judgment layer in the first business collaboration judgment model; if the business collaboration judgment result is a non-empty set, output the real-time transfer department; the first service data upload node locates the real-time middleware among the K middleware according to the real-time transfer department and routes the first real-time service data to the real-time middleware.

[0073] Furthermore, the platform further includes a local storage module to perform the following operation steps: If the business collaboration judgment result is an empty set, activate the data formatting layer of the first business collaboration judgment model; perform formatting processing on the first real-time service data through the data formatting layer to obtain first formatted data; locally store the first formatted data at the first service data upload node.

[0074] Through the foregoing detailed description of the data storage optimization method for business integration in this specification, those skilled in the art can clearly know the data storage optimization platform for business integration in this embodiment. Since it corresponds to the method disclosed in the embodiment, the description is relatively simple. For related parts, refer to the description in the method part.

[0075] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present application. Various modifications to these embodiments will be obvious to those skilled in the art, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A data storage optimization method for business integration, characterized in that: The method comprises: Extract the departmental structure of the target enterprise and obtain multiple business functional departments; Calling business data of the multiple business functional departments to obtain multiple historical business data; Based on the multiple historical business data, business interaction analysis of the multiple business functional departments is performed to construct and generate a data integration topology network, wherein the data integration topology network includes multiple business data upload nodes mapped to the multiple business functional departments and K middlewares, and the K middlewares are used for pre-storing business integration data; According to the data flow directions of the plurality of historical business data, the plurality of historical business data are divided into a plurality of business data subsets, wherein each of the plurality of business data subsets includes a local data subset and a plurality of cross-department data subsets; Building multiple business collaboration judgment models based on the multiple business data subsets, and mapping and synchronizing the multiple business collaboration judgment models to the multiple business data upload nodes to complete the function enhancement of the data integration topology network; In the data integration topology network, the multiple business data upload nodes predict the data flow of the business data dynamically uploaded by the multiple business functional departments through the multiple business collaboration judgment models, and perform intelligent data routing based on the prediction results to complete the data storage update of the K middlewares.

2. The data storage optimization method for business integration according to claim 1, characterized in that: Based on the plurality of historical business data, business interaction analysis of the plurality of business functional departments is performed to construct and generate a data integration topology network, and the method further includes: By performing data flow analysis on the plurality of historical business data, a plurality of flow department sets are obtained; Configuring the multiple business data uploading nodes according to the multiple business function departments; According to the plurality of transfer department sets, the plurality of business data upload nodes are communicated and connected to obtain a business interaction topology network, wherein the business interaction topology network includes K communication connection paths; In the business interaction topology network, middleware is deployed on the K communication connection paths to obtain the data integration topology network.

3. The data storage optimization method for business integration according to claim 2, characterized in that: By performing data flow analysis on the plurality of historical business data, a plurality of flow department sets are obtained, and the method further comprises: Tracing data flow paths of the plurality of historical business data to obtain a plurality of out-degree business data; Using the multiple business function departments as the analysis starting point, traversing the multiple out-degree business data, and obtaining multiple discrete department call sequences; The plurality of discrete department call sequences are aggregated into business function departments to obtain the plurality of circulation department sets.

4. The data storage optimization method for business integration according to claim 3, characterized in that: Constructing multiple business collaboration judgment models based on the multiple groups of business data subsets, the method further includes: Based on the first business function department, mapping data is called on the multiple out-degree business data and the multiple circulation department sets to obtain the first out-degree business data and the first circulation department set, wherein the first business function department is any one of the multiple business function departments; Splitting the first out-degree business data according to the first circulation department set to obtain K cross-department data subsets, wherein the first circulation department set includes K circulation departments; Extracting first historical business data mapped to a first business function department from the plurality of historical business data, and removing the first out-degree business data from the first historical business data to obtain a first local data subset; Using the first local data subset and K cross-departmental data subsets as a first group of business data subsets to construct a first business collaboration judgment model; By analogy, the multiple business collaboration judgment models are constructed based on the multiple groups of business data subsets.

5. The data storage optimization method for business integration according to claim 4, characterized in that: The first local data subset and K cross-departmental data subsets are used as a first group of business data subsets to construct a first business collaboration judgment model, and the method further includes: Extracting feature vectors from the K cross-departmental data subsets to obtain K business collaboration feature sets; Building a standard collaboration judgment model based on a multi-layer perceptron, using the K business collaboration feature sets and K cross-departmental data subsets to perform model differentiation of the standard collaboration judgment model, and obtaining K business collaboration judgment branches; The K business collaboration judgment branches are connected in parallel to obtain a business collaboration judgment layer; Divide the first local data subset based on the data reception time to obtain multiple updated data subsets; Performing format feature analysis on the multiple update data subsets to obtain a standard data format; Constructing and obtaining a data formatting layer based on the standard data format; The business collaboration judgment layer and the data formatting layer are cascaded to complete the construction of the first business collaboration judgment model.

6. The data storage optimization method for business integration according to claim 5, characterized in that: Extracting feature vectors from the K cross-departmental data subsets to obtain K business collaboration feature sets, the method further comprising: Extracting explicit information from the K cross-departmental data subsets to obtain K explicit information sets; Performing implicit information mining on the K cross-department data subsets to obtain K implicit information sets, wherein the K explicit information sets and the K implicit information sets both use the K circulation department identifiers; Feature vectors are extracted based on the K explicit information sets and the K implicit information sets to obtain the K business collaboration feature sets.

7. The data storage optimization method for business integration according to claim 5, characterized in that: The method further comprises: When the first business function department generates first real-time business data, the first business function department uploads the first real-time business data to a first business data uploading node in the data integration topology network; At the first business data uploading node, data flow direction prediction is synchronously performed through K business collaboration judgment branches of the business collaboration judgment layer in the first business collaboration judgment model; If the result of the business collaboration judgment is a non-empty set, the real-time flow department is output; The first service data uploading node locates a real-time middleware among the K middlewares according to the real-time flow department, and routes the first real-time service data to the real-time middleware.

8. The data storage optimization method for business integration according to claim 7, characterized in that: The method further comprises: If the business collaboration judgment result is an empty set, activating the data formatting layer of the first business collaboration judgment model; Formatting the first real-time service data through the data formatting layer to obtain first formatted data; The first formatted data is locally stored at the first service data uploading node.

9. A data storage optimization platform for business integration, characterized in that: For implementing the data storage optimization method for business integration according to any one of claims 1 to 8, the platform comprises: A department structure extraction module, which is used to extract the department structure of the target enterprise and obtain multiple business function departments; A business data calling module, wherein the business data calling module is used to call business data of the multiple business function departments to obtain multiple historical business data; A business interaction analysis module, the business interaction analysis module is used to perform business interaction analysis of the multiple business function departments based on the multiple historical business data, and construct a data integration topology network, wherein the data integration topology network includes multiple business data upload nodes mapped to the multiple business function departments and K middlewares, and the K middlewares are used to pre-store business integration data; A historical business data segmentation module, the historical business data segmentation module is used to segment the multiple historical business data into multiple business data subsets according to the data flow direction of the multiple historical business data, wherein each of the multiple business data subsets includes a local data subset and multiple cross-departmental data subsets; A topology network function enhancement module, which is used to construct multiple business collaboration judgment models based on the multiple business data subsets, and synchronize the mapping of the multiple business collaboration judgment models to the multiple business data upload nodes to complete the function enhancement of the data integration topology network; An intelligent data routing module is used in the data integration topology network. The multiple business data uploading nodes predict the data flow of the business data dynamically uploaded by the multiple business functional departments through the multiple business collaboration judgment models, and perform intelligent data routing based on the prediction results to complete the data storage update of the K middlewares.

Images