Dynamic data pipeline construction method based on artificial intelligence and multi-modal data processing

Abstract

The invention relates to a dynamic data pipeline construction method based on artificial intelligence and multi-modal data processing, and belongs to the technical field of data fusion treatment and artificial intelligence. According to the method, a multi-modal deep learning model is adopted, semantic analysis is performed on business documents, user requirements and function specification instructions, and key indexes, data requirements and logic constraints are extracted; constructing a multi-modal knowledge graph, performing unified representation and feature extraction on structured and unstructured data resources, and performing embedded learning by using a graph neural network; based on a deep semantic matching model and a hybrid recommendation system, realizing accurate matching of business requirements and data resources; modeling generation of a data processing flow into a Markov decision process by adopting a reinforcement learning strategy, and automatically generating an optimized data processing workflow; the adaptability of the model to new business requirements is improved by utilizing a meta-learning algorithm; and dynamic scheduling and execution of tasks are realized through a workflow engine. According to the invention, the data processing efficiency can be improved.

Description

Technical Field

[0001] The present invention belongs to the field of data fusion management and artificial intelligence technology, and specifically relates to a dynamic data pipeline construction method based on artificial intelligence and multimodal data processing. Background Art

[0002] With the rapid development of information technology, enterprises are faced with the challenge of quickly adapting to new business needs. Traditional data pipeline construction methods often lack flexibility and efficiency, and cannot respond to business changes in a timely manner, resulting in long development cycles and high costs. In addition, existing methods have limited processing and analysis capabilities for multimodal data and cannot fully utilize the rich multi-source heterogeneous data resources within the enterprise, resulting in low data reuse rates and affecting business innovation and development.

[0003] Therefore, there is an urgent need for a method that can intelligently analyze and match data resources and automatically generate data processing flows to improve the efficiency of building data pipelines and meet the needs of rapid deployment of new businesses. Summary of the invention

[0004] 1. Technical issues to be resolved

[0005] The technical problem to be solved by the present invention is how to provide a dynamic data pipeline construction method based on artificial intelligence and multimodal data processing, so as to solve the problem in the prior art that, during the interconnection process of multi-source heterogeneous data, data integration and fusion are complex and consistency is difficult to ensure due to inconsistency in data format and semantics; to solve the problem in the prior art that data pipeline construction is inefficient, it is difficult to quickly respond to new business needs, resulting in a long development cycle and high costs; to solve the problem in the prior art that the ability of multimodal data processing and intelligent matching of data resources is insufficient, and the multi-source heterogeneous data resources within the enterprise cannot be fully utilized, resulting in low data reuse rate and affecting business innovation and development.

[0006] (II) Technical solution

[0007] In order to solve the above technical problems, the present invention proposes a method for constructing a dynamic data pipeline based on artificial intelligence and multimodal data processing, which comprises the following steps:

[0008] S1. Business requirements understanding: Use multimodal deep learning models to perform semantic analysis on business documents, user requirements, and functional specifications, extract key indicators, data requirements, and logical constraints; including: semantic analysis, knowledge graph fusion, intent recognition and slot filling, and structured requirements representation generation steps;

[0009] S2. Data resource analysis and matching: Construct a multimodal knowledge graph, uniformly represent and extract features of structured and unstructured data resources, and use graph neural networks for embedded learning; based on deep semantic matching models and hybrid recommendation systems, accurately match business needs with data resources, and sort and filter candidate data resources; including: multimodal knowledge graph construction and intelligent data recommendation steps;

[0010] S3. Data processing flow generation: Using reinforcement learning strategies, the generation of data processing flows is modeled as a Markov decision process, and optimized data processing workflows are automatically generated. Meta-learning algorithms are used to improve the model's adaptability to new business needs. Dynamic scheduling and execution of tasks are achieved through a workflow engine, and containerization and microservice architecture are used to improve the flexibility and maintainability of the system. This includes: automated workflow generation, dynamic task scheduling, and execution steps.

[0011] (III) Beneficial effects

[0012] The present invention proposes a method for constructing a dynamic data pipeline based on artificial intelligence and multimodal data processing. The present invention proposes an intelligent fusion and optimization method for multi-source heterogeneous data, which is suitable for scenarios such as data fusion management and new business development. The implementation of this method can achieve the following beneficial effects:

[0013] 1. Improve the efficiency of data pipeline construction: By automatically generating data processing processes, manual participation can be significantly reduced, the development cycle can be shortened, and the efficiency of data processing can be improved.

[0014] 2. Enhance the flexibility and adaptability of the system: By using multimodal deep learning models and meta-learning algorithms, the system can quickly adapt to new business needs and data changes, and improve its ability to respond to diverse needs.

[0015] 3. Optimize data resource utilization: Through intelligent analysis and matching of existing data resources, fully tap the value of data, improve data reuse rate, and reduce data management and storage costs.

[0016] 4. Improve the accuracy and reliability of data processing results: Use deep semantic matching models and reinforcement learning strategies to optimize the data processing process, ensure high accuracy and stability of the results, and meet business requirements for data quality.

[0017] 5. Reduce development and operation costs: Automated and intelligent data pipeline construction reduces manpower input and error rates, reduces system development and maintenance costs, and improves overall operational efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 It is the overall flow chart of the present invention;

[0019] Figure 2 This is a schematic diagram of BERT-BiLSTM-CRF;

[0020] Figure 3 This is the schematic diagram of xDeepFM;

[0021] Figure 4 This is a schematic diagram of the CIN module. DETAILED DESCRIPTION

[0022] In order to make the purpose, content and advantages of the present invention more clear, the specific implementation methods of the present invention are further described in detail below in conjunction with the drawings and examples.

[0023] The present invention relates to the fields of data fusion governance and artificial intelligence technology, and in particular to a dynamic data pipeline construction method and system, which aims to achieve rapid analysis of existing data resources and generate efficient data support solutions by combining multimodal data processing with artificial intelligence technology.

[0024] 1. Overall Architecture

[0025] The present invention proposes a dynamic data pipeline construction method based on artificial intelligence and multimodal data processing. The overall process is as follows: Figure 1 shown.

[0026] S1. Understanding business requirements: Using multimodal deep learning models, semantic analysis is performed on business documents, user requirements, and functional specifications to extract key indicators, data requirements, and logical constraints. Specifically, it includes the following steps: semantic analysis, knowledge graph fusion, intent recognition and slot filling, and structured requirement representation generation.

[0027] S2. Data resource analysis and matching: Build a multimodal knowledge graph, unify the representation and feature extraction of structured and unstructured data resources, and use graph neural networks for embedded learning. Based on deep semantic matching models and hybrid recommendation systems, achieve accurate matching of business needs and data resources, and sort and filter candidate data resources.

[0028] S3. Data processing flow generation: Using reinforcement learning strategies, the generation of data processing flows is modeled as a Markov decision process, and optimized data processing workflows are automatically generated. Meta-learning algorithms are used to improve the adaptability of the model to new business needs. Dynamic scheduling and execution of tasks are achieved through the workflow engine, and containerization and microservice architecture are used to improve the flexibility and maintainability of the system.

[0029] 2. Specific implementation steps

[0030] The present invention provides a method for constructing a dynamic data pipeline based on artificial intelligence and multimodal data processing. The dynamic data pipeline construction is intended to meet new business needs. By combining multimodal data processing with artificial intelligence technology, the existing data resources are quickly analyzed to generate efficient data support solutions. The dynamic data pipeline construction is centered on intelligent data recommendation and automated process generation to achieve efficient data reuse and rapid deployment of new businesses. The main processes include understanding business needs, analyzing and matching data resources, and generating data processing processes.

[0031] S1. Understanding of business requirements

[0032] Business requirements understanding uses a multimodal deep learning model to conduct in-depth semantic analysis of business documents, user requirements, and functional specifications, extract key indicators, data requirements, and logical constraints, and provide a basis for subsequent data pipeline generation. Business requirements understanding mainly includes processes such as semantic analysis, knowledge graph fusion, intent recognition and slot filling, and structured requirement representation generation.

[0033] S1.1 Semantic analysis

[0034] The semantic parsing step includes the following processes: data preprocessing, multimodal feature encoding, domain adaptation pre-training, and multi-task learning;

[0035] (1) Data preprocessing

[0036] This step preprocesses multimodal data related to business needs, including text, images, audio, video, documents, and tables, to eliminate data noise and unify data formats.

[0037] (a) The processing of text data includes word segmentation, removal of stop words, part-of-speech tagging and text normalization operations.

[0038] Word segmentation divides continuous text strings into independent words or phrases to obtain basic semantic units;

[0039] Stop word removal will filter out high-frequency words that have no actual semantic contribution in text processing, including conjunctions, adverbs, prepositions, etc.;

[0040] Part-of-speech tagging assigns each word a corresponding part-of-speech tag such as noun, verb, adjective, etc.

[0041] Normalization processing uniformly formats special symbols, numbers, dates, units, etc. in the text, solves the problems of synonym replacement and word form restoration, and ensures the consistency and standardization of text data.

[0042] (b) For images and videos, data cleaning, resizing, normalization, and data enhancement are used to process visual data. Data cleaning mainly removes damaged frames, blurred images, or noise-interfered clips; resizing adjusts image and video frames to a uniform size and resolution through methods such as bilinear interpolation and nearest neighbor interpolation; normalization standardizes pixel values ​​to eliminate differences in illumination and contrast; data enhancement generates an extended data set through random cropping, rotation, translation, scaling, horizontal or vertical flipping, brightness and contrast adjustment, etc., to improve the generalization ability of the model; frame extraction and resampling perform refined processing on video data, using scene change detection, motion vector analysis, and other technologies to extract key frames, and adjust the frame rate through time series interpolation and downsampling methods to adapt to the resolution requirements of the target analysis task and optimize storage efficiency.

[0043] (c) The audio data is processed through noise reduction, spectrogram conversion and other processes to obtain high-quality spectrum information. The noise reduction process uses an adaptive filter to remove background noise and interference in the audio signal; the audio signal is converted into a spectrogram through short-time Fourier transform (STFT) to obtain the time-frequency characteristics of the audio.

[0044] (d) Document and table data use OCR recognition, layout analysis and table structure recognition algorithms to convert the extracted text and table data into structured formats, such as JSON, XML or relational database formats, to facilitate subsequent data query and analysis. OCR recognition performs text recognition on scanned or photographed PDF, DOCX documents and images to generate editable text; layout analysis can identify layout elements such as paragraphs, titles, headers, and footers to extract the organizational structure information of the text; table structure recognition can detect the rows, columns and cell boundaries of the table, and use image processing and machine learning methods to parse the structure of the table and extract cell content, header information and data relationships.

[0045] (2) Multimodal feature encoding

[0046] This step uses a multimodal pre-trained model to encode the preprocessed multimodal data and generate a high-dimensional semantic vector representation. Data in different modalities such as text, images, audio, and video are input into the corresponding encoder to extract their respective feature representations. Through the cross-modal alignment mechanism, the features of different modalities are mapped to a unified semantic space to capture the correlation between modalities.

[0047] (3) Domain Adaptive Pre-training

[0048] In order to enhance the model's understanding of industry-specific and domain-specific terminology and visual features, this step uses a large amount of unlabeled multimodal data in the field for domain-adaptive pre-training based on the general pre-trained model. The tasks used include but are not limited to Masked Language Modeling and Masked Region Modeling to capture domain-specific language patterns, visual concepts, and cross-modal associations, and enhance the model's ability to represent domain data.

[0049] (4) Multi-task learning

[0050] Build a multi-task learning model to perform multiple related natural language processing and computer vision tasks simultaneously, including text classification, sequence labeling, relationship extraction, image classification, object detection, etc. By sharing model parameters, information between different tasks promotes each other, improving the model's ability to fully understand business needs.

[0051] S1.2 Knowledge Graph Fusion

[0052] The knowledge graph fusion step includes the following processes:

[0053] (1) Construction of Domain Knowledge Graph

[0054] Based on domain expert knowledge and public data sources, a knowledge graph containing domain entities and relationships is constructed. The nodes in the knowledge graph represent concepts, entities, and terms in the domain, and the edges represent the relationships between entities.

[0055] (2) Knowledge Graph Embedding

[0056] Through the knowledge graph, entities, relations and multimodal information are mapped into low-dimensional vector representations to capture the semantic associations and structural information between entities. For nodes containing multimodal information, a multimodal graph neural network (MM-GNN) is used for embedding to fuse the features of different modalities.

[0057] (3) Semantic Representation Fusion

[0058] The knowledge graph embedding is fused with the semantic vector representation generated by the multimodal pre-trained model to generate an enhanced semantic representation that integrates the knowledge graph: when it is necessary to use both representations at the same time without favoring either one, the semantic vector of the pre-trained model is directly concatenated with the knowledge graph embedding vector to form a new feature vector; when a certain representation needs to be emphasized according to the context, the two vectors are weighted and summed according to a certain weight to balance the model's attention to pre-trained features and knowledge graph information; when the model needs to focus on certain parts of the input data, the attention mechanism is used to calculate the correlation weights between the input data elements and the knowledge graph entities, and the fusion ratio is dynamically adjusted so that the model can pay attention to semantic information at different levels.

[0059] S1.3 Intent Recognition and Slot Filling

[0060] The intent identification and slot filling steps include the following processes:

[0061] (1) Building an Intent Recognition Model

[0062] Based on the enhanced semantic representation integrated with the knowledge graph, an intent classification model is constructed. The model uses Roberta through a fully connected layer and a Softmax activation function to output intent categories corresponding to business needs (such as "data analysis", "report generation", "prediction model", etc.).

[0063] (2) Constructing a slot filling model

[0064] The sequence annotation model BERT-BiLSTM-CRF is used to annotate the business requirement text word by word to identify the slots (such as "data type", "time range", "indicator name", etc.) and their values ​​related to the business requirements. Figure 2 shown.

[0065] (3) Attention Fusion

[0066] Introducing a multi-head attention mechanism into the intent recognition model and slot filling model, focusing on words and phrases in the text that are closely related to business needs. By calculating the importance weight of each word in the text sequence, the model's ability to capture key information is improved, and the accuracy of intent recognition and slot filling is improved.

[0067] S1.4 Structured Requirements Representation Generation

[0068] The structured requirements representation generation step includes the following processes:

[0069] (1) Results integration

[0070] Integrate the results of intent recognition and slot filling to form a complete structured requirement representation. With the identified intent as the root node, the slot as the intermediate node, and the slot value as the leaf node, the various components of the requirement and their logical associations are clearly represented. Through the hierarchical relationship of the tree structure, the dependencies and constraints between the slots are clarified to ensure the integrity and consistency of the requirement representation.

[0071] (2) Data Standardization

[0072] Standardize the slot values ​​to ensure data consistency and standardization. Specific measures include: converting date and time information into international standard formats (such as ISO 8601) to unify the representation of time data; using domain vocabulary or synonym dictionary to map synonyms and near synonyms into unified standard terms to eliminate semantic ambiguity; converting units for numerical data, such as converting centimeters to meters, to ensure the comparability of numerical data; using unified coding rules for categorical variables to facilitate data storage, retrieval and statistical analysis. Through these standardized operations, data quality is improved, laying a solid foundation for subsequent processing and analysis.

[0073] (3) Storage and management

[0074] Store structured demand representations in relational databases or graph databases to facilitate subsequent queries and calls.

[0075] (4) Interface provided

[0076] Provides a standardized interface for subsequent data resource analysis and matching modules, and supports the acquisition and delivery of structured demand representations.

[0077] S2. Data resource analysis and matching

[0078] This method conducts in-depth analysis and matching of data resources, aiming to intelligently recommend the most relevant data resources based on business needs, mainly including processes such as multimodal knowledge graph construction and intelligent data recommendation.

[0079] S2.1 Construction of multimodal knowledge graph

[0080] (1) Unified representation of data resources

[0081] For structured data, such as tables, fields, primary keys and foreign keys in relational databases, database metadata extraction tools are used to obtain their schema and attribute information. The extracted features include table and field information, data statistical features and relationship features.

[0082] For unstructured data, such as text, images, audio, and video, feature extraction algorithms are used to convert them into structured representations, and multimodal feature fusion is performed.

[0083] (a) Text data: Natural language processing technology is used to perform word segmentation, part-of-speech tagging, named entity recognition and dependency syntax analysis on the text to extract key entities and semantic features.

[0084] (b) Image data: Use pre-trained convolutional neural networks (such as Resnet, VGG, Inception) to extract high-level semantic features of the image, and perform target detection and image annotation to obtain object and scene information in the image.

[0085] (c) Audio data: Through speech recognition technology, the audio is converted into text, and audio features are extracted to obtain the audio content, time-frequency representation and emotional information.

[0086] (d) Video data: Combining image and audio processing techniques to extract the spatiotemporal features of the video, including visual objects, scene changes, and audio content.

[0087] (e) Multimodal feature fusion maps data features of different modalities to a common latent space, and realizes cross-modal feature fusion through joint representation, feature alignment, and cross-modal translation. Specifically, feature representation alignment is performed by sharing hidden layer parameters, and the semantic gap problem between multimodal data is solved through optimized learning of the joint feature space; at the same time, cross-modal translation technology is used to convert the features of one modality into the features of another modality, further improving the generalization ability of cross-modal tasks.

[0088] The data resource entity information obtained from the above processing is used as the nodes of the knowledge graph, which are mainly divided into two categories:

[0089] The first is the data entity node, including database tables, fields, files, documents, images, audio, video and other specific data resources;

[0090] The second is the concepts and entity nodes extracted from multimodal data (such as text, audio, and images), such as names of people, places, products, topics, etc.

[0091] Edges are used to represent the relationship between nodes, including data association relationships (such as foreign key relationships in database tables or inclusion relationships between files and folders), semantic relationships (such as hierarchical relationships, synonymous relationships, or association relationships based on knowledge bases and semantic similarity calculations), and multimodal associations (such as the relationship between images and text descriptions or the association between video and audio content). On this basis, the richness and accuracy of the knowledge graph can be further improved through knowledge fusion and completion. Specifically, the extracted entities and relationships can be mapped to the existing knowledge base through knowledge base alignment to expand the content of the graph; at the same time, the knowledge graph completion algorithm TransE is used to infer potential but unlabeled relationships to improve the structure and information of the knowledge graph.

[0092] (2) Graph Neural Network Embedding

[0093] In order to capture the complex associations and semantic information of data resources in the knowledge graph, a graph neural network (GNN) is used to embed the knowledge graph. The specific settings are as follows:

[0094] (a) Model selection: We select the graph neural network model Graph Transformer, which is suitable for large-scale knowledge graphs. This model can efficiently process the neighborhood information of nodes and capture the global and local features in the graph structure.

[0095] (b) Training strategy selection: Depending on the amount of labeled data, the GNN model can be trained in an unsupervised or semi-supervised manner. Unsupervised training can use techniques such as random walks and negative sampling to minimize the distance between similar nodes and maximize the distance between dissimilar nodes. Semi-supervised training can use partially labeled node information and adopt a cross-entropy loss function to guide the model to learn effective embedding representations.

[0096] (c) Feature aggregation: For each node, the GNN model updates its representation by aggregating the features of its neighboring nodes. The aggregation function can use averaging, weighted averaging, or attention mechanisms to emphasize important neighboring nodes.

[0097] (d) Embedding learning: Define the initial feature vector of the node, including the attribute features and category information of the node. Through multi-layer GNN calculation, iteratively update the feature representation of the node, and finally obtain a high-dimensional embedding vector for each node.

[0098] S2.2 Intelligent Data Recommendation

[0099] (1) Building a deep semantic matching model

[0100] In order to achieve accurate matching between business requirements and data resources, a deep semantic matching model is built. The specific settings of the model are as follows:

[0101] (a) Model architecture: A dual-tower model structure is adopted, which consists of two deep neural networks with shared parameters, processing the description of business needs and data resources respectively.

[0102] (b) Input representation: includes the input representation corresponding to the business requirement description and the data resource description.

[0103] Business requirement description: Convert the structured business requirement representation into text or feature vector form as the input of the model.

[0104] Data resource description: Use node features in the multimodal knowledge graph, as well as metadata and content summaries to construct the input representation of data resources.

[0105] (c) Feature encoding: For text input, a pre-trained language model is used for encoding to obtain a semantically rich vector representation. For non-text input, a fully connected layer or autoencoder is used for encoding.

[0106] (d) Semantic vector generation: Through forward propagation, the semantic vector representation of business requirements and data resources is obtained.

[0107] (e) Similarity calculation: Use cosine similarity, Euclidean distance or neural network-based matching function to calculate the matching degree between business requirements and data resources.

[0108] (f) Training strategy: Use the contrastive learning method to construct positive and negative sample pairs. Positive samples are business needs and relevant data resources, and negative samples are business needs and irrelevant data resources. The selection of positive and negative samples can be based on historical interaction data or expert annotations. The loss function selects contrastive loss or triplet loss, which aims to minimize the distance of positive samples and maximize the distance of negative samples. The training process uses the self-optimization algorithm Adam to update the model parameters, and adopts L2 regularization and Dropout techniques to prevent overfitting.

[0109] (2) Building a hybrid recommendation system

[0110] In order to further improve the recommendation effect, a hybrid recommendation system is constructed by combining collaborative filtering and content recommendation methods. The specific settings are as follows:

[0111] (a) Model selection: The deep factorization machine model xDeepFM is used to capture both low-order and high-order interaction information of features. Figure 3 shown.

[0112] (b) Input features: mainly include user features, project features, etc.

[0113] User characteristics: including semantic vector representation of business needs, user historical behavior characteristics (such as click, browse, download records), user preference tags, etc.

[0114] Project characteristics: including the embedding vector of the data resource, metadata characteristics (such as data type, subject, creation time), content characteristics (such as text description, keywords), etc.

[0115] (c) Feature preprocessing: One-hot encoding or label encoding is performed on discrete features (such as categories and labels), and continuous features are normalized.

[0116] (d) Feature interaction: The model embeds the sparse features of the input, and then learns the complex interactive relationships between the features through the embedding layer, the compressed sensing network (CIN) and the fully connected network. Figure 4 shown.

[0117] (e) Prediction output: The model outputs the matching score between business requirements and data resources.

[0118] (f) Training strategy: A binary cross entropy loss function is used and historical interaction data is used for supervised training. The model learns the implicit relationship between users and data resources by minimizing the difference between predicted ratings and actual interaction behaviors.

[0119] (3) Data resource sorting and filtering

[0120] (a) Multi-index sorting

[0121] The candidate data resources are ranked by comprehensively considering factors such as matching score, data quality, update frequency, access rights, credibility of data sources, etc. The weighted comprehensive scoring method is adopted to assign different weights to each indicator according to its importance and calculate the total score.

[0122] (b) Filtering strategy

[0123] Filter data resources that do not meet the conditions according to the specific constraints of business requirements (such as the timeliness, region, and field of data). Filter data resources that users do not have access to or involve sensitive information according to user permissions and data security policies. Exclude data resources with low quality, poor historical evaluation, or explicitly marked as irrelevant by users.

[0124] (4) Recommendation result generation and feedback

[0125] (a) Results display

[0126] According to the comprehensive scores, the candidate data resources are sorted in descending order to generate a recommendation list. The recommended data resource list is displayed on the user interface, including the name of the data resource, a brief description, matching score, key indicators and other information.

[0127] (b) Interactive Feedback

[0128] Allow users to operate on the recommendation results, such as viewing details, collecting, downloading, and evaluating, and record user interaction behaviors and feedback information (such as clicks, dwell time, and ratings) as incremental training data for the model.

[0129] Using user feedback data collected in real time, the incremental learning algorithm SGD is used to update model parameters to improve the accuracy and personalization of recommendations. The model is retrained regularly to incorporate the latest data and feedback, and to update feature representation and model structure.

[0130] S3. Data processing flow generation

[0131] S3.1. Automated workflow generation

[0132] (a) Generation based on reinforcement learning

[0133] The generation of the data processing flow is modeled as a Markov Decision Process (MDP), where:

[0134] State space (S): It consists of the current business requirements, selected data resources, and current process steps. The state representation is in the form of a feature vector, which includes the semantic vector of the business requirements, the embedding vector of the data resources, and the process history records.

[0135] Action space (A): Contains a set of optional data processing operations, including data preprocessing methods (such as missing value filling, data cleaning, data normalization), feature engineering methods (such as feature selection, feature extraction, feature conversion), model training algorithms (such as linear regression, decision tree, random forest, support vector machine, neural network), etc.

[0136] Strategy (π): The policy network selects the best next action based on the current state to maximize the cumulative reward. The policy network uses a deep neural network structure, with the input being the state feature vector and the output being the probability distribution or value evaluation of the action.

[0137] Reward function (R): Give corresponding rewards based on the performance indicators of the generated process on the validation set (such as accuracy, recall, F1-score, execution time, resource consumption). The reward function is designed to guide the model to generate efficient and high-performance processing flows. The specific formula is:

[0138] R=αPerformance-βTime-γResource

[0139] Among them, α, β, and γ are weight coefficients, Performance represents the performance index of the process, and Time and Resource represent the execution time and resource consumption respectively.

[0140] State transition function (T): After executing the selected action, the process transfers from the current state to the next state, updating the completed steps and the remaining operation space.

[0141] Reinforcement learning algorithm: Use deep reinforcement learning algorithms, such as Deep Q-Network (DQN) or Policy Gradient method, to train the policy network. The network structure includes input layer (state representation), hidden layer (multi-layer perceptron or convolutional layer), and output layer (action value or policy distribution).

[0142] Model training: Use Experience Replay and a fixed target network to stabilize the training process and prevent the model from falling into a local optimum. The key steps in the training process include:

[0143] Collect experience: interact with the environment under the current strategy to generate a state-action-reward-next state (SARS) sequence.

[0144] Update strategy: Use the collected experience to update the parameters of the strategy network according to the Bellman equation to minimize the loss function.

[0145] Strategy improvement: Regularly update the target network to ensure the stability and convergence of training.

[0146] (b) Improvements based on meta-learning

[0147] In order to enable the model to quickly adapt to new business needs, the model-agnostic meta learning algorithm (MAML) is used. The specific implementation is as follows:

[0148] Task definition: Different business requirements are regarded as different tasks, and the meta-learning model is trained on multiple tasks.

[0149] Meta-training process:

[0150] Initialize the model parameters θ.

[0151] For each meta-training task T i :

[0152] Inner loop: Starting from θ, use a small amount of gradient update to get the task T i The parameter θ i :

[0153]

[0154] Where α is the learning rate, For task T i The loss function is represents the derivative of θ.

[0155] Calculating task loss

[0156] Outer loop: Update θ based on the losses of all tasks to minimize the total loss:

[0157]

[0158] Where β is the meta-learning rate.

[0159] S3.2 Dynamic Task Scheduling and Execution

[0160] (a) Workflow engine: The workflow engine is used to realize automatic scheduling and dependency management of data processing tasks to ensure efficient execution of the process.

[0161] (b) Containerization and microservice architecture: Containerization technologies such as Docker and Kubernetes are used to encapsulate data processing tasks into independent microservices to facilitate deployment and expansion, and improve the flexibility and maintainability of the system.

[0162] The present invention proposes an intelligent fusion and optimization method for multi-source heterogeneous data, which is suitable for scenarios such as data fusion management and new business development. The implementation of this method can achieve the following beneficial effects:

[0163] 1. Improve the efficiency of data pipeline construction: By automatically generating data processing processes, manual participation can be significantly reduced, the development cycle can be shortened, and the efficiency of data processing can be improved.

[0164] 2. Enhance the flexibility and adaptability of the system: By using multimodal deep learning models and meta-learning algorithms, the system can quickly adapt to new business needs and data changes, and improve its ability to respond to diverse needs.

[0165] 3. Optimize data resource utilization: Through intelligent analysis and matching of existing data resources, fully tap the value of data, improve data reuse rate, and reduce data management and storage costs.

[0166] 4. Improve the accuracy and reliability of data processing results: Use deep semantic matching models and reinforcement learning strategies to optimize the data processing process, ensure high accuracy and stability of the results, and meet business requirements for data quality.

[0167] 5. Reduce development and operation costs: Automated and intelligent data pipeline construction reduces manpower input and error rates, reduces system development and maintenance costs, and improves overall operational efficiency.

[0168] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the technical principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A method for constructing a dynamic data pipeline based on artificial intelligence and multimodal data processing, characterized in that: The method comprises the following steps: S1. Business requirements understanding: Use multimodal deep learning models to perform semantic analysis on business documents, user requirements, and functional specifications, extract key indicators, data requirements, and logical constraints; including: semantic analysis, knowledge graph fusion, intent recognition and slot filling, and structured requirements representation generation steps; S2. Data resource analysis and matching: Construct a multimodal knowledge graph, uniformly represent and extract features of structured and unstructured data resources, and use graph neural networks for embedded learning; based on deep semantic matching models and hybrid recommendation systems, accurately match business needs with data resources, and sort and filter candidate data resources; including: multimodal knowledge graph construction and intelligent data recommendation steps; S3. Data processing flow generation: Using reinforcement learning strategies, the generation of data processing flows is modeled as a Markov decision process, and optimized data processing workflows are automatically generated. Meta-learning algorithms are used to improve the model's adaptability to new business needs. Dynamic scheduling and execution of tasks are achieved through a workflow engine, and containerization and microservice architecture are used to improve the flexibility and maintainability of the system. This includes: automated workflow generation, dynamic task scheduling, and execution steps.

2. The method for constructing a dynamic data pipeline based on artificial intelligence and multimodal data processing according to claim 1, characterized in that: The semantic parsing step includes: data preprocessing, multimodal feature encoding, domain adaptive pre-training and multi-task learning; Data preprocessing: Preprocess multimodal data related to business needs, including text, images, audio, video, documents and tables, to eliminate data noise and unify data formats; the processing of text data includes word segmentation, stop word removal, part-of-speech tagging and text normalization operations; for images and videos, use data cleaning, resizing, normalization and data enhancement methods to process visual data; audio data obtains high-quality spectrum information through noise reduction and spectrogram conversion processes; document and table data use OCR recognition, layout analysis and table structure recognition algorithms to convert extracted text and table data into a structured format; Multimodal feature encoding: Use the multimodal pre-trained model to encode the preprocessed multimodal data to generate a high-dimensional semantic vector representation. Input the data of text, image, audio, and video modalities into the corresponding encoder to extract their respective feature representations. Through the cross-modal alignment mechanism, the features of different modalities are mapped to a unified semantic space. Domain-adaptive pre-training: Based on the general pre-training model, a large amount of unlabeled multimodal data in the domain is used for domain-adaptive pre-training. The tasks used include masked language model and masked region model to capture domain-specific language patterns, visual concepts and cross-modal associations. Multi-task learning: Build a multi-task learning model to perform multiple related natural language processing and computer vision tasks simultaneously, including text classification, sequence labeling, relation extraction, image classification, and object detection. By sharing model parameters, information between different tasks can promote each other.

3. The method for constructing a dynamic data pipeline based on artificial intelligence and multimodal data processing according to claim 2, characterized in that: The knowledge graph fusion step includes: Domain knowledge graph construction: Based on domain expert knowledge and public data sources, a knowledge graph containing domain entities and relationships is constructed. The nodes in the knowledge graph represent concepts, entities, and terms in the domain, and the edges represent the relationships between entities. Knowledge graph embedding: Entities, relationships, and multimodal information are mapped into low-dimensional vector representations through knowledge graphs to capture the semantic associations and structural information between entities. For nodes containing multimodal information, multimodal graph neural networks (MM-GNN) are used for embedding to fuse features of different modalities. Semantic representation fusion: The knowledge graph embedding is integrated with the semantic vector representation generated by the multimodal pre-trained model to generate an enhanced semantic representation that integrates the knowledge graph. When both representations need to be used at the same time without favoring either one, the semantic vector of the pre-trained model is directly concatenated with the knowledge graph embedding vector to form a new feature vector. When a certain representation needs to be emphasized according to the context, the two vectors are weighted and summed according to a certain weight to balance the model's attention to pre-trained features and knowledge graph information. When the model needs to focus on certain parts of the input data, the attention mechanism is used to calculate the correlation weights between the input data elements and the knowledge graph entities, and the fusion ratio is dynamically adjusted to enable the model to focus on semantic information at different levels.

4. The method for constructing a dynamic data pipeline based on artificial intelligence and multimodal data processing according to claim 3, characterized in that: The intention identification and slot filling steps include: Build an intent recognition model: Based on the enhanced semantic representation integrated with the knowledge graph, build an intent classification model. The model uses Roberta through a fully connected layer and a Softmax activation function to output the intent category corresponding to the business needs. Build a slot filling model: Use the sequence annotation model BERT-BiLSTM-CRF to annotate the business requirement text word by word and identify the slots and their values ​​related to the business requirements; Attention fusion: A multi-head attention mechanism is introduced into the intent recognition model and slot filling model, focusing on words and phrases in the text that are closely related to business needs. By calculating the importance weight of each word in the text sequence, the model's ability to capture key information is improved.

5. The method for constructing a dynamic data pipeline based on artificial intelligence and multimodal data processing according to claim 4, characterized in that: The structured requirement representation generation step comprises: Result integration: Integrate the results of intent recognition and slot filling to form a complete structured demand representation; use the identified intent as the root node, the slot as the intermediate node, and the slot value as the leaf node to clearly represent the various components of the demand and their logical associations; clarify the dependencies and constraints between the slots through the hierarchical relationship of the tree structure; Data standardization: standardize slot values ​​to ensure data consistency and standardization; Storage and management: Store structured demand representations in relational databases or graph databases to facilitate subsequent queries and calls; Interface provision: Provides standardized interfaces for subsequent data resource analysis and matching modules, and supports the acquisition and transmission of structured demand representations.

6. The method for constructing a dynamic data pipeline based on artificial intelligence and multimodal data processing according to any one of claims 1 to 5, characterized in that: The multimodal knowledge graph construction step includes: Unified representation of data resources: For structured data, use database metadata extraction tools to obtain its mode and attribute information. The extracted features include table and field information, data statistical features, and relationship features. For unstructured data, use feature extraction algorithms to convert it into a structured representation and perform multimodal feature fusion. The data resource entity information obtained from the above processing is used as the nodes of the knowledge graph, and the edges are used to represent the relationships between nodes, including data association relationships, semantic relationships, and multimodal relationships. On this basis, the richness and accuracy of the knowledge graph are further improved through knowledge fusion and completion. The extracted entities and relationships are mapped to the existing knowledge base through knowledge base alignment to expand the graph content. At the same time, the knowledge graph completion algorithm TransE is used to infer potential but unlabeled relationships to improve the structure and information of the knowledge graph. Graph neural network embedding: Graph Transformer, a graph neural network model suitable for large-scale knowledge graphs, is used to process node neighborhood information and capture global and local features in the graph structure. The GNN model is trained in an unsupervised or semi-supervised manner according to the amount of labeled data. Unsupervised training uses random walks and negative sampling techniques to minimize the distance between similar nodes and maximize the distance between dissimilar nodes. Semi-supervised training uses partially labeled node information and adopts a cross-entropy loss function to guide the model to learn effective embedding representations. Feature aggregation: For each node, the GNN model updates its own representation by aggregating the features of its neighboring nodes; Embedding learning: Define the initial feature vector of the node, including the attribute features and category information of the node, calculate through multiple layers of GNN, iteratively update the feature representation of the node, and finally obtain the high-dimensional embedding vector of each node.

7. The method for constructing a dynamic data pipeline based on artificial intelligence and multimodal data processing according to claim 6, characterized in that: The intelligent data recommendation step comprises: Constructing a deep semantic matching model: adopting a dual-tower model structure, which consists of two deep neural networks with shared parameters, processing the descriptions of business requirements and data resources respectively; the input representation includes the input representation corresponding to the business requirement description and the data resource description; feature encoding: for text input, use a pre-trained language model for encoding to obtain a semantically rich vector representation, and for non-text input, use a fully connected layer or autoencoder for encoding; semantic vector generation: obtain the semantic vector representation of business requirements and data resources through forward propagation; similarity calculation: use cosine similarity, Euclidean distance or a matching function based on a neural network to calculate the matching degree between business requirements and data resources; using the contrastive learning method, construct positive and negative sample pairs, where the positive sample is the business requirement and the related data resource, and the negative sample is the business requirement and the unrelated data resource. The selection of positive and negative samples is based on historical interaction data or expert annotations; the loss function selects contrastive loss or triplet loss, which aims to minimize the distance of positive samples and maximize the distance of negative samples. The training process uses the self-optimization algorithm Adam to update the model parameters, and uses L2 regularization and Dropout and other technologies to prevent overfitting; Build a hybrid recommendation system: Use the extremely deep factorization machine model xDeepFM to capture both low-order and high-order interaction information of features; input features include user features and project features; One-Hot encoding or LabelEncoding is performed on discrete features, and continuous features are normalized; the model embeds the input sparse features, and then learns the complex interaction relationship between features through the embedding layer, compressed sensing network CIN and fully connected network; the model outputs the matching score between business needs and data resources; adopts the binary cross entropy loss function and uses historical interaction data for supervised training. The model learns the implicit relationship between users and data resources by minimizing the difference between predicted scores and actual interaction behaviors; Data resource sorting and filtering: Consider the matching score, data quality, update frequency, access rights, and data source credibility factors, sort the candidate data resources, and use the weighted comprehensive scoring method to assign different weights to each indicator according to its importance to calculate the total score; filter out data resources that do not meet the conditions according to the specific constraints of business needs; filter out data resources that users do not have access to or involve sensitive information according to user permissions and data security policies, and exclude data resources with low quality, poor historical evaluation, or explicitly marked as irrelevant by users; Recommendation result generation and feedback: Based on the comprehensive score, the candidate data resources are sorted in descending order to generate a recommendation list, and the recommended data resource list is displayed in the user interface; users are allowed to operate on the recommendation results, and the user's interactive behavior and feedback information are recorded as incremental training data for the model; using real-time collected user feedback data, the incremental learning algorithm SGD is used to update the model parameters to improve the accuracy and personalization of recommendations; the model is retrained regularly to incorporate the latest data and feedback, and to update the feature representation and model structure.

8. The method for constructing a dynamic data pipeline based on artificial intelligence and multimodal data processing according to claim 7, characterized in that: The automated workflow generation includes: generation based on reinforcement learning: modeling the generation of data processing flow as a Markov decision process, State space: It consists of current business requirements, selected data resources, and current process steps. The state is represented in the form of feature vectors, which include the semantic vectors of business requirements, the embedded vectors of data resources, and the process history records. Action space: contains a set of optional data processing operations, including data preprocessing methods, feature engineering methods, and model training algorithms; Strategy: The strategy network selects the best next action based on the current state to maximize the cumulative reward. The strategy network uses a deep neural network structure, with the input being the state feature vector and the output being the probability distribution or value assessment of the action. Reward function: Give corresponding rewards based on the performance indicators of the generated process on the validation set; the reward function is designed to guide the model to generate efficient and high-performance processing processes. The specific formula is: R=αPerformance-βTime-γResource Among them, α, β, and γ are weight coefficients, Performance represents the performance index of the process, and Time and Resource represent the execution time and resource consumption respectively; State transfer function: After executing the selected action, the process transfers from the current state to the next state, updating the completed steps and the remaining operation space; Reinforcement learning algorithm: Use deep reinforcement learning algorithm to train the strategy network; the network structure includes input layer, hidden layer, and output layer; Model training: Use experience replay and fixed target network to stabilize the training process and prevent the model from falling into local optimality; the key steps in the training process include: Collect experience: interact with the environment under the current strategy to generate a state-action-reward-next state sequence; Update strategy: Use the collected experience to update the parameters of the strategy network according to the Bellman equation to minimize the loss function; Strategy improvement: Regularly update the target network to ensure the stability and convergence of training; Improvements based on meta-learning: The model-independent meta-learning algorithm MAML is used to improve the model's adaptability to new business needs.

9. The method for constructing a dynamic data pipeline based on artificial intelligence and multimodal data processing according to claim 8, characterized in that: The meta-learning algorithm MAML includes: Task definition: Different business requirements are considered as different tasks, and the meta-learning model is trained on multiple tasks; Meta-training process: Initialize model parameters θ; For each meta-training task T i : Inner loop: Starting from θ, use a small amount of gradient update to get the task T i The parameter θ i : Where α is the learning rate, For task T i The loss function is represents the derivative of θ; Calculating task loss Outer loop: Update θ based on the losses of all tasks to minimize the total loss: Where β is the meta-learning rate.

10. The method for constructing a dynamic data pipeline based on artificial intelligence and multimodal data processing according to claim 8, characterized in that: The dynamic task scheduling and execution includes: Workflow engine: Use the workflow engine to realize automatic scheduling and dependency management of data processing tasks; Containerization and microservice architecture: Docker and Kubernetes containerization technologies are used to encapsulate data processing tasks into independent microservices for easy deployment and expansion.

Images