Data processing rule generation method and device, equipment, storage medium and product

Abstract

The invention discloses a data processing rule generation method and device, equipment, a storage medium and a product, which are applied to the technical field of financial information, and the method comprises the following steps: obtaining a data processing demand of a target project, and carrying out feature extraction to obtain a data processing demand vector; calculating the similarity between the data processing demand vector and a plurality of preset reference vectors, wherein the preset reference vectors are determined according to reference names and reference definitions of metadata in a plurality of data sources related to the target item; according to the similarity between the data processing demand vector and each preset reference vector, determining a fusion weight of each preset reference vector; carrying out weighted fusion on each preset reference vector according to the fusion weight, and determining the vector after weighted fusion as a data processing rule vector; and inputting the data processing rule vector into a trained data processing rule generation model, and outputting a data processing rule corresponding to the data processing demand by adopting the model. The accuracy of the generated data processing rule can be improved.

Description

Technical Field

[0001] This application belongs to the field of financial information technology, and in particular relates to a method, apparatus, device, storage medium and product for generating data processing rules. Background Technology

[0002] Many large enterprises disperse key business data across multiple independent software systems during their operations and management. For example, data from technology projects is often scattered across project management systems, code repositories, and operations and maintenance platforms. Therefore, to complete a comprehensive project management task, such as project risk warning or task expediting, it is necessary to extract project data from multiple heterogeneous data sources according to the data processing rules of different businesses, perform cleaning, correlation, and fusion calculations, and trigger corresponding processing actions based on the calculation results.

[0003] Currently, in actual project processing, data processing rules also need to be added or modified as business needs change. Compared to relying on technical personnel to manually write or modify data processing rules, natural language processing models are more efficient. However, when using NLP models trained on general corpora to generate or modify data processing rules for specific projects, they have difficulty accurately identifying and understanding technical terms. This results in data processing rules that, while grammatically correct, have semantics that deviate significantly from actual business needs, making them unusable in practice.

[0004] In summary, current natural language processing models trained on general corpora are not accurate enough when used to generate data processing rules. Summary of the Invention

[0005] This application provides a method, apparatus, device, and computer storage medium for generating data processing rules, which can generate data processing rules that accurately reflect the semantics of data processing needs and improve the accuracy of the generated data processing rules.

[0006] In a first aspect, embodiments of this application provide a method for generating data processing rules, the method comprising: Obtain the data processing requirements of the target project, and extract features from the data processing requirements to obtain the data processing requirement vector; Calculate the similarity between the data processing requirement vector and multiple preset reference vectors of the target project. The preset reference vectors are determined based on the reference names and reference definitions of metadata in multiple data sources involved in the target project. Based on the similarity between the data processing requirement vector and each of the preset reference vectors, the fusion weight of each preset reference vector is determined; The preset reference vectors are weighted and fused according to their respective fusion weights, and the weighted and fused vector is determined as the data processing rule vector. The data processing rule vector is input into the trained data processing rule generation model, and the trained data processing rule generation model is used to output the data processing rule corresponding to the data processing requirement.

[0007] Secondly, embodiments of this application provide a data processing rule generation apparatus, the apparatus comprising: The acquisition module is used to acquire the data processing requirements of the target project, and to extract features from the data processing requirements to obtain the data processing requirement vector. The calculation module is used to calculate the similarity between the data processing requirement vector and multiple preset reference vectors of the target project. The preset reference vectors are determined based on the reference names and reference definitions of metadata in multiple data sources involved in the target project. The determining module is used to determine the fusion weight of each preset reference vector based on the similarity between the data processing requirement vector and each preset reference vector; The fusion module is used to perform weighted fusion on each of the preset reference vectors according to the fusion weight of each preset reference vector, and determine the weighted fused vector as the data processing rule vector; The generation module is used to input the data processing rule vector into the trained data processing rule generation model, and use the trained data processing rule generation model to output the data processing rule corresponding to the data processing requirement.

[0008] Thirdly, embodiments of this application provide an electronic device, the device comprising: A processor and a memory storing computer program instructions; a method for generating data processing rules that implement any of the above when the processor executes the computer program instructions.

[0009] Fourthly, embodiments of this application provide a computer storage medium on which computer program instructions are stored, and when the computer program instructions are executed by a processor, a method for generating data processing rules that implements any of the above-mentioned methods.

[0010] Fifthly, embodiments of this application provide a computer program product in which instructions, when executed by a processor of an electronic device, enable the electronic device to execute a method for generating data processing rules as described above.

[0011] The data processing rule generation method, apparatus, device, computer storage medium, and computer program product of this application embodiment obtain the data processing requirements of a target project, extract features from the data processing requirements to obtain a data processing requirement vector, calculate the similarity between the data processing requirement vector and multiple preset reference vectors of the target project, accurately associate the data processing requirement vector with the preset reference vectors of the target project, avoid terminological ambiguity and misunderstanding, determine the fusion weight of each preset reference vector based on the similarity between the data processing requirement vector and each preset reference vector, determine the importance of different preset reference vectors in the data processing requirements, perform weighted fusion of each preset reference vector based on the fusion weight of each preset reference vector, and determine the weighted fused vector as the data processing rule vector, so that the data processing rule vector accurately reflects the semantics of the data processing requirements, and input the data processing rule vector into a trained data processing rule generation model, and use the trained data processing rule generation model to output the data processing rules corresponding to the data processing requirements, generate data processing rules that accurately reflect the semantics of the data processing requirements, and improve the accuracy of the generated data processing rules. Attached Figure Description

[0012] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments of this application will be briefly introduced below. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0013] Figure 1 This is a flowchart illustrating a method for generating data processing rules according to an embodiment of this application; Figure 2 This is a schematic diagram of the structure of a data processing rule generation apparatus provided in another embodiment of this application; Figure 3 This is a schematic diagram of the structure of an electronic device provided in another embodiment of this application. Detailed Implementation

[0014] The features and exemplary embodiments of various aspects of this application will be described in detail below. To make the objectives, technical solutions, and advantages of this application clearer, the application will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only intended to explain this application and not to limit it. For those skilled in the art, this application can be implemented without some of these specific details. The following description of the embodiments is merely to provide a better understanding of this application by illustrating examples.

[0015] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising..." does not exclude the presence of additional identical elements in the process, method, article, or apparatus that includes said element.

[0016] It should be noted that the acquisition, storage, use, and processing of data in this application embodiment all comply with the relevant provisions of national laws and regulations.

[0017] It should be noted that in the embodiments of this application, certain software, components, models and other existing solutions in the industry may be mentioned. These should be regarded as exemplary and are only intended to illustrate the feasibility of implementing the technical solution of this application. However, it does not mean that the applicant has used or necessarily used the solution.

[0018] First, the background technology involved in this application will be introduced.

[0019] Many large enterprises disperse key business data across multiple independent software systems during their operations and management. For example, data from technology projects is often scattered across project management systems, code repositories, and operations and maintenance platforms. Therefore, to complete a comprehensive project management task, such as project risk warning or task expediting, it is necessary to extract project data from multiple heterogeneous data sources according to the data processing rules of different businesses, perform cleaning, correlation, and fusion calculations, and trigger corresponding processing actions based on the calculation results.

[0020] Currently, in actual project processing, data processing rules also need to be added or modified as business needs change. Compared to relying on technical personnel to manually write or modify data processing rules, natural language processing models are more efficient. However, when using NLP models trained on general corpora to generate or modify data processing rules for specific projects, they have difficulty accurately identifying and understanding technical terms. This results in data processing rules that, while grammatically correct, have semantics that deviate significantly from actual business needs, making them unusable in practice.

[0021] In summary, current natural language processing models trained on general corpora are not accurate enough when used to generate data processing rules.

[0022] To address the problems in the prior art, embodiments of this application provide a method, apparatus, device, computer storage medium, and computer program product for generating data processing rules.

[0023] The method for generating data processing rules provided in the embodiments of this application will be described below.

[0024] Figure 1 This is a flowchart illustrating a method for generating data processing rules according to an embodiment of this application. Figure 1 As shown, the execution subject of the data processing rule generation method provided in one embodiment of this application can be an electronic device, and the method may include steps 101 to 105.

[0025] Step 101: Obtain the data processing requirements of the target project, and extract features from the data processing requirements to obtain a data processing requirement vector.

[0026] A target project refers to a project with an independent and complete data system and specific business rules. For example, a target project could be an online credit approval system in a commercial bank or a user profiling analysis module in an e-commerce platform. A target project has its own dedicated data source, data model, and defined business metrics.

[0027] Data processing requirements refer to the expression of intent to manipulate, analyze, or acquire project data in order to achieve specific business objectives. Data processing requirements can take many forms, including natural language problem descriptions, sketches drawn during meetings, voice messages containing requirements, existing script fragments, or combinations of these forms.

[0028] A data processing requirement vector is a fixed-dimensional numerical vector obtained by extracting and encoding features of data processing requirements through technical means. In mathematical space, the data processing requirement vector represents the deep semantics of the data processing requirements.

[0029] In some embodiments, the electronic device receives data processing requests sent by a client device, or it may receive data processing requests input by a user through a graphical user interface. The electronic device can directly extract features from the data processing requests, converting them into a numerical feature vector, i.e., a data processing request vector.

[0030] For example, when the data processing requirement is natural language text, the electronic device can use a natural language processing model to extract semantic features and determine the extracted semantic features as the data processing requirement.

[0031] When the data processing requirement is in a modality other than natural language text, the electronic device can first convert the data processing requirement into a natural language text representation, and then extract features from the converted natural language text to obtain a data processing requirement vector. Alternatively, the electronic device can use an encoder corresponding to the modality of the data processing requirement to extract features from the data processing requirement, and then determine the extracted features as the data processing requirement.

[0032] Step 102: Calculate the similarity between the data processing requirement vector and multiple preset reference vectors of the target project. The preset reference vectors are determined based on the reference names and reference definitions of metadata in multiple data sources involved in the target project.

[0033] A pre-defined reference vector is a standard semantic vector that is created and stored in advance to represent metadata in a target project. Each pre-defined reference vector corresponds to a piece of metadata in the target project. Metadata in a target project refers to structured information used to describe and define the core data assets and business logic of the project. For example, metadata of a target project may include, but is not limited to, the name, storage location, subject, and logical relationship with other data tables of a data table; the name, data type, data length, whether nullables are allowed, default value, and explanation or calculation description of the business meaning of data fields; and the name, detailed business definition, precise calculation formula or logical description, statistical period, responsible department, and related data source of business indicators.

[0034] A data table is the basic structural unit describing the data stored in a database. Data fields are the specific columns in a data table. Business metrics are quantitative standards used to measure business status or performance.

[0035] After obtaining the data processing requirement vector, the electronic device can calculate the similarity between the data processing requirement vector and each preset reference vector of the target project. Specifically, the similarity between the data processing requirement vector and the preset reference vector can be cosine similarity or dot product.

[0036] Step 103: Determine the fusion weight of each preset reference vector based on the similarity between the data processing requirement vector and each preset reference vector.

[0037] The fusion weight is a coefficient used to characterize the importance of a preset reference vector in the data processing requirements. The importance coefficient is a non-negative real number; the higher the value, the more critical the metadata corresponding to the preset reference vector is in the data processing requirements.

[0038] Electronic devices can normalize the similarity between the data processing requirement vector and each preset reference vector, and can determine the normalized similarity between each preset reference vector and the data processing requirement vector as the fusion weight.

[0039] Step 104: Perform weighted fusion on each preset reference vector according to the fusion weight of each preset reference vector, and determine the weighted fused vector as the data processing rule vector.

[0040] The electronic device can multiply each preset reference vector by its corresponding fusion weight, and then add all the weighted vectors together to obtain a new vector, namely the weighted fused vector, and can determine the weighted fused vector as the data processing rule vector.

[0041] Step 105: Input the data processing rule vector into the trained data processing rule generation model, and use the trained data processing rule generation model to output the data processing rules corresponding to the data processing requirements.

[0042] A trained data processing rule generation model is an artificial intelligence model trained on a large amount of data, capable of converting semantic vectors into structured program code or query statements. This model understands the semantics of data processing rule vectors and generates syntactically correct data processing scripts that conform to business logic.

[0043] Electronic devices can input data processing rule vectors into a trained data processing rule generation model. This model, through its internal decoding mechanism, can predict and generate structured, executable data processing rule code, such as an SQL query conforming to specific database syntax or a Python script for data cleaning. The data processing rules generated by the trained model accurately reflect the semantic intent of the rule vectors.

[0044] The method for generating data processing rules provided in the embodiments of this application will be described below through examples.

[0045] Suppose that in a bank's fintech project, a user proposes a data processing requirement to analyze the recent transaction patterns of high-risk customers.

[0046] In one example, the method for generating data processing rules provided in this application includes the following steps.

[0047] Step one: The electronic device transforms the natural language description of the sentence into a data processing requirement vector V using a text encoding model. req .

[0048] Step 2, the electronic device calculates V req The similarity to all pre-defined reference vectors of fintech projects. These pre-defined reference vectors can include vectors corresponding to a wide range of metadata, such as customer risk level, transaction time, transaction amount, and product type. Calculations show that V... reqThe data has the highest similarity to the preset reference vectors of the two metadata: high-risk customers and transaction records.

[0049] Step 3, the electronic device according to V req The similarity between the vectors and the pre-defined reference vectors of the fintech projects determines the fusion weights of the pre-defined reference vectors of the fintech projects. For example, for V... req The similarity to each preset reference vector of the fintech project is normalized, and the normalized similarity is determined as the fusion weight of each preset reference vector of the fintech project. Since V req The data has the highest similarity to the preset reference vectors of high-risk customers and transaction records. Therefore, the fusion weight of the preset reference vectors of high-risk customers and transaction records is also the highest.

[0050] Step four: The electronic device performs a weighted summation of the preset reference vectors of metadata such as high-risk customers, transaction records, transaction amounts, and transaction times, according to the fusion weights determined in step 104, to obtain a new data processing rule vector V. rule V rule This comprehensively expresses the intention to analyze the transaction amount and time distribution of high-risk customers.

[0051] Step 5, Electronic device V rule The trained data processing rule generation model is input into the pre-trained data. The pre-trained data processing rule generation model is based on V. rule The semantics are used to generate SQL rules that can query and summarize the transaction information of high-risk customers within a preset time window from multiple data sources.

[0052] The data processing rule generation method provided in this application obtains the data processing requirements of a target project, extracts features from the data processing requirements to obtain a data processing requirement vector, calculates the similarity between the data processing requirement vector and multiple preset reference vectors of the target project, and can accurately associate the data processing requirement vector with the preset reference vectors of the target project, avoiding ambiguity in terminology and misunderstanding. By determining the fusion weight of each preset reference vector based on the similarity between the data processing requirement vector and each preset reference vector, the importance of different preset reference vectors in the data processing requirements can be determined. By weighted fusion of each preset reference vector according to the fusion weight of each preset reference vector, and determining the weighted fused vector as the data processing rule vector, the data processing rule vector can accurately reflect the semantics of the data processing requirements. By inputting the data processing rule vector into a trained data processing rule generation model and using the trained data processing rule generation model to output the data processing rules corresponding to the data processing requirements, data processing rules that can accurately reflect the semantics of the data processing requirements can be generated, improving the accuracy of the generated data processing rules.

[0053] In some embodiments, in order to more accurately determine the data processing requirement vector, the data processing requirement is multimodal data and includes at least two of the following modalities: natural language text, images, audio, video, and code scripts. Step 101 involves feature extraction of the data processing requirement to obtain a refined data processing requirement vector, which includes steps 201 to 204.

[0054] Step 201: Input each modal data into the corresponding modal encoder, and use the modal encoder to extract the feature vector of each modal data in the original feature space.

[0055] Multimodal data refers to data that contains two or more different types of information carriers within the same data processing requirement. For example, a data processing requirement may simultaneously include a natural language text description and a related chart or image, or an audio description accompanied by a code script snippet.

[0056] A modal encoder is an algorithmic model specifically designed to process a particular type of data. Each modal encoder operates in its corresponding native feature space, transforming the raw data of that modality into feature vectors that can represent its core information. The native feature space is a high-dimensional mathematical space composed of the output feature vectors of a specific modal encoder, and the native feature spaces of different modalities are independent of each other.

[0057] Upon receiving a multimodal data processing request, the electronic device first classifies the input data. The natural language text portion is input into a text encoder to obtain text feature vectors. Simultaneously, each non-text modal data is input into its corresponding modal encoder. For example, image data is input into a visual encoder, audio data into an audio encoder, and code scripts into a code encoder. Each encoder operates independently, extracting feature vectors from its input data.

[0058] In some embodiments, the image encoder may employ models such as Vision Transformer. The encoding process includes: image segmentation, linear embedding, adding positional encoding, extracting features through a Transformer encoding layer, and finally pooling to obtain the image feature representation. During training, image-text pairs are used as samples.

[0059] Audio encoders can employ models such as DeepSpeech. The encoding process includes: audio sampling, Fourier transform, feature extraction, and neural network training to establish a mapping from speech to text, using audio-text pairs as samples during training.

[0060] Video encoders can employ models such as VideoPrism, which include a spatial encoder that processes single frames and a temporal encoder that processes inter-frame temporal relationships. During training, video caption pairs are used as samples.

[0061] Code or SQL script encoders can use technologies such as Code2Vec to parse source code into an abstract syntax tree, extract paths, and encode them into vectors to represent the semantics and structure of the code.

[0062] Step 202: Input the feature vectors of non-text modal data in the original feature space into the trained projection matrix corresponding to each non-text modal data, and use the trained projection matrix to map the feature vectors of non-text modal data in the original feature space to the text feature space, thereby obtaining the semantic alignment feature vectors of each non-text modality in the text feature space.

[0063] The text feature space is a high-dimensional semantic space composed of feature vectors generated by the text encoder processing natural language text. In this application embodiment, the text feature space serves as the baseline semantic space for the unified processing and understanding of all modal information.

[0064] A trained projection matrix is ​​a mathematical transformation function obtained through machine learning training, typically represented as a parameter matrix. Each non-textual modality has its corresponding trained projection matrix. The trained projection matrix is ​​used to perform a mapping transformation from the modality's native feature space to the textual feature space, achieving cross-modal semantic alignment.

[0065] The electronic device inputs the feature vector of each non-textual modality data in the native feature space into a trained projection matrix corresponding to that modality. Through the trained projection matrix, the information of different modalities is transformed into a unified and comparable textual semantic benchmark space, thereby generating a semantically aligned feature vector. The semantically aligned feature vector represents the semantics that are the same as or similar to those of the non-textual modality data in the textual feature space.

[0066] The trained projection matrix is ​​obtained by training a preset projection matrix using training data, which includes feature vectors of data processing requirements with the same or similar semantics in the text feature space and non-text feature space. The training objective of the projection matrix is ​​to make text and non-text data with the same or similar semantics close in distance in the projected space.

[0067] For any non-text modality, the training dataset for training its corresponding projection matrix includes multiple training samples. Each training sample includes non-text modality data samples and text modality data samples. Specifically, when the non-text modality is an image, the non-text modality data sample can be a chart image; when the non-text modality is audio, the non-text modality data sample can be a speech segment; when the modality is video, the non-text modality data sample can be a video segment; and when the non-text modality is code script, the non-text modality data sample can be a code file. The text modality data sample is a piece of natural language text, and the semantics of this natural language text are the same as or highly similar to the metadata described by the non-text modality data in the same sample.

[0068] Step 203: Determine the fusion weight of each modality based on the semantic clarity of each modality.

[0069] Semantic clarity is a quantitative assessment of the quality, reliability, and explicitness of feature vectors extracted from a given modality of input data. Semantic clarity reflects the contribution and confidence level of that modality of input in expressing the user's intent.

[0070] Electronic devices can evaluate the semantic clarity of each modality of data. Evaluation factors may include, but are not limited to: for image modalities, evaluating clarity and completeness of chart elements; for audio modalities, evaluating speech recognition confidence and background noise levels; for code / script modalities, evaluating grammatical correctness; and for text modalities, evaluating domain terminology density and syntactic completeness. Based on the resulting clarity score, a fusion weight is assigned to the semantic alignment feature vector of each modality. Higher semantic clarity results in a larger assigned fusion weight.

[0071] Step 204: Based on the fusion weights of each modality data, the semantic alignment feature vectors of each modality data in the text feature space are weighted and fused, and the fused vector is determined as the data processing requirement vector.

[0072] After obtaining the semantic alignment feature vectors and their fusion weights for all modalities in a unified text feature space, the electronic device performs a weighted fusion operation. Specifically, the semantic alignment feature vector of each modality is multiplied by its corresponding fusion weight, and then all weighted vectors are summed to obtain a new, comprehensive vector. This vector is then used as the final data processing requirement vector, achieving the fusion of multimodal information and making high-quality, high-definition modal data have a greater impact on the data processing requirement vector.

[0073] The method for generating data processing rules provided in the embodiments of this application will be described below through examples.

[0074] Suppose that in a data analysis project on an e-commerce platform, the user's data processing requirements include: a natural language text query showing the sales percentage of each category in the previous quarter, and a hand-drawn pie chart sketch illustrating the desired display format.

[0075] In one example, the method for generating data processing rules provided in this application includes the following steps.

[0076] Step one: The electronic device converts the text description into a text feature vector V using a text encoder. text Simultaneously, the pie chart sketch is processed by a visual encoder to obtain the feature vector V of the image in the original visual feature space. image_raw .

[0077] Step two, the electronic device will V image_raw Input a projection matrix W specifically designed for image modality training image After calculation, V image_aligned =W image ×V image_raw This yields the semantic alignment feature vector V of the image in the text feature space. image_aligned At this time, V image_aligned With V text They are located in the same semantic space and are comparable.

[0078] Step 3: The electronic device evaluates the semantic clarity of the two modal inputs. The text description is specific and clear, containing key information such as the previous quarter, product categories, and sales percentage, achieving a clarity score of S. text High. The hand-drawn sketches are rather rough, lacking precise data labels, and the clarity score is S. image Relatively low. Based on the clarity score, a higher fusion weight w is assigned to the text features. text Assign lower fusion weights w to image features image .

[0079] Step four: The electronic devices perform weighted fusion to calculate the final data processing requirement vector V. req =w text V text +w image ×V image_aligned Data processing requirement vector V req It comprehensively expresses semantic information from both text and image modalities, with text information as the primary element, while also taking into account the result display format expected by users.

[0080] The data processing rule generation method provided in this application supports users in making requests in diverse and natural ways by configuring dedicated encoders and projection matrices for different modalities, avoiding the loss caused by forced information transformation. By projecting non-textual modal features onto a unified text feature space, the problem of heterogeneous and incomparable multimodal information is solved, providing a unified input for subsequent processing. Through semantic clarity-based weighted fusion, the quality of different inputs can be evaluated, interference from low-quality inputs can be suppressed, the robustness and accuracy of the data processing request vector representation can be enhanced, and thus the accuracy of the generated data processing rules can be improved.

[0081] In some embodiments, in order to accurately generate data processing rules that meet actual business needs, steps 301 to 303 are included before step 201.

[0082] Step 301: Extract reference names and reference definitions of metadata from multiple data sources involved in the target project. The metadata includes at least two of the following: data tables, data fields, and business metrics.

[0083] Metadata is structured information describing the core data assets and business logic of a target project, and may include data tables, data fields, and business metrics. Reference names are unique identifiers for metadata within the project, such as table names, field names, and business metric names. Reference definitions are detailed textual descriptions of the business meaning, calculation rules, or technical specifications of this metadata.

[0084] Electronic devices can access multiple data sources involved in the target project, such as the database system, data warehouse, indicator management platform, and data dictionary documents to which the target project belongs. They can scan, parse, and extract naming information and definition descriptions of data tables, data fields, and business indicators from multiple data sources. For example, they can read table names and field names and their comments from the system tables of the database, and indicator names and calculation formulas from the configuration files of the indicator management platform.

[0085] Step 302: Concatenate the reference names and reference definitions of each metadata to obtain the metadata description text.

[0086] To integrate metadata identifiers and semantic information into a coherent text, electronic devices can concatenate the reference name of each metadata element with its corresponding reference definition to form a complete metadata description text. Predefined delimiters or fixed template formats can be used during concatenation to create a semantically rich text paragraph that each metadata element can understand by a natural language processing model.

[0087] Step 303: Input the metadata description text into the trained text encoding model, and use the trained text encoding model to output the preset reference vector corresponding to the metadata description text. The trained text encoding model is obtained by training a general text encoding model using professional terminology text corpus of the target project's industry.

[0088] The electronic device can sequentially input each metadata description text generated in step 302 into the trained text encoding model, and use the trained text encoding model to encode each metadata description text, outputting a preset reference vector corresponding to each metadata description text. Each preset reference vector uniquely represents the precise business semantics of its corresponding metadata in the vector space. Since the trained text encoding model is trained on a general text encoding model using professional terminology text corpus of the target project's industry, the trained text encoding model can deeply understand the terminology system, expression habits, and business logic of a specific industry, thereby accurately encoding the metadata description text, so that the preset reference vector can accurately reflect the professional concepts in the metadata description text.

[0089] The data processing rule generation method provided in this application extracts reference names and definitions of metadata from multiple data sources involved in the target project, concatenates the reference names and definitions of each metadata to obtain metadata description text, inputs the metadata description text into a trained text encoding model, and uses the trained text encoding model to output a preset reference vector corresponding to the metadata description text. This can transform each metadata description text into a semantic vector, thereby determining multiple preset reference vectors corresponding to the target project. When generating data processing rules subsequently, the preset reference vectors are used as a reference for understanding data processing requirements, fundamentally ensuring the accuracy of understanding the professional terms involved in the data processing requirements of the target project, avoiding semantic drift or misunderstanding that may occur in the vertical domain due to the general model, and improving the generated data processing rules to meet actual needs.

[0090] In some embodiments, in order to obtain more accurate fusion weights, step 103 is further refined to include steps 401 to 403.

[0091] Step 401: Determine the initial weights of each preset reference vector based on the similarity.

[0092] After calculating the similarity between the data processing requirement vector and all preset reference vectors, the electronic device can determine the initial weights of each preset reference vector based on their similarity to the data processing requirement vector. For example, the similarity between all preset reference vectors and the data processing requirement vector is normalized, for instance, using the Softmax function, so that the sum of the similarities between all preset reference vectors and the data processing requirement vector is 1, and each similarity value is a value between 0 and 1, with higher values ​​directly corresponding to higher similarity. The normalized similarity values ​​are then used as the initial weights of each preset reference vector.

[0093] Step 402: Obtain the business association relationship between the metadata corresponding to each preset reference vector. The business association relationship is represented as a graph structure or association matrix. The graph structure includes nodes and edges. Nodes are used to represent metadata, and edges are used to connect two metadata. The value of an edge is used to represent the association strength between the two metadata connected by the edge. The value of an element in the association matrix is ​​used to represent the association strength between the two metadata corresponding to the row index and column index of the element.

[0094] Business relationships are used to characterize the inherent, structured connections between different metadata within a target project. Business relationships can take the form of a graph structure or an association matrix. Since each predefined reference vector uniquely corresponds to one piece of metadata, the relationships between metadata directly define the semantic relationships between their corresponding predefined reference vectors.

[0095] In a graph structure, nodes represent metadata, and edges connect two pieces of metadata that are related in terms of business logic. Each edge has a numerical value, called its value, which quantifies the strength of the association between the two pieces of metadata connected by the edge. A higher association strength value indicates a closer connection between the two pieces of metadata in terms of business logic or technical dependence. In an association matrix, both row and column indices correspond to metadata, and the value of each element in the matrix represents the association strength between the two pieces of metadata corresponding to its row and column indices. An association matrix is ​​an equivalent mathematical representation of a graph structure.

[0096] Since there is a one-to-one correspondence between the preset reference vectors and the metadata, the graph structure of the metadata association is equivalent to a semantic association structure graph with the preset reference vectors as nodes, and the value of the edge also represents the association strength between the two preset reference vectors connected. In the association matrix, the value of each element can also be equivalent to representing the association strength between the two preset reference vectors of the metadata corresponding to its row index and column index.

[0097] Electronic devices can obtain the business relationships between the metadata corresponding to each preset reference vector by querying or calculating.

[0098] Step 403: Optimize the initial weights based on business relationships to obtain the optimized weights of each preset reference vector.

[0099] In a graph structure or association matrix defined by association relationships, the initial weight of a predefined reference vector is propagated and diffused to other predefined reference vectors along the edges representing the strength of association between its corresponding metadata and other metadata. Through this optimization process, the final output fused weights not only reflect the direct semantic matching degree between data processing requirements and individual predefined reference vectors, but also reflect the structural association information between predefined reference vectors throughout the entire target project. This can reasonably increase the weights of predefined reference vectors that are strongly associated with predefined reference vectors with high initial weights but have low initial weights, while suppressing the weights of predefined reference vectors that are isolated or weakly associated with predefined reference vectors with high initial weights.

[0100] Specifically, electronic devices can take the initial weights as input and perform iterative propagation calculations on a graph structure or association matrix that represents business relationships until the weight distribution stabilizes, thereby obtaining the optimized weights.

[0101] The method for generating data processing rules provided in the embodiments of this application will be described below through examples.

[0102] Suppose that in a financial analysis project, the user's data processing requirement is to analyze profit margin. Preset reference vectors include: V_ROI, corresponding to profit margin metadata; V_Revenue, corresponding to sales revenue metadata; V_Cost, corresponding to cost metadata; and V_Staff, corresponding to employee data.

[0103] In one example, the method for generating data processing rules provided in this application includes the following steps.

[0104] Step 1: Determine the initial weights of each preset reference vector based on similarity. ROI Because it is directly related to demand, it has the highest initial weight, for example, 0.7; V Revenue and V Cost To obtain moderate weights, such as 0.2 and 0.1; V Staff The weight is close to 0.

[0105] Step 2: Obtain the business relationship between the metadata corresponding to each preset reference vector. In the business relationship, the profit margin metadata is strongly dependent on the sales and cost metadata in the calculation formula, that is, the correlation strength value is high, while the number of employees is extremely weakly correlated with the first three.

[0106] Step 3: Optimize the initial weights based on business relationships. Specifically, using V... ROI V Revenue VCost V Staff Starting with the initial weights, it propagates on the network defined by the aforementioned correlation strength. Since V ROI With V Revenue V Cost The correlation strength is high, V ROI High weights will significantly flow to V Revenue and V Cost V ROI V Revenue V Cost V Staff The optimized weights may be distributed as 0.5, 0.3, 0.2, and 0.0.

[0107] The data processing rule generation method provided in this application determines the initial weight of each preset reference vector based on similarity, obtains the business association relationship between the metadata corresponding to each preset reference vector, optimizes the initial weight based on the business association relationship, and determines the fusion weight of each preset reference vector based on the optimized weight. This can overcome the one-sided weight allocation problem that may be caused by simply relying on surface semantic similarity, so that the fusion weight of each preset reference vector can more comprehensively and evenly reflect the true importance of each metadata in meeting the current needs, thereby improving the accuracy of the subsequently generated data processing rules and enabling the data processing rules to meet the actual business needs.

[0108] In some embodiments, in order to accurately optimize the initial weights of each preset reference vector, step 402 is further refined to include steps 501 to 504.

[0109] Step 501: Determine the semantic association between any two metadata items according to the reference definition.

[0110] Semantic relevance is used to quantify the semantic similarity of any two metadata items at the textual description level. A reference definition of metadata is a detailed textual description of its business meaning, computational logic, or technical specifications.

[0111] Electronic devices determine semantic relevance by calculating the semantic similarity between the reference names and reference definition texts of two metadata items. Specifically, a semantic model trained on domain-specific text can be used to encode the two texts into semantic vectors. Then, the cosine similarity between these two vectors is calculated, and this cosine similarity is used as the semantic relevance. A higher semantic relevance indicates that the two metadata items are more similar or related in their business descriptions.

[0112] Step 502: For any two metadata, if it is determined that there is a reference relationship in the reference definition of the metadata, then determine the degree of derivative dependency between the two metadata based on the number of references and the reference method.

[0113] Derivative dependency is used to quantify the strength of direct logical dependencies between metadata arising from their definitions. A reference relationship refers to a metadata definition that explicitly mentions or includes the name of another metadata.

[0114] Electronic devices identify whether other metadata names are explicitly referenced by parsing the reference definition text of metadata. For example, a formula for a business metric might contain the name of a specific data field, or a table description might link to another table name. If a reference exists, a dependency is established from the referenced metadata to the referrer's metadata. The degree of association can be quantified based on the number of references and their centrality in the definition; for example, direct references in calculation formulas are given higher strength, while mentions in descriptive text are given lower strength.

[0115] Step 503: Determine the type association between any two metadata based on their types.

[0116] Type association is established based on the classification attributes of the metadata itself. Metadata types include data tables, data fields, business metrics, etc.

[0117] Electronic devices can determine type association based on preset classification rules. For example, if two metadata items belong to the same predefined type, business subject area, or naming prefix specification, a basic positive association is assigned between them. For instance, all data fields belonging to the same customer information subject have a basic association; all business indicators that are financial indicators have a basic association.

[0118] Step 504: Determine the business association strength between any two metadata items based on semantic association, derived dependency association, and type association, and construct the business association relationship based on the association strength.

[0119] Business association strength is a comprehensive quantitative value that measures the overall degree of closeness between any two pieces of metadata.

[0120] Electronic devices can fuse semantic relevance, derived dependency relevance, and type relevance. The fusion method can be weighted summation, where a weight coefficient is assigned to each relevance level, and the weighted sum is calculated as the final business relevance strength. Based on this relevance strength value, the electronic device can then construct a graph structure or association matrix representing the business relevance relationships between metadata. In the graph structure, each metadata element is a node, and the business relevance strength between each pair of metadata elements is the value of the edge connecting them. In the association matrix, the values ​​of the matrix elements represent the business relevance strength between metadata pairs with corresponding row and column indices.

[0121] The data processing rule generation method provided in this application determines the semantic correlation between any two metadata based on a reference definition. For any two metadata, if it is determined that there is a reference relationship in the reference definition of the metadata, the derivative dependency correlation between the two metadata is determined based on the number of references and the reference method. Based on the type of the metadata, the type correlation between any two metadata is determined. This method can obtain the correlation between metadata in different dimensions, and thus determine the accurate correlation between metadata in the target project.

[0122] In some embodiments, to further improve the accuracy of the generated data processing rules, steps 601 to 603 are included after step 105.

[0123] Step 601: Extract project data from multiple data sources based on data processing rules, process the project data, and obtain data processing results.

[0124] The data processing result is the actual data output obtained after executing the data processing rules output by the trained data processing rule generation model. Its form can be a data table, statistical chart, numerical report or file, etc.

[0125] After generating data processing rules, electronic devices can execute these rules in a sandbox or test environment associated with the target project. The execution process may include: accessing the data source specified by the rules, extracting relevant project data, and performing calculations, transformations, filtering, or aggregation operations defined by the rules, based on the logic within the rules. Finally, data processing results are generated for user review and use, and presented to the user through a graphical interface or other means.

[0126] Step 602: Obtain correction feedback information for the data processing results.

[0127] Correction feedback can be information provided by users to the system regarding the initial data processing results, indicating how the results need to be adjusted to fully meet their business intentions. Correction feedback is used to characterize the difference between the data processing results and the user's expected results.

[0128] The electronic device can provide an interactive interface, allowing users to evaluate the data processing results obtained in step 601. Users can provide feedback in various ways, such as: directly editing the generated data processing rule script; marking the parts that need to be modified on the visualization results and adding textual explanations; or re-entering a more precise natural language description to express their desired results. The electronic device records these user interactions or inputs in a structured manner as corrective feedback information.

[0129] Step 603: Based on the correction feedback information, adjust at least one of the trained data processing rule generation model and the preset reference vector.

[0130] In some embodiments, the electronic device may employ corrective feedback information to adjust at least one of the trained data processing rule generation model and the preset reference vector to improve the accuracy of the data processing rules continuously generated after the trained data processing rule generation model.

[0131] Electronic devices can construct a high-quality supervised learning sample from the user-confirmed correction process, such as changes from the initial rule to the final rule. This sample includes the user's original data processing requirements, the initial rule, the correction instructions, and the final rule. Using such samples, an already trained data processing rule generation model can be incrementally trained or fine-tuned to optimize the model's parameters. This allows the model to directly generate rules that more closely resemble the user's expectations when encountering similar semantic needs in the future, reducing the need for corrections.

[0132] Corrective feedback may reveal subtle discrepancies between the semantics represented by the preset reference vector and the actual business understanding of the project. For example, a user might correct a system-generated rule related to revenue to explicitly point to after-tax revenue. This indicates that the semantic positioning of the preset reference vector, revenue, may need to be more precisely aligned with after-tax revenue. Based on such feedback, electronic devices can fine-tune the text encoding process that generates the preset reference vector, or directly make minor adjustments to the preset reference vector in the vector space, to better align it with the actual terminology and business definitions of the target project.

[0133] The data processing rule generation method provided in this application, by obtaining correction feedback information of data processing results, and adjusting at least one of the trained data processing rule generation model and the preset reference vector based on the correction feedback information, enables the trained data processing rule generation model and the preset reference vector of the target project to dynamically adapt to changes in business details and differences in user habits, thereby achieving the effect of becoming more accurate and intelligent with use, and improving the accuracy of the generated data processing rules.

[0134] The data processing rule generation method provided in this application can directly receive and understand multimodal requirements such as text, images, voice, and code, without manual conversion, reducing information loss. It can also transform ambiguous requirements into precise data processing rules, significantly reducing reliance on professional technicians to write and maintain scripts, and improving the efficiency and reliability of data acquisition and decision support. Furthermore, it possesses online learning and knowledge accumulation capabilities, and can continuously self-optimize as projects evolve and user feedback is received, demonstrating strong adaptability.

[0135] Based on the data processing rule generation method provided in the above embodiments, this application also provides specific implementations of the data processing rule generation apparatus. Please refer to the following embodiments.

[0136] Figure 2 This is a schematic diagram of the structure of a data processing rule generation apparatus provided in another embodiment of this application. For example... Figure 2 As shown, the data processing rule generation apparatus 20 provided in this application embodiment includes: The acquisition module 21 is used to acquire the data processing requirements of the target project, and to extract features from the data processing requirements to obtain a data processing requirement vector. The calculation module 22 is used to calculate the similarity between the data processing requirement vector and multiple preset reference vectors of the target project. The preset reference vectors are determined based on the reference names and reference definitions of metadata in multiple data sources involved in the target project. The determination module 23 is used to determine the fusion weight of each preset reference vector based on the similarity between the data processing requirement vector and each preset reference vector. The fusion module 24 is used to perform weighted fusion of each preset reference vector according to the fusion weight of each preset reference vector, and determine the weighted fused vector as the data processing rule vector; The generation module 25 is used to input the data processing rule vector into the trained data processing rule generation model, and use the trained data processing rule generation model to output the data processing rules corresponding to the data processing requirements.

[0137] In some possible implementations, the data processing requirement is multimodal data, including at least two of the following modalities: natural language text, images, audio, video, and code scripts. The acquisition module 21 is specifically used for: Each modal data is input into its corresponding modal encoder, and the modal encoder is used to extract the feature vector of each modal data in the original feature space. The feature vectors of non-text modal data in the original feature space are input into the trained projection matrix corresponding to each non-text modal data. The trained projection matrix is ​​then used to map the feature vectors of non-text modal data in the original feature space to the text feature space, thereby obtaining the semantic alignment feature vectors of each non-text modality in the text feature space. The trained projection matrix is ​​obtained by training a preset projection matrix using training data, which includes feature vectors of the same semantic data processing requirements in the text feature space and non-text feature space. The fusion weights of each modality are determined based on the semantic clarity of each modality. The semantic alignment feature vectors of each modality in the text feature space are weighted and fused according to the fusion weights of each modality, and the fused vector is determined as the data processing requirement vector.

[0138] In some possible implementations, the data processing rule generation device 20 further includes: The extraction module is used to extract reference names and reference definitions of metadata from multiple data sources involved in the target project. The metadata includes at least two of the following: data tables, data fields, and business metrics. The concatenation module is used to concatenate the reference names and reference definitions of various metadata to obtain metadata description text; The encoding module is used to input metadata description text into a trained text encoding model and output a preset reference vector corresponding to the metadata description text using the trained text encoding model. The trained text encoding model is obtained by training a general text encoding model using a corpus of professional terminology texts of the target project's industry.

[0139] In some possible implementations, module 23 is specifically used for: The initial weights of each preset reference vector are determined based on similarity. Obtain the business association relationship between the metadata corresponding to each preset reference vector. The business association relationship is represented as a graph structure or association matrix. The graph structure includes nodes and edges. Nodes are used to represent metadata, and edges are used to connect two metadata. The value of an edge is used to represent the association strength between the two metadata connected by the edge. The value of an element in the association matrix is ​​used to represent the association strength between the two metadata corresponding to the row index and column index of the element. Based on business relationships, the initial weights are optimized, and the optimized weights are used to determine the fusion weights of each preset reference vector.

[0140] In some possible implementations, module 23 is also specifically used for: Determine the semantic association between any two metadata items based on the reference definition; For any two metadata, if it is determined that there is a reference relationship in the reference definition of the metadata, then the degree of derivative dependency between the two metadata is determined according to the number of references and the reference method. Determine the type association between any two metadata items based on their type. Based on semantic relevance, derivative dependency relevance, and type relevance, determine the business relevance strength between any two metadata items, and construct business relationship based on the relevance strength.

[0141] In some possible implementations, the data processing rule generation device 20 further includes: The processing module is used to extract project data from multiple data sources based on data processing rules, process the project data, and obtain data processing results. The second acquisition module is used to acquire correction feedback information of the data processing results; The adjustment module is used to adjust at least one of the trained data processing rule generation model and the preset reference vector based on the correction feedback information.

[0142] The various modules of the data processing rule generation device provided in this application embodiment can achieve... Figure 1 It provides the functionality for each step of the data processing rule generation method and achieves the corresponding technical effects. For the sake of brevity, it will not be elaborated here.

[0143] Figure 3 This is a schematic diagram of the structure of an electronic device provided in another embodiment of this application. For example... Figure 3 As shown, the data processing rule generation method in the above embodiments can be implemented by an electronic device in this application embodiment. The electronic device 30 includes a processor 31 and a memory 32 storing computer program instructions. When the processor 31 executes the computer program instructions, it implements any of the data processing rule generation methods in the above embodiments.

[0144] The data processing rule generation method in the above embodiments can be implemented using a computer storage medium in this application. This computer storage medium stores computer program instructions; when these computer program instructions are executed by a processor, they implement any of the data processing rule generation methods in the above embodiments.

[0145] This application also provides a computer program product, including a computer program, which, when executed, implements any of the data processing rule generation methods described in the above embodiments.

[0146] It should be noted that the exemplary embodiments mentioned in this application describe methods or systems based on a series of steps or apparatus. However, this application is not limited to the order of the above steps; that is, the steps can be performed in the order mentioned in the embodiments, or in a different order, or several steps can be performed simultaneously.

[0147] The aspects of this disclosure have been described above with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this disclosure. It should be understood that each block in the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that these instructions, executable via the processor of the computer or other programmable data processing apparatus, enable the implementation of the functions / actions specified in one or more blocks of the flowchart illustrations and / or block diagrams. Such a processor can be, but is not limited to, a general-purpose processor, a special-purpose processor, a special application processor, or a field-programmable logic circuit. It is also understood that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can also be implemented by special-purpose hardware performing the specified functions or actions, or can be implemented by a combination of special-purpose hardware and computer instructions.

[0148] The above are merely specific embodiments of this application. Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, modules, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here. It should be understood that the protection scope of this application is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in this application, and these modifications or substitutions should all be covered within the protection scope of this application.

Claims

1. A method for generating data processing rules, characterized in that, The method includes: Obtain the data processing requirements of the target project, and extract features from the data processing requirements to obtain a data processing requirement vector; Calculate the similarity between the data processing requirement vector and multiple preset reference vectors of the target project. The preset reference vectors are determined based on the reference names and reference definitions of metadata in multiple data sources involved in the target project. Based on the similarity between the data processing requirement vector and each of the preset reference vectors, the fusion weight of each preset reference vector is determined; The preset reference vectors are weighted and fused according to their respective fusion weights, and the weighted and fused vector is determined as the data processing rule vector. The data processing rule vector is input into the trained data processing rule generation model, and the trained data processing rule generation model is used to output the data processing rule corresponding to the data processing requirement.

2. The method according to claim 1, characterized in that, The data processing requirement is multimodal data, including at least two of the following modalities: natural language text, images, audio, video, and code scripts; the feature extraction of the data processing requirement to obtain the data processing requirement vector includes: Each modal data is input into the corresponding modal encoder, and the modal encoder is used to extract the feature vector of each modal data in the original feature space; The feature vectors of non-text modal data in the original feature space are input into the trained projection matrix corresponding to each non-text modal data, and the trained projection matrix is ​​used to map the feature vectors of the non-text modal data in the original feature space to the text feature space, thereby obtaining the semantic alignment feature vectors of each non-text modality in the text feature space. The trained projection matrix is ​​obtained by training a preset projection matrix with training data, and the training data includes the feature vectors of the same semantic data processing requirements in the text feature space and non-text feature space. The fusion weights of each modality are determined based on the semantic clarity of each modality. The semantic alignment feature vectors of each modality data in the text feature space are weighted and fused according to the fusion weight of each modality data, and the fused vector is determined as the data processing requirement vector.

3. The method according to claim 1, characterized in that, Before calculating the similarity between the data processing requirement vector and multiple preset reference vectors of the target project, the method further includes: From the multiple data sources involved in the target project, extract the reference names and reference definitions of the metadata, which include at least two of the following: data tables, data fields, and business metrics; The reference names and their reference definitions of each metadata element are concatenated to obtain the metadata description text; The metadata description text is input into a trained text encoding model, and the trained text encoding model is used to output the preset reference vector corresponding to the metadata description text. The trained text encoding model is obtained by training a general text encoding model using a corpus of professional terminology texts of the industry to which the target project belongs.

4. The method according to claim 3, characterized in that, The step of determining the fusion weight of each preset reference vector based on the similarity between the data processing requirement vector and each preset reference vector includes: The initial weights of each of the preset reference vectors are determined based on the similarity. Obtain the business association relationship between the metadata corresponding to each preset reference vector, wherein the business association relationship is represented as a graph structure or an association matrix, the graph structure includes nodes and edges, the nodes are used to represent the metadata, the edges are used to connect two metadata, and the value of the edge is used to represent the association strength between the two metadata connected by the edge, and the value of the element in the association matrix is ​​used to represent the association strength between the two metadata corresponding to the row index and column index of the element; Based on the business relationship, the initial weights are optimized, and the optimized weights are used to determine the fusion weights of each preset reference vector.

5. The method according to claim 1, characterized in that, According to the method of claim 4, the step of obtaining the business association relationship between the metadata corresponding to each of the preset reference vectors includes: Determine the semantic association degree between any two of the metadata based on the reference definition; For any two metadata, if it is determined that there is a reference relationship in the reference definition of the metadata, then the degree of derivative dependency between the two metadata is determined according to the number of references and the reference method. Based on the type of the metadata, determine the type association between any two metadata items; Based on the semantic relevance, the derived dependency relevance, and the type relevance, the business relevance strength between any two metadata items is determined, and the business relevance relationship is constructed based on the relevance strength.

6. The method according to claim 1, characterized in that, After inputting the data processing rule vector into the trained data processing rule generation model and using the trained data processing rule generation model to output the data processing rule corresponding to the data processing requirement, the method further includes: Based on the data processing rules, project data is extracted from the multiple data sources, and the project data is processed to obtain data processing results; Obtain correction feedback information for the data processing results; Based on the correction feedback information, adjust at least one of the trained data processing rule generation model and the preset reference vector.

7. A data processing rule generation apparatus, characterized in that, The device includes: The acquisition module is used to acquire the data processing requirements of the target project, and to extract features from the data processing requirements to obtain the data processing requirement vector. The calculation module is used to calculate the similarity between the data processing requirement vector and multiple preset reference vectors of the target project. The preset reference vectors are determined based on the reference names and reference definitions of metadata in multiple data sources involved in the target project. The determining module is used to determine the fusion weight of each preset reference vector based on the similarity between the data processing requirement vector and each preset reference vector; The fusion module is used to perform weighted fusion on each of the preset reference vectors according to the fusion weight of each preset reference vector, and determine the weighted fused vector as the data processing rule vector; The generation module is used to input the data processing rule vector into the trained data processing rule generation model, and use the trained data processing rule generation model to output the data processing rule corresponding to the data processing requirement.

8. An electronic device, characterized in that, The device includes: a processor and a memory storing computer program instructions; the processor, when executing the computer program instructions, implements the method as described in any one of claims 1-6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer program instructions that, when executed by a processor, implement the method as described in any one of claims 1-6.

10. A computer program product, characterized in that, When the instructions in the computer program product are executed by the processor of the electronic device, the electronic device is able to perform the method as described in any one of claims 1-6.

Images