Data part scene intelligent matching and code-free assembling method and system

Abstract

The invention relates to a data part scene intelligent matching and code-free assembling method and system. The method comprises the steps that 1, scene requirements are analyzed; 2, multi-dimensional part matching is carried out; step 3, carrying out visual code-free assembly; step 4, protocol adaptive conversion; step 5, automatic test verification; according to the method, accurate matching and rapid combination application of the data parts and the business scenes are realized, the problems of low scene adaptation efficiency, complex assembly splicing and high technical threshold in traditional data product development are solved, and the circulation efficiency and scenarized application value of the data parts are greatly improved.

Description

Technical Field

[0001] This invention relates to the field of data element circulation technology, specifically to a method and system for intelligent matching and no-code assembly of data components in various scenarios, used for accurate matching and rapid combination of data components with business scenarios. Background Technology

[0002] With the accelerated marketization of data elements, data components (independently tradable data processing units) have become the core carriers for realizing data value. However, the current application of data components faces two major challenges: Low scenario adaptation efficiency: Traditional part matching relies on manual keyword search (such as searching by part name or description), which cannot accurately identify cross-domain adaptation scenarios. For example, the "user profile generation" scenario requires "regional part extraction", but the part name may not contain the keyword "user profile", resulting in matching omissions.

[0003] The development threshold for assembly is high: assembling parts requires technical personnel to write code to implement protocol conversion (such as REST→gRPC) and data flow logic, resulting in long development cycles, high testing costs, and non-technical personnel cannot participate.

[0004] Existing solutions have significant shortcomings: Semantic matching technology is based solely on text similarity and lacks comprehensive evaluation across multiple dimensions, including functionality, data patterns, and compliance attributes. The assembly platform lacks protocol adaptation capabilities and requires manual configuration of data conversion rules; Test verification relies on manual test case writing and cannot automatically generate scenario-based test cases; Therefore, there is an urgent need in the market for a method and system for intelligent matching and no-code assembly of data components to solve the above problems and promote the efficient circulation and value release of data elements. Summary of the Invention

[0005] This invention proposes a method and system for intelligent matching and no-code assembly of data components in various scenarios. The system includes scenario requirement analysis, multi-dimensional component matching, visual no-code assembly, adaptive protocol conversion, and automated testing and verification. This invention enables accurate matching and rapid combination of data components with business scenarios, solving the problems of low scenario adaptation efficiency, complex component splicing, and high technical threshold in traditional data product development. It significantly improves the circulation efficiency and scenario-based application value of data components.

[0006] The following definitions apply in this invention: Scenario-part matching: Based on the requirements of the target scenario (functionality, data, compliance, performance), identify and recommend the most suitable data parts from the parts library; No-code assembly: Configure the data flow relationship between parts through visual drag and drop, and complete the assembly of parts without writing code; Protocol Adaptive: Automatically identifies and converts interface protocols between different parts to achieve seamless data flow.

[0007] The present invention adopts the following solution: A method for intelligent matching and no-code assembly of data components in a given scenario includes the following steps: Step 1: Scenario Requirements Analysis; Extract the functional, data, compliance, and performance requirements of the scenario using natural language processing technology; Step 2: Multi-dimensional parts matching; Based on semantic indexing and multi-dimensional similarity calculation, the most suitable parts are recommended from the parts library; Step 3: Visualize and assemble without code; Configure data flow relationships between parts through a drag-and-drop interface; Step 4: Protocol adaptive conversion; Automatically identify and convert interface protocols between different parts; Step 5: Automated testing and verification; Generate and execute test cases to verify the functionality and performance of the combined product.

[0008] Furthermore, the input in step 1 is: the scenario requirements described in natural language, such as generating a financial risk control user profile, which should include region, spending power, and credit score.

[0009] Use the BERT model to extract functional requirements, data requirements, compliance requirements, and performance requirements from the scenario; construct a structured requirement feature vector.

[0010] Furthermore, in step 2, based on semantic indexing and multi-dimensional similarity calculation, the method for recommending the most suitable parts from the parts library is as follows: S21. Semantic index construction; The part metadata (functional description, input / output schema, compliance rules, etc.) is encoded into a 768-dimensional vector using BERT, and the part vectors are stored and managed using the Milvus vector database, supporting efficient similarity search.

[0011] S22, Multi-dimensional matching algorithm; A weighted comprehensive scoring model is adopted. Score = w1 * Functional fit + w2 * Data fit + w3 * Compliance fit + w4 * Performance fit; The multi-dimensional matching includes: Functional matching degree calculation is based on the cosine similarity of functional tags; Calculate data matching degree and evaluate input / output schema compatibility; Compliance matching degree calculation, checking the consistency of compliance rules; Performance matching degree calculation, comparing the degree of compliance of performance indicators; Weighted composite score: Score = w1 Functionality + w2 Data + w3 Compliance + w4 Performance.

[0012] S23. Recommendation results are generated; Candidate parts are sorted in descending order of their overall scores. Output the Top-N recommendation list (default N=5). Provide detailed scores for each dimension of the recommendation with an explainable rationale.

[0013] Furthermore, the visual no-code assembly in step 3 includes: Drag-and-drop component layout; Drag and drop recommended parts from the parts library onto the canvas area to automatically generate part icons and basic information previews; Connected data stream orchestration; The data flow between parts is defined by connecting lines, the mapping relationship between input and output fields is automatically identified, and conditional branching and loop control structures are supported. Graphical parameter configuration; The system allows for graphical configuration of input parameter default values, setting output field aliases and format conversion rules, and defining error handling strategies and retry mechanisms.

[0014] Real-time data mapping preview.

[0015] Furthermore, the protocol adaptive conversion in step 4 includes: Multi-protocol identification: REST / gRPC / WASI; Unified intermediate format conversion; Enhanced data format validation; It validates data format integrity based on JSON Schema, automatically fills in default values ​​for missing fields, and automatically converts data types such as string to number.

[0016] Message queues deliver messages asynchronously. Asynchronous data delivery is achieved using Kafka message queues, which support data buffering and flow control, and provide at least-once delivery semantics.

[0017] Furthermore, the automated testing and verification in step 5 includes: Test cases are generated automatically; Automatically generate scenario-based test cases based on component test cases, supporting boundary value analysis and abnormal scenario coverage.

[0018] Functional correctness verification; Input test data, execute the combined product, and verify the output to include the required fields.

[0019] Performance indicator evaluation; Use JMeter to simulate high-concurrency scenarios, verify response latency and service throughput, and generate performance test reports and optimization suggestions.

[0020] Compliance audit and inspection; It automatically checks the compliance of combined products, generates a compliance assessment report, and identifies potential compliance risks.

[0021] A system for implementing the method of any one of claims 1-5, comprising: Scene analysis module: Extracts scene requirement features based on NLP technology; Semantic matching engine: Enables part vectorization and similarity calculation; Visual assembly platform: provides a drag-and-drop interface and arrangement tools; Protocol adaptation engine: Enables multi-protocol conversion and data verification; Test and verification framework: Executes automated tests and evaluations; Feedback optimization system: Collects user feedback and optimizes the matching algorithm.

[0022] The present invention provides a method and system for intelligent matching and no-code assembly of data components in a scenario, which has the following beneficial effects: This invention achieves accurate matching and rapid combination of data components with business scenarios through a semantic matching engine, visual orchestration tools, protocol adaptive conversion, and automated testing framework, solving the problems of low scenario adaptation efficiency, complex component splicing, and high technical threshold in traditional data product development. Attached Figure Description

[0023] Figure 1 : Flowchart of a data component scene intelligent matching and no-code assembly method according to the present invention. Detailed Implementation

[0024] This invention proposes a method and system for intelligent matching and no-code assembly of data components in various scenarios. The system includes scenario requirement analysis, multi-dimensional component matching, visual no-code assembly, adaptive protocol conversion, and automated testing and verification. This invention enables accurate matching and rapid combination of data components with business scenarios, solving the problems of low scenario adaptation efficiency, complex component splicing, and high technical threshold in traditional data product development. It significantly improves the circulation efficiency and scenario-based application value of data components.

[0025] The following definitions apply in this invention: Scenario-part matching: Based on the requirements of the target scenario (functionality, data, compliance, performance), identify and recommend the most suitable data parts from the parts library; No-code assembly: Configure the data flow relationship between parts through visual drag and drop, and complete the assembly of parts without writing code; Protocol Adaptive: Automatically identifies and converts interface protocols between different parts to achieve seamless data flow.

[0026] The present invention adopts the following solution: A method for intelligent matching and no-code assembly of data components in a given scenario includes the following steps: Step 1: Scenario Requirements Analysis; Extract the functional, data, compliance, and performance requirements of the scenario using natural language processing technology; Step 2: Multi-dimensional parts matching; Based on semantic indexing and multi-dimensional similarity calculation, the most suitable parts are recommended from the parts library; Step 3: Visualize and assemble without code; Configure data flow relationships between parts through a drag-and-drop interface; Step 4: Protocol adaptive conversion; Automatically identify and convert interface protocols between different parts; Step 5: Automated testing and verification; Generate and execute test cases to verify the functionality and performance of the combined product.

[0027] Furthermore, the input in step 1 is: the scenario requirements described in natural language, such as generating a financial risk control user profile, which should include region, spending power, and credit score.

[0028] Use the BERT model to extract functional requirements, data requirements, compliance requirements, and performance requirements from the scenario; construct a structured requirement feature vector.

[0029] Furthermore, in step 2, based on semantic indexing and multi-dimensional similarity calculation, the method for recommending the most suitable parts from the parts library is as follows: S21. Semantic index construction; The part metadata (functional description, input / output schema, compliance rules, etc.) is encoded into a 768-dimensional vector using BERT, and the part vectors are stored and managed using the Milvus vector database, supporting efficient similarity search.

[0030] S22, Multi-dimensional matching algorithm; A weighted comprehensive scoring model is adopted. Score = w1 * Functional fit + w2 * Data fit + w3 * Compliance fit + w4 * Performance fit; The multi-dimensional matching includes: Functional matching degree calculation is based on the cosine similarity of functional tags; Calculate data matching degree and evaluate input / output schema compatibility; Compliance matching degree calculation, checking the consistency of compliance rules; Performance matching degree calculation, comparing the degree of compliance of performance indicators; Weighted composite score: Score = w1 Functionality + w2 Data + w3 Compliance + w4 Performance.

[0031] S23. Recommendation results are generated; Candidate parts are sorted in descending order of their overall scores. Output the Top-N recommendation list (default N=5). Provide detailed scores for each dimension of the recommendation with an explainable rationale.

[0032] Furthermore, the visual no-code assembly in step 3 includes: Drag-and-drop component layout; Drag and drop recommended parts from the parts library onto the canvas area to automatically generate part icons and basic information previews; Connected data stream orchestration; The data flow between parts is defined by connecting lines, the mapping relationship between input and output fields is automatically identified, and conditional branching and loop control structures are supported. Graphical parameter configuration; The system allows for graphical configuration of input parameter default values, setting output field aliases and format conversion rules, and defining error handling strategies and retry mechanisms.

[0033] Real-time data mapping preview.

[0034] Furthermore, the protocol adaptive conversion in step 4 includes: Multi-protocol identification: REST / gRPC / WASI; Unified intermediate format conversion; Enhanced data format validation; It validates data format integrity based on JSON Schema, automatically fills in default values ​​for missing fields, and automatically converts data types such as string to number.

[0035] Message queues deliver messages asynchronously. Asynchronous data delivery is achieved using Kafka message queues, which support data buffering and flow control, and provide at least-once delivery semantics.

[0036] Furthermore, the automated testing and verification in step 5 includes: Test cases are generated automatically; Automatically generate scenario-based test cases based on component test cases, supporting boundary value analysis and abnormal scenario coverage.

[0037] Functional correctness verification; Input test data, execute the combined product, and verify the output to include the required fields.

[0038] Performance indicator evaluation; Use JMeter to simulate high-concurrency scenarios, verify response latency and service throughput, and generate performance test reports and optimization suggestions.

[0039] Compliance audit and inspection; It automatically checks the compliance of combined products, generates a compliance assessment report, and identifies potential compliance risks.

[0040] A system for implementing the method of any one of claims 1-5, comprising: Scene analysis module: Extracts scene requirement features based on NLP technology; Semantic matching engine: Enables part vectorization and similarity calculation; Visual assembly platform: provides a drag-and-drop interface and arrangement tools; Protocol adaptation engine: Enables multi-protocol conversion and data verification; Test and verification framework: Executes automated tests and evaluations; Feedback optimization system: Collects user feedback and optimizes the matching algorithm.

[0041] Taking the generation of user profiles for financial risk control as an example: Step 1: Scene Analysis Input: "Generate a user profile for financial risk control, including region, spending power, and credit score." Output feature vector: Functional Requirements: ["Geographic Location Extraction", "Spending Capacity Calculation", "Credit Score"] "Data Requirements": { Input: ["user_id", "ip", "transaction_records"], Output: ["province", "monthly_spend", "credit_score"] }, "Constraints": ["Data used within China", "Anonymization level: AES-256", "Latency ≤ 100ms"]} Step 2: Part Matching Retrieve candidate parts from the parts library Calculate multi-dimensional matching degree: Parts extracted by region: Functionality 0.95, Data 0.90, Compliance 1.0, Performance 0.92 → Overall 0.94 Consumer spending power calculation component: Functionality 0.88, Data 0.85, Compliance 0.95, Performance 0.90 → Overall 0.90 Credit scoring components: Functionality 0.92, Data 0.88, Compliance 0.98, Performance 0.85 → Overall 0.91 Recommended order: Location-based extraction → Credit score → Spending capacity calculation Step 3: No-code assembly Drag three recommended parts onto the canvas Data flow defined by connection: Geographic location extraction → Spending capacity calculation → Credit scoring Configuration parameters: Parts extracted by region: IP field mapped to ${user_ip} Component for calculating spending power: The transaction record field alias is set to "Transaction History". Step 4: Protocol Adaptation Geographically extracted parts REST protocol → Consumption capacity calculation parts gRPC protocol The protocol conversion engine automatically performs format conversion. Data flows through the Kafka message queue. Step 5: Automated Testing Generate test cases: Input test user data, verify that the output contains all required fields. Performance testing: With 1000 concurrent requests, the average latency is 85ms and the throughput is 5200 QPS. Compliance check: All data processing was completed within the country and complies with requirements.

[0042] The present invention provides a method and system for intelligent matching and no-code assembly of data components in a scenario, which has the following beneficial effects: This invention achieves accurate matching and rapid combination of data components with business scenarios through a semantic matching engine, visual orchestration tools, protocol adaptive conversion, and automated testing framework, solving the problems of low scenario adaptation efficiency, complex component splicing, and high technical threshold in traditional data product development. The above-described embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for intelligent matching and no-code assembly of data components in a scenario, comprising the following steps: Step 1: Scenario Requirements Analysis; Extract the functional, data, compliance, and performance requirements of the scenario using natural language processing technology; Step 2: Multi-dimensional parts matching; Based on semantic indexing and multi-dimensional similarity calculation, the most suitable parts are recommended from the parts library; Step 3: Visualize and assemble without code; Configure data flow relationships between parts through a drag-and-drop interface; Step 4: Protocol adaptive conversion; Automatically identify and convert interface protocols between different parts; Step 5: Automated testing and verification; Generate and execute test cases to verify the functionality and performance of the combined products.

2. The intelligent matching and code-free assembly method for data components according to claim 1, characterized in that: The multi-dimensional matching in step 2 includes: Functional matching degree calculation is based on the cosine similarity of functional tags; Calculate data matching degree and evaluate input / output schema compatibility; Compliance matching degree calculation, checking the consistency of compliance rules; Performance matching degree calculation, comparing the degree of compliance of performance indicators; Weighted composite score: Score = w1 Functionality + w2 Data + w3 Compliance + w4 Performance.

3. The intelligent matching and code-free assembly method for data components according to claim 1, characterized in that: The visual no-code assembly in step 3 includes: Drag-and-drop component layout; Connected data stream orchestration; Graphical parameter configuration; Real-time data mapping preview.

4. The intelligent matching and code-free assembly method for data components according to claim 1, characterized in that... The protocol adaptive conversion described in step 4 includes: Multi-protocol identification: REST / gRPC / WASI; Unified intermediate format conversion; Enhanced data format validation; Message queues deliver messages asynchronously.

5. The intelligent matching and code-free assembly method for data components according to claim 1, characterized in that: The automated testing and verification in step 5 includes: Test cases are generated automatically; Functional correctness verification; Performance indicator evaluation; Compliance audit and inspection.

6. A system for implementing the method according to any one of claims 1-5, characterized in that... include: Scene analysis module: Extracts scene requirement features based on NLP technology; Semantic matching engine: Enables part vectorization and similarity calculation; Visual assembly platform: provides a drag-and-drop interface and arrangement tools; Protocol adaptation engine: Enables multi-protocol conversion and data verification; Test and verification framework: Executes automated tests and evaluations; Feedback optimization system: Collects user feedback and optimizes the matching algorithm.

Images