Intelligent access API (Application Program Interface) generation method, system and equipment and medium

Abstract

The invention discloses an intelligent access API generation method, system and device and a medium, and the method specifically comprises the steps: generating a dynamic dragging interface based on user role information, and obtaining function access parameters customized by a user through the dynamic dragging interface; according to semantic features and operation sequences in historical user interaction data, user intentions are analyzed, and target data topic logic is generated in combination with function access parameters; converting the subject logic of the target data into an executable query statement by utilizing an SQL splicing tool, and recommending an index optimization scheme to a database engine based on a high-frequency condition and field combination in a historical query mode; and distributing the optimized executable query statements to Serverless workflow nodes adapted to different query engines for execution, and packaging returned data into a standardized API (Application Program Interface) for output. According to the method and the device, full-process automation from parameter customization by a user to standardized API output is realized, and the efficiency and the flexibility of data acquisition are remarkably improved.

Description

Technical Field

[0001] The present invention relates to the field of data analysis technology, and in particular to a method, system, device and medium for generating an intelligent data acquisition API. Background Art

[0002] In modern enterprise operations, data analysis has become a core element driving business decision-making and strategic planning. With the explosive growth of enterprise data volumes and the increasing complexity of business scenarios, the demand for report generation and data acquisition is growing exponentially. Traditional manual data extraction methods, such as directly writing SQL queries or extracting data through fixed reporting tools, are no longer able to meet the requirements of modern enterprises for rapid response, efficient processing, and high flexibility. Specifically, traditional methods face the following key challenges:

[0003] 1. Inefficiency: Manually writing SQL statements is not only time-consuming but also error-prone. Especially when dealing with complex query logic, developers need to have in-depth database knowledge.

[0004] 2. Lack of flexibility: As business needs change, data query logic needs to be frequently adjusted. Traditional methods are unable to quickly adapt to these changes, resulting in delayed data acquisition and affecting the timeliness of business decisions.

[0005] 3. Difficulty integrating multi-source data: Modern enterprises often use multiple data sources and query engines, such as relational databases, NoSQL databases, and big data platforms. Data formats and query syntax vary significantly between different data sources, making cross-source data integration extremely complex.

[0006] 4. Low resource utilization: Traditionally, data query tasks typically require a fixed amount of server resources to be occupied for extended periods of time, resulting in low resource utilization and high costs. This resource waste is particularly prominent in scenarios where query demand fluctuates significantly.

[0007] To address the above issues, some existing solutions have attempted to simplify the data acquisition process through automated tools or platforms. However, these solutions are often limited to specific data sources or query engines and lack versatility and scalability. Summary of the Invention

[0008] The purpose of the present invention is to provide an intelligent data acquisition API generation method, system, device and medium, which realizes the full process automation from user-defined parameters to standardized API output, significantly improves the efficiency and flexibility of data acquisition, lowers the technical threshold, and optimizes resource utilization to solve at least one of the above-mentioned existing technical problems.

[0009] In a first aspect, the present invention provides a method for generating an intelligent data acquisition API, the method specifically comprising:

[0010] Generate a dynamic drag-and-drop interface based on user role information, and obtain user-defined function input and output parameters through the dynamic drag-and-drop interface;

[0011] Based on the semantic features and operation sequences in historical user interaction data, a Transformer-based dual-channel neural network model is used to analyze user intent and generate the target data theme logic based on function input and output parameters.

[0012] Use SQL splicing tools to convert the target data subject logic into executable query statements. At the same time, based on the high-frequency conditions and field combinations in historical query patterns, recommend index optimization solutions to the database engine.

[0013] By monitoring the time series characteristics of historical query traffic in real time, we can predict future traffic peak periods and dynamically adjust the resource allocation strategy of Serverless workflows.

[0014] The optimized executable query statements are distributed to Serverless workflow nodes adapted to different query engines for execution, and the returned data is encapsulated as a standardized API for output.

[0015] In a second aspect, the present invention provides an intelligent data acquisition API generation system, the system specifically comprising:

[0016] The first generation module is used to generate a dynamic drag interface based on the user role information, and obtain the user-defined function input parameters and function output parameters through the dynamic drag interface;

[0017] The second generation module is used to analyze user intent based on the semantic features and operation sequences in historical user interaction data using a Transformer-based dual-channel neural network model, and generate the target data theme logic based on the function input parameters;

[0018] The third generation module is used to convert the target data subject logic into executable query statements using SQL splicing tools. At the same time, based on the high-frequency conditions and field combinations in historical query patterns, it recommends index optimization solutions to the database engine;

[0019] The fourth generation module is used to monitor the time series characteristics of historical query traffic in real time, predict future traffic peak periods, and dynamically adjust the resource allocation strategy of the Serverless workflow;

[0020] The fifth generation module is used to distribute the optimized executable query statements to Serverless workflow nodes adapted to different query engines for execution, and encapsulate the returned data into a standardized API for output.

[0021] In a third aspect, the present invention provides a computer device comprising: a memory and a processor and a computer program stored in the memory, wherein when the computer program is executed on the processor, the intelligent data acquisition API generation method as described in any one of the above methods is implemented.

[0022] In a fourth aspect, the present invention provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the method for generating an intelligent data acquisition API as described in any one of the above methods is implemented.

[0023] Compared with the prior art, the present invention has at least one of the following technical effects:

[0024] 1. This invention realizes the automation of the entire process from user-defined parameters to standardized API output, significantly improving the efficiency and flexibility of data acquisition, lowering the technical threshold, and optimizing resource utilization.

[0025] 2. The present invention generates a dynamic drag interface based on user role information, so that users can intuitively customize the input and output parameters of the function by dragging and dropping, which greatly simplifies the parameter configuration process and improves the efficiency of data acquisition.

[0026] 3. The present invention utilizes a dual-channel neural network model based on Transformer, which can intelligently analyze users' historical interaction data, accurately understand user intentions, and automatically generate target data theme logic that meets business needs in combination with user-defined function input and output parameters, further improving the intelligent level of data acquisition.

[0027] 4. This invention uses serverless workflows to seamlessly connect multiple query engines and data sources, enabling unified query and management of cross-source data. Users no longer need to worry about differences in underlying data sources; they can simply access the required data through a unified API.

[0028] 5. By real-time monitoring of the temporal characteristics of historical query traffic, the present invention can accurately predict future traffic peak periods and dynamically adjust the resource allocation strategy of the Serverless workflow to ensure that sufficient computing resources can be provided during peak query demand periods, while releasing idle resources during low periods, thereby significantly improving resource utilization and reducing operating costs.

[0029] 6. The present invention dynamically generates a subset of operable fields based on the role-field permission matrix, ensuring the security and compliance of user operations. At the same time, it checks the legality of parameter combinations through a lightweight syntax tree, further improving the accuracy and reliability of user-defined parameters.

[0030] 7. The present invention adopts a Transformer-based dual-channel neural network model to analyze user intentions, achieving a deep integration of user operations and natural language semantics. The target data theme logic generated by combining function input and output parameters is closer to the actual needs of users and improves the accuracy of data acquisition.

[0031] 8. The present invention utilizes SQL splicing tools and index optimization solutions, which not only efficiently converts the target data subject logic into executable query statements, but also significantly improves query performance and reduces database load by recommending index optimization solutions for high-frequency conditions and field combinations.

[0032] 9. This invention predicts future traffic peak periods by real-time monitoring of historical query traffic time series characteristics and dynamically adjusts the resource allocation strategy of the Serverless workflow, thereby achieving efficient resource utilization and effective cost control, while ensuring the stability and reliability of the query service.

[0033] 10. The target instance number calculation method of the present invention comprehensively considers multiple factors such as the total traffic forecast value, time window, safety factor, maximum concurrent query number of a single instance and basic instance number, ensuring the rationality and accuracy of resource allocation and further improving the overall performance of the system.

[0034] 11. The present invention realizes the intelligent distribution and execution of query statements among different query engines through the engine adapter set and routing weight function, and dynamically adjusts the adapter priority in combination with real-time monitoring data, thereby ensuring the efficiency and stability of the query service, simplifying the data acquisition process, and improving the user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0036] Figure 1 This is a flow chart of a method for generating an intelligent data acquisition API provided by the first embodiment of the present invention;

[0037] Figure 2 This is a flow chart of a method for generating an intelligent data acquisition API provided by the second embodiment of the present invention;

[0038] Figure 3 This is a schematic diagram of the structure of an intelligent data acquisition API generation system provided by one embodiment of the present invention;

[0039] Figure 4It is a structural diagram of a computer device provided by one embodiment of the present invention. DETAILED DESCRIPTION

[0040] In the following description, specific details such as specific system structures and techniques are provided for purposes of illustration rather than limitation to facilitate a thorough understanding of the embodiments of the present application. However, it will be apparent to those skilled in the art that the present application may be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to avoid obscuring the description of the present application with unnecessary detail.

[0041] It should be understood that when used in the present specification and the appended claims, the term "comprising" indicates the presence of described features, integers, steps, operations, elements and / or components, but does not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or collections thereof.

[0042] It will also be understood that the term "and / or" used in this specification and the appended claims refers to and includes any and all possible combinations of one or more of the associated listed items.

[0043] As used in this specification and the appended claims, the term "if" can be interpreted as "when" or "upon" or "in response to determining" or "in response to detecting," depending on the context. Similarly, the phrase "if it is determined" or "if [described condition or event] is detected" can be interpreted as meaning "upon determination" or "in response to determining" or "upon detection of [described condition or event]" or "in response to detecting [described condition or event]," depending on the context.

[0044] In addition, in the description of the present application specification and the appended claims, the terms "first", "second", "third", etc. are only used to distinguish the descriptions and cannot be understood as indicating or implying relative importance.

[0045] References to "one embodiment" or "some embodiments" in this specification mean that a particular feature, structure, or characteristic described in conjunction with that embodiment is included in one or more embodiments of the present application. Thus, phrases such as "in one embodiment," "in some embodiments," "in other embodiments," and "in other embodiments" appearing in various places in this specification do not necessarily refer to the same embodiment, but rather mean "one or more but not all embodiments," unless otherwise specifically emphasized. The terms "including," "comprising," "having," and variations thereof all mean "including but not limited to," unless otherwise specifically emphasized.

[0046] In the embodiments of the present application, the execution subject of the process includes a terminal device, which includes but is not limited to: a server, a computer, a smart phone, a tablet computer, and other devices capable of executing the method disclosed in the present application. Figure 1 The following is a flow chart of the method for generating an intelligent data acquisition API according to the first embodiment of the present invention, which is described in detail as follows:

[0047] S101 , generating a dynamic drag interface based on user role information, and obtaining user-defined function input and output parameters through the dynamic drag interface.

[0048] In this embodiment, in modern enterprise operations, with the explosive growth of data volume and the complexity of business scenarios, traditional manual data acquisition methods have become difficult to meet the requirements of rapid response, efficient processing, and high flexibility. In particular, for the diverse needs of different user roles (such as data analysts, business decision makers, etc.), traditional methods cannot provide a personalized data acquisition experience. Therefore, it is particularly important to develop a method that can dynamically generate a drag-and-drop interface based on user roles and efficiently obtain the input and output parameters of user-defined functions.

[0049] Specifically, users register and log in through the company's internal unified identity authentication system. The system records basic user information, including but not limited to name, department, and position. Based on user position information and business needs, system administrators predefine various user roles, such as data analyst, business decision maker, and general employee, and assign corresponding permissions and data access scopes to each role. User role information is stored in the database, including fields such as role ID, role name, and permission list, to facilitate the subsequent generation of dynamic drag-and-drop interfaces.

[0050] Choose a front-end framework that supports dynamic component rendering, such as React and Vue, to build the user interface. Design a variety of interface templates based on the needs of different user roles. For example, design a template for data analysts that includes functions such as data filtering and aggregation function selection; design a template for business decision makers that includes data visualization components and key indicator displays. When a user logs in to the system, the front-end framework retrieves the corresponding interface template from the back-end based on the user's role information and dynamically renders a drag-and-drop interface. Components on the interface (such as input boxes, drop-down boxes, buttons, etc.) can be dragged and combined according to user needs.

[0051] On the front-end interface, the user's drag and drop operations are captured in real time through the JavaScript event monitoring mechanism. When the user drags a component to a specified location, the corresponding event processing function is triggered. Each component contains a set of configuration information, such as the component type (input box, drop-down box, etc.), default value, optional value range, etc. When the user completes the drag and drop operation, the front-end framework collects this configuration information and associates it with the user-defined function input and output parameters. Before the user submits the drag and drop result, the front-end framework verifies the user-defined function input and output parameters to ensure the legitimacy and integrity of the parameters. After verification, the parameters are submitted to the back-end server in JSON format.

[0052] After receiving the parameters submitted by the frontend, the backend server parses and stores them. It then associates the user-defined function input and output parameters with the corresponding user role information to facilitate subsequent data queries and API generation. Based on the user-defined function input and output parameters, the backend server invokes the corresponding data query logic to retrieve the required data from the database. Simultaneously, it uses SQL concatenation tools to convert the query logic into executable query statements and encapsulates them into standardized API output. The backend server then feeds the query results and API information back to the frontend for user review and use.

[0053] In this embodiment, users can quickly build data query logic through a drag-and-drop interface, eliminating the need to manually write SQL query statements, significantly improving data acquisition efficiency. Different user roles can customize the data acquisition interface and function input and output parameters to meet diverse business needs. The intuitiveness and ease of use of the drag-and-drop interface lower the user barrier to entry, making it easy for even non-technical users to get started. By collecting and classifying user role information, the system can implement fine-grained permission control for different user roles, ensuring data security.

[0054] S102: Based on the semantic features and operation sequences in historical user interaction data, a Transformer-based dual-channel neural network model is used to analyze user intentions, and the target data theme logic is generated in combination with function input and output parameters.

[0055] In this embodiment, historical user interaction data is collected from the enterprise data analysis platform, including but not limited to user query records, operation logs, click behaviors, etc. These data record the interaction process between the user and the system and are an important basis for analyzing user intentions. The collected historical user interaction data is cleaned to remove noise data and irrelevant information. At the same time, the data is labeled, such as the user query intention type (such as data screening, data aggregation, data comparison, etc.), query conditions, operation sequence, etc. The labeled data will be used as input for model training. Key information is extracted from user-defined function input and output parameters, such as parameter name, parameter type, parameter value range, etc. This information will be used as model input together with user interaction data to generate target data theme logic.

[0056] A two-channel neural network model based on Transformer is designed. This model consists of two channels: one for processing the semantic features of user interaction data, and the other for processing the structured information of function input and output parameters. The outputs of the two channels are fused through an attention mechanism to capture the relationship between user intent and function parameters. In the semantic feature channel, a Transformer model is used to encode the textual information of user interaction data and extract its semantic features. These features reflect the user's query intent and context. In the function input and output parameter channel, a model such as a multi-layer perceptron (MLP) or convolutional neural network (CNN) is used to encode the structured information of function input and output parameters and extract its key features. These features reflect the user-defined query conditions and parameter constraints.

[0057] The extracted semantic features and structured information are fed into a two-channel neural network model, which calculates the distribution of user intent through forward propagation. The model outputs a probability distribution representing the likelihood that a user query belongs to different intent types.

[0058] Based on the results of user intent analysis and the function input and output parameters, the target data theme logic is generated. This logic describes key information such as the data range, query conditions, and aggregation method that the user wants to obtain. For example, if the user intent is to filter data, the target data theme logic will include the filtering conditions; if the user intent is to aggregate data, the target data theme logic will include the aggregation function and grouping fields.

[0059] The generated target data topic logic is verified to ensure compliance with business rules and query syntax. At the same time, the logic is optimized based on high-frequency conditions and field combinations in historical query patterns to improve query efficiency and accuracy.

[0060] In this embodiment, through intelligent analysis of user intent, the generated target data theme logic better meets user needs, improving the accuracy of data acquisition. Users can adjust query conditions by customizing function input and output parameters to meet diverse business needs. The model can quickly adapt to changing business needs, improving the flexibility of data acquisition. The Transformer-based dual-channel neural network model can automatically learn the semantic features of user interaction data and the structured information of function input and output parameters, improving the intelligence level of the system.

[0061] S103: Use the SQL splicing tool to convert the target data subject logic into an executable query statement, and recommend index optimization solutions to the database engine based on the high-frequency conditions and field combinations in the historical query pattern.

[0062] In this embodiment, in modern database applications, with the continuous growth of data volume and the increase in query complexity, how to efficiently execute query statements has become an important issue. The traditional way of writing SQL query statements often relies on the experience of developers and is difficult to optimize for specific query scenarios. At the same time, the creation and management of database indexes are often lacking in systematicity, resulting in low query efficiency. Therefore, the present invention automatically generates optimized query statements by intelligently analyzing the target data subject logic and historical query patterns, and recommends appropriate index optimization solutions to the database engine to improve query efficiency.

[0063] Specifically, the generated target data subject logic is parsed to extract key information such as query conditions, aggregation methods, and grouping fields. This information is the basis for generating SQL query statements. Utilize SQL splicing tools to automatically generate corresponding SQL query statements based on the parsed target data subject logic. The splicing tool should support multiple database dialects to ensure that the generated query statements can be executed on different database engines. Validate the generated SQL query statements to ensure that their syntax is correct and complies with business rules. The validation process can include syntax checking, logic validation, etc.

[0064] Collect historical query data from database logs or query analysis tools, including query statements, execution time, and returned results. This data reflects past user query behavior and actual database usage. Perform statistical analysis on the collected historical query data to extract frequently occurring query conditions and field combinations. High-frequency conditions refer to query conditions that frequently appear in multiple queries, such as specific WHERE clauses; high-frequency field combinations refer to sets of fields that frequently appear together in multiple queries. Categorize historical query patterns based on high-frequency condition and field combinations. For example, query patterns can be categorized into types such as "simple filtering queries" and "complex aggregation queries" to facilitate subsequent targeted index optimization.

[0065] Generate index creation recommendations based on the extracted high-frequency conditions and field combinations, combined with the index types and features of the database engine. For example, for frequently occurring single-field query conditions, it may be recommended to create a single-column index; for frequently occurring multi-field combination query conditions, it may be recommended to create a composite index. Perform a performance evaluation on the generated index creation recommendations to simulate the query execution efficiency after creating the index in the database. The evaluation process can be performed using the database's own performance analysis tools or third-party performance testing tools. Based on the performance evaluation results, recommend the optimal index optimization plan to the database engine. The recommended plan should include key information such as index type, index field, and index location to facilitate implementation by database administrators or automated tools.

[0066] Submit the recommended index optimization plan to the database administrator or automated tools for implementation. The implementation process should adhere to database best practices to ensure that index creation and management meet database security and stability requirements. After the index optimization plan is implemented, continuously monitor database query performance. Monitoring metrics can include query execution time and resource utilization. Based on the monitoring results, provide timely feedback to the database administrator on the optimization results for further adjustments and optimizations. Based on the feedback and actual usage, iterate and optimize the index optimization plan. For example, if a particular index is found to be ineffective in actual use, adjust the index type or field combination to improve query efficiency.

[0067] In this embodiment, the efficiency of database query execution is significantly improved by automatically generating optimized query statements and recommending appropriate index optimization solutions. Developers can obtain efficient query results without having to manually write and optimize query statements or possess in-depth database knowledge, thus reducing development costs. By intelligently analyzing historical query patterns and recommending index optimization solutions, database index management becomes more systematic and automated, enhancing system maintainability.

[0068] S104, by real-time monitoring of historical query traffic time series characteristics, predicting future traffic peak periods, and dynamically adjusting the resource allocation strategy of the Serverless workflow.

[0069] In this embodiment, the Serverless architecture has been widely used in various application scenarios due to its advantages such as elastic scalability and pay-as-you-go. However, when processing high-concurrency queries, the Serverless workflow still faces problems such as unreasonable resource allocation and increased response delays. Traditional resource allocation strategies are often based on fixed rules or empirical settings, which are difficult to adapt to the dynamic changes in actual query traffic. Therefore, the present invention monitors historical query traffic in real time, uses time series analysis technology to predict future traffic peak periods, and dynamically adjusts the resource allocation strategy of the Serverless workflow accordingly to improve resource utilization and query response speed.

[0070] Specifically, a traffic monitoring module is deployed at the front-end or middleware layer of the Serverless workflow to collect real-time query request data, including request time, request type, request parameters, and response time. The collected raw data is cleaned to remove noise, outliers, and duplicates to ensure data accuracy and consistency. The cleaned data is aggregated based on business needs and time granularity (such as minutes or hours) to generate a time-series query traffic dataset.

[0071] Using time series analysis techniques (such as ARIMA models and LSTM neural networks), we extract key features from aggregated time series data, including trends, seasonality, and cyclicality. We select and reduce the extracted time series features based on their importance, relevance, and redundancy to reduce computational complexity and improve forecast accuracy. We train time series forecasting models using historical time series data and employ methods such as cross-validation to validate and optimize the models and ensure good forecasting performance.

[0072] Apply the trained time series prediction model to the current time series data to predict query traffic trends over the next period of time. Based on the prediction results, identify potential peak traffic periods, including key information such as peak start time, duration, and peak traffic volume. Evaluate the uncertainty of the prediction results, quantify the prediction error and confidence interval, and provide a reference for the subsequent development of resource allocation strategies.

[0073] Based on the predicted peak traffic periods and uncertainty assessment results, formulate a resource allocation strategy for the serverless workflow. The strategy should include key parameters such as the amount of resource addition, when to add resources, and when to reduce resources. Deploy the formulated resource allocation strategy to the serverless workflow management platform or automation tool to implement dynamic resource adjustments. The adjustment process should follow the principle of smooth transition to avoid impacting ongoing query requests. After implementing the resource allocation strategy, monitor the serverless workflow's resource usage and query response speed in real time. Based on the monitoring results, adjust the resource allocation strategy promptly to ensure optimal resource utilization and query response speed.

[0074] Regularly evaluate the effectiveness of resource allocation strategies, including metrics such as resource utilization, query response speed, and cost-effectiveness. Based on the evaluation results, diagnose problems and deficiencies in the resource allocation strategy and propose improvement measures and optimization suggestions. These improvement measures and optimization suggestions are incorporated into the resource allocation strategy for iteration and upgrade. Through continuous optimization and iteration, the adaptability and effectiveness of the resource allocation strategy are continuously improved.

[0075] In this embodiment, by dynamically adjusting the resource allocation strategy of the serverless workflow, resources can be elastically scaled based on actual query traffic, improving resource utilization. Increasing resource allocation in advance during peak traffic periods effectively avoids query request queuing and blocking, reducing query response latency. Real-time monitoring and feedback mechanisms enable timely identification and resolution of issues and deficiencies in the resource allocation strategy, enhancing system stability and reliability.

[0076] S105 distributes the optimized executable query statements to Serverless workflow nodes adapted to different query engines for execution, and encapsulates the returned data into a standardized API for output.

[0077] In this embodiment, the Serverless architecture has been widely used in the field of data processing and analysis due to its advantages such as elastic scalability and pay-as-you-go. However, in actual applications, enterprises often need to support multiple database query engines (such as MySQL, PostgreSQL, MongoDB, Elasticsearch, etc.) at the same time to meet the needs of different business scenarios. Different query engines have differences in syntax, execution plans, data formats, etc., which makes the writing and execution of query statements complicated. At the same time, how to output the query results to the front-end application in a standardized manner is also a problem that needs to be solved urgently. Therefore, the present invention proposes to realize the intelligent distribution of query statements, cross-engine execution and standardized output of results by building a universal query execution framework, thereby improving query efficiency and system scalability.

[0078] Specifically, it receives the original query request submitted by the user and optimizes the query statement using SQL optimization techniques (such as index optimization and query rewriting) to generate an efficient and executable query statement. It then parses the optimized query statement to extract key information such as the query type (such as SELECT, INSERT, UPDATE), target data source, and query conditions. This information is used for subsequent query engine adaptation and execution plan generation.

[0079] Based on the parsed target data source information, determine the appropriate query engine type (e.g., relational database, NoSQL database, search engine, etc.). Build a query engine adapter library that includes connection configuration, syntax conversion, and execution plan generation for various query engines. Generate a specific execution plan for each compatible query engine. The execution plan should include query statement conversion, index usage recommendations, and parallel execution strategies.

[0080] Define a serverless workflow consisting of multiple execution nodes, each corresponding to a query engine adapter instance. Nodes communicate with each other through message queues or event-driven mechanisms, enabling asynchronous query distribution and result collection. Based on the execution plan, optimized queries are distributed to the corresponding serverless workflow nodes. After receiving a query, each node invokes the adapted query engine to execute the query and returns the results to the workflow controller. Fault tolerance and retry mechanisms are implemented for each workflow node to ensure automatic retry or failover to a backup node if a query fails.

[0081] Receive query results returned by each workflow node and standardize the format of the result data. For example, convert data returned by different query engines (such as JSON, XML, CSV, etc.) into a unified internal data structure. Based on the needs of the front-end application, encapsulate the unified data into a standardized API output format. For example, this can be encapsulated as a RESTful API interface that supports JSON input and output. The standardized API should include necessary metadata (such as request ID, execution time, error code, etc.) to facilitate error handling and performance monitoring by the front-end application.

[0082] Deploy an API gateway as a unified entry point for standardized APIs. The API gateway is responsible for routing requests, verifying identities, and performing rate limiting. Implement access control and authentication mechanisms within the API gateway to ensure that only authorized users or applications can access the standardized APIs. Authentication protocols such as OAuth2.0 and JWT can be used to ensure secure API access. Provide detailed API documentation and usage examples to help front-end developers quickly integrate and use the standardized APIs.

[0083] In this embodiment, intelligently distributing query statements to the appropriate query engine for execution eliminates the need for manual writing and debugging of queries for different engines, improving query efficiency. The dynamic expansion and contraction capabilities of Serverless workflow nodes enable the system to easily adapt to changes in query traffic, improving system scalability. The standardized API output format reduces integration complexity and development costs for front-end developers, improving development efficiency.

[0084] As a brief description of the above steps S101 to S105, refer to Figure 2 Users customize function input and output parameters through a drag-and-drop interface. These parameters are then passed to the SQL assembler, which constructs topic logic and passes it to the serverless workflow. Based on the topic logic, the serverless workflow sends data requests to the query engine data source, which then returns the data to the custom data retrieval API. For example, an e-commerce company needs to generate a report on user purchase behavior. Users can use the drag-and-drop interface to select function input and output parameters such as user ID, purchase time, and purchased items. The SQL assembler then constructs query logic based on these function inputs and outputs, such as "SELECT user_id, purchase_time, purchase_product FROM purchase_table WHERE purchase_time > '2021-01-01'." The serverless workflow then sends this query logic to the query engine data source, such as MySQL. MySQL returns the corresponding data, which the custom data retrieval API processes to generate a report on user purchase behavior.

[0085] In some embodiments, in the above step S101, generating a dynamic drag interface based on the user role information and obtaining user-defined function input parameters and function output parameters through the dynamic drag interface specifically includes:

[0086] Re-obtain the role tag set from the enterprise identity management system, obtain the table field set from the database metadata, and build a role-field permission matrix based on the role tag set and the table field set;

[0087] Determine the user's current role ID and generate a subset of operable fields based on the role-field permission matrix and the current role ID.

[0088] When the user drags a field to the input / output parameter area, the metadata constraint relationship between the bound field and the function parameter is determined based on the operable field subset, parameter type constraints are generated, and the validity of the parameter combination is checked through a lightweight syntax tree.

[0089] Obtain the historical operation records corresponding to the current role ID and the current session semantic features, calculate the rendering priority of the interface components through a weighted formula, and dynamically generate the field arrangement layout within the drag area.

[0090] In this embodiment, in modern enterprise applications, data analysis and processing functions often require flexible customization capabilities for users with different roles. For example, different roles such as data analysts, business personnel, and administrators have different requirements for data access and operation permissions. Traditional interface design methods usually adopt fixed layouts and parameter configuration methods, which are difficult to meet the diverse customization needs of users and also increase development and maintenance costs. Therefore, the present invention realizes dynamic adaptation of user roles and interface interactions by constructing a role-field permission matrix, generating a subset of operable fields, checking the legality of parameter combinations, and dynamically generating a drag area layout, thereby improving the flexibility and convenience of user customization.

[0091] Specifically, obtain the role tag set for all users from the enterprise identity management system (such as LDAP, OAuth, etc.). The role tag set includes the unique identifier and role name corresponding to each role. Obtain the field set of all tables from the database metadata. The field set includes the table name, field name, data type, description information, etc. of each field. Based on the role tag set and the table field set, construct a role-field permission matrix. The rows of the matrix represent roles, the columns represent fields, and the matrix elements represent the access rights of the role to the field (such as read-only, read-write, no permission, etc.).

[0092] When a user logs into the system, the user's current role ID is determined. The role ID can be obtained through user login information, session information, etc. Based on the role-field permission matrix and the current role ID, the set of fields to which the role has access permissions is filtered to generate a subset of operable fields.

[0093] A drag area is provided on the user interface, and users can drag fields from the subset of operable fields to the input / output parameter area. When the user drags a field, the system determines the binding relationship between the field and the function parameter based on the field's metadata (such as data type, description information, etc.) and generates parameter type constraints. A lightweight syntax tree (such as a simplified version of the abstract syntax tree AST) is used to check the legality of the parameter combination dragged by the user. The syntax tree is used to represent the structure and dependency relationship of the function parameters. By traversing the syntax tree, it is possible to check whether the parameter combination meets the requirements of the function definition. If the parameter combination is illegal, the system will display an error message to the user and prevent the user from submitting an illegal parameter combination.

[0094] Obtain the historical operation records corresponding to the current role ID. Historical operation records include information about past user operations on fields, such as dragging, sorting, and deleting. Analyze the semantic features of the current session, such as the query statement entered by the user and the selected function type. Session semantic features can be extracted using natural language processing (NLP) technology. Design a weighted formula that comprehensively considers historical operation records and session semantic features to calculate the rendering priority of each field within the drag area. Based on the rendering priority, dynamically generate the field layout within the drag area. Fields with higher priority will be displayed in a more prominent position, making it easier for users to find and drag them quickly.

[0095] In this embodiment, users can dynamically drag fields to the input / output parameter area based on their roles and needs to customize the configuration of function parameters. By dynamically generating the drag area layout and parameter type constraints, the user's learning cost and operational difficulty are reduced, and the convenience of interface interaction is improved. A lightweight syntax tree is used to check the legality of parameter combinations to ensure that the parameter combinations submitted by users meet the requirements of the function definition, avoiding system anomalies caused by parameter errors. By constructing a role-field permission matrix and dynamically generating the drag area layout, dynamic adaptation of user roles and interface interactions is achieved, reducing the development and maintenance costs of the system.

[0096] In some embodiments, in step S102, analyzing user intent using a Transformer-based dual-channel neural network model based on semantic features and operation sequences in historical user interaction data, and generating target data topic logic in combination with function input and output parameters, specifically includes:

[0097] Obtain historical user interaction data, extract user operation sequences and conversation text sets from the historical user interaction data, and generate semantic feature vectors through a pre-trained language model;

[0098] Input the user operation sequence into the first Transformer encoder channel to generate the operation sequence hidden state;

[0099] Input the semantic feature vector into the second Transformer encoder channel to generate the semantic hidden state;

[0100] The feature fusion of the operation sequence hidden state and the semantic hidden state is performed through a learnable weight matrix to obtain a joint representation;

[0101] Based on the attention mechanism, the matching degree between the joint representation and the preset logic template library is calculated to determine the optimal template. The placeholders are replaced in combination with the current function input and output parameters to generate the target data theme logic.

[0102] In this embodiment, in data analysis and business processing scenarios, users usually need to input operation instructions or query statements through an interactive interface to obtain the required data or perform specific tasks. However, due to the diversity of user expression habits and the complexity of business logic, it is often difficult for the system to accurately understand the user's true intentions, resulting in deviations between the generated query results or processing logic and user needs. Traditional intent recognition methods are usually based on rule matching or simple machine learning models, which are difficult to handle complex user expressions and dynamically changing business needs. Therefore, the present invention extracts user operation sequences and conversation text sets, generates semantic feature vectors using a pre-trained language model, and uses a Transformer-based dual-channel neural network model to encode and fuse the operation sequences and semantic features, and finally generates target data subject logic that conforms to user intentions and business logic.

[0103] Specifically, historical user interaction data is obtained from sources such as system logs and user operation records. Historical user interaction data includes user input operation instructions, query statements, click behaviors, etc. The historical user interaction data is preprocessed to extract user operation sequences and conversation text collections. The operation sequence represents a series of operations performed by the user in chronological order, and the conversation text collection represents the conversation record between the user and the system. The conversation text collection is encoded using a pre-trained language model (such as BERT, GPT, etc.) to generate a semantic feature vector. The semantic feature vector represents the semantic information and contextual relationships of the conversation text.

[0104] The user action sequence is input into the first Transformer encoder channel. The first Transformer encoder channel uses a multi-layer self-attention mechanism and a feedforward neural network to encode the action sequence and generate an action sequence hidden state. The action sequence hidden state represents the dependencies and semantic information between each action in the action sequence.

[0105] The semantic feature vector is input into the second Transformer encoder channel. This second Transformer encoder channel also uses a multi-layer self-attention mechanism and a feedforward neural network to encode the semantic feature vector and generate a semantic hidden state. The semantic hidden state represents the deep semantic information and contextual relationships of the conversation text.

[0106] A learnable weight matrix is designed to perform feature fusion on the hidden state of the operation sequence and the semantic hidden state. The goal of feature fusion is to organically combine the information of the operation sequence and semantic features to generate a more comprehensive representation of user intent. This feature fusion operation yields a joint representation. This joint representation represents the comprehensive information between the user operation sequence and semantic features, and serves as the basis for subsequent intent analysis and logic generation.

[0107] A pre-built logic template library contains a variety of common business logic templates. A logic template represents the data query or processing logic for a specific business scenario and typically contains placeholders for replacing specific parameter values. An attention mechanism is used to calculate the match between the joint representation and the pre-set logic template library. The attention mechanism measures the similarity between the joint representation and each logic template to determine the optimal template. Based on the current function input and output parameters, the placeholders in the optimal template are replaced to generate the target data topic logic. The target data topic logic represents data query or processing instructions that align with user intent and business logic and can be used for subsequent data analysis and business processing.

[0108] In this embodiment, a Transformer-based dual-channel neural network model is used to encode and fuse user operation sequences and semantic features, which can more accurately capture the user's true intentions and improve the accuracy of intent recognition. By building a logical template library and using placeholder replacement, target data subject logic that meets different business scenarios and user needs can be generated, enhancing the flexibility of logic generation. Since the target data subject logic is more consistent with user intentions and business logic, it can provide more accurate and relevant query results or processing logic, thereby improving the user experience. By adopting a pre-trained language model and a Transformer-based neural network model, it is possible to reduce dependence on specific business scenarios, reduce system development costs and maintenance difficulties.

[0109] In some embodiments, in step S103, the SQL splicing tool is used to convert the target data subject logic into an executable query statement, and based on the high-frequency conditions and field combinations in the historical query pattern, an index optimization solution is recommended to the database engine, specifically including:

[0110] Parse the target data subject logic into a directed acyclic graph, and use a graph traversal algorithm to extract the SELECT clause field set, WHERE clause condition set, and JOIN relationship set from the directed acyclic graph;

[0111] Based on the frequency of occurrence of fields in the WHERE clause condition set, a condition-field frequency matrix is constructed and the index recommendation score is calculated based on the field co-occurrence probability.

[0112] Generate a joint index creation instruction for the field combination whose index recommendation score is greater than or equal to the preset score threshold, and submit it to the automatic execution queue of the database engine;

[0113] Based on the joint index creation instruction, the SELECT clause field set, WHERE clause condition set and JOIN relationship set are used as input, and a standardized SQL query statement is generated through syntax tree reconstruction. Parameterized placeholders are injected to obtain an index optimization solution.

[0114] In this embodiment, in the database query scenario, with the growth of data volume and the improvement of query complexity, query performance issues are becoming increasingly prominent. In traditional methods, database administrators usually need to manually analyze query logs, identify high-frequency query patterns, and design indexes based on experience to optimize query performance. However, this method is not only time-consuming and error-prone, but also difficult to adapt to dynamically changing query requirements. Therefore, the present invention parses the target data subject logic, extracts query key elements, constructs a condition-field frequency matrix, calculates index recommendation scores, and generates joint index creation instructions, and finally converts query key elements into standardized SQL query statements to achieve automatic optimization of query performance.

[0115] Specifically, the target data subject logic is parsed into a directed acyclic graph (DAG). The target data subject logic usually exists in some intermediate representation form, such as an abstract syntax tree (AST) or a custom logical representation. The parsing process involves converting the logical representation into a DAG, where nodes represent query operations or conditions, and edges represent dependencies between operations or conditions. A graph traversal algorithm (such as depth-first search and breadth-first search) is used to extract the SELECT clause field set, the WHERE clause condition set, and the JOIN relationship set from the DAG. These sets are the basis for generating executable SQL query statements.

[0116] Based on the frequency of occurrence of fields in the WHERE clause condition set, a condition-field frequency matrix is constructed. The rows of the matrix represent conditions, the columns represent fields, and the matrix elements represent the number of times a field appears in the conditions. Combined with historical query logs, the co-occurrence probability between fields is calculated. The co-occurrence probability indicates how often two fields appear together in the same query. Based on the condition-field frequency matrix and the field co-occurrence probability, an index recommendation score is calculated for each field combination. Score calculation methods can use weighted summation, machine learning models, and other methods, comprehensively considering factors such as field frequency, co-occurrence probability, and query performance impact.

[0117] Generate joint index creation instructions for field combinations whose index recommendation scores are greater than or equal to the preset score threshold. These instructions contain information such as the index name, index field list, and index type. Submit the generated joint index creation instructions to the database engine's automatic execution queue. The database engine automatically executes these instructions during idle time to create the corresponding joint index.

[0118] Based on the joint index creation instruction (although the index may not be created immediately when the statement is generated, it can be used as an optimization reference), the SELECT clause field set, the WHERE clause condition set and the JOIN relationship set are used as input, and a standardized SQL query statement is generated through syntax tree reconstruction. The syntax tree reconstruction process includes mapping key elements to SQL syntax structures and generating query statements that comply with SQL standards. Parameterized placeholders are injected into the generated SQL query statement. Parameterized placeholders are used to dynamically bind query parameters when executing queries, improving the security and performance of queries. The generated standardized SQL query statement is combined with the joint index creation instruction (or the created index information) to form an index optimization plan. The index optimization plan includes optimized query statements and recommended index information, which can be used for subsequent query performance analysis and optimization.

[0119] In this example, by automatically parsing the target data subject logic and generating optimized SQL query statements, combined with recommended index optimization solutions, database query performance is significantly improved. Automated index recommendation and SQL query statement generation reduce the need for manual intervention and lower database maintenance costs. The index recommendation mechanism based on historical query patterns can adapt to dynamically changing query requirements, improving system flexibility and scalability.

[0120] In some embodiments, in step S104 above, the process of dynamically adjusting the resource allocation strategy of the Serverless workflow by real-time monitoring of historical query traffic time series characteristics and predicting future traffic peak periods may include:

[0121] The time series data of historical query traffic is collected in real time and separated into K modal components and residual terms using the variational mode decomposition algorithm;

[0122] Each modal component is input into the temporal convolutional network, and the residual term is input into the long short-term memory network. The component value and residual value in the future time window are predicted respectively to obtain the prediction result;

[0123] Perform modal reorganization on the prediction results to obtain the total traffic prediction value, and calculate the target number of instances based on the preset safety factor and instance capacity;

[0124] Send the target number of instances to the Serverless platform control interface to trigger scaling operations on workflow nodes.

[0125] The safety factor is dynamically updated based on real-time monitoring of query latency and error rate data.

[0126] In this embodiment, the Serverless architecture has attracted widespread attention due to its elastic scalability and pay-as-you-go features. However, in a Serverless environment, the resource allocation strategy of the workflow is usually based on static configuration or simple rules, which is difficult to adapt to dynamically changing query traffic. During peak traffic periods, if resources are insufficiently allocated, it may lead to increased query latency and increased error rates; while during low traffic periods, if resources are excessively allocated, it will cause resource waste. Therefore, the present invention collects time series data of historical query traffic in real time, uses the variational mode decomposition algorithm (VMD) to separate the time series data into multiple modal components and residual terms, and then uses the temporal convolutional network (TCN) and the long short-term memory network (LSTM) for prediction, and finally obtains the total traffic prediction value through modal recombination, and dynamically adjusts the resource allocation strategy of the Serverless workflow accordingly.

[0127] Specifically, a data collection module is deployed in the Serverless platform to collect real-time time series data of historical query traffic. This time series data includes key metrics such as query request timestamps, number of requests, and response time. The collected time series data is preprocessed through cleaning and normalization to remove noise and outliers, improving data quality. The variational mode decomposition algorithm is used to separate the preprocessed time series data into K modal components and a residual term. The VMD algorithm adaptively decomposes time series data into multiple modal components with different frequency characteristics and a residual term representing the overall trend. Each modal component is fed into a temporal convolutional network for prediction. TCN networks are capable of capturing long-range dependencies and are suitable for processing local patterns in time series data. By training the TCN network, the values of each modal component in future time windows can be predicted. The residual term is fed into a long short-term memory network for prediction. An LSTM network is capable of processing long-range dependencies in sequence data and is suitable for capturing overall trends in time series data. By training the LSTM network, the residual value in future time windows can be predicted. The prediction results of the TCN network and the LSTM network are integrated to obtain the total traffic prediction value in the future time window.

[0128] The target number of instances is calculated based on the total traffic forecast, combining a preset safety factor and instance capacity. The safety factor accounts for uncertainties such as forecast error and traffic bursts. Instance capacity indicates the number of query requests each instance can handle. The calculated target number of instances is sent to the Serverless platform control interface, triggering scaling of workflow nodes. The Serverless platform automatically adjusts the number of workflow nodes based on the target number of instances to accommodate future peak traffic periods.

[0129] Deploy a real-time monitoring module on the Serverless platform to collect key performance indicators (KPIs) such as query latency and error rate. Based on this real-time monitoring data, evaluate the effectiveness of the current resource allocation strategy. If query latency and error rate consistently exceed preset thresholds, this indicates insufficient resource allocation and requires an appropriate increase in the safety factor. Conversely, if resource utilization is low, this indicates excessive resource allocation and requires an appropriate reduction in the safety factor. Dynamically update the safety factor based on the evaluation results and apply this updated safety factor to subsequent resource allocation strategy adjustments.

[0130] In this example, by dynamically adjusting the resource allocation strategy of the Serverless workflow, resource allocation is more accurately aligned with actual query traffic demand, improving resource utilization. During peak traffic periods, increasing the number of workflow nodes effectively reduces query latency and error rates, improving the user experience. By real-time monitoring and dynamically adjusting resource allocation strategies, the system can better cope with sudden traffic bursts and changes in query patterns, enhancing system stability and reliability.

[0131] Furthermore, the target number of instances satisfies

[0132]

[0133] in, represents the number of target instances, represents the total flow prediction value, t represents the time window, μ represents the safety factor, Q max Indicates the maximum number of concurrent queries per instance, N base Indicates the number of base instances.

[0134] In this embodiment, the target number of instances represents the number of running instances that the system needs to dynamically adjust, the total traffic forecast represents the total number of query requests expected within a future time window, the time window represents the time span (e.g., seconds or minutes) corresponding to the predicted traffic, the safety factor represents the reserved redundancy factor (>1) used to cope with sudden traffic or fluctuations, the maximum number of concurrent queries per instance represents the upper limit of query requests that a single instance can handle simultaneously, and the basic number of instances represents the minimum number of instances that the system runs by default to ensure the availability of basic services. Accurate resource allocation is achieved through multi-dimensional parameters such as traffic forecast, safety factor, and instance performance.

[0135] In some embodiments, in step S105 above, distributing the optimized executable query statement to a serverless workflow node adapted to different query engines for execution, and encapsulating the returned data into a standardized API for output, specifically includes:

[0136] Setting an engine adapter set of the query engine, wherein the engine adapter set includes multiple adapters, each adapter corresponding to a database type;

[0137] Generate a compatibility tag for the engine adapter set based on the grammatical features of the optimized executable query statement and the target data source metadata, wherein the compatibility tag is used to convert the executable query statement into a target engine statement;

[0138] An engine status matrix is built based on real-time load, network latency, and historical success rates. Based on the engine status matrix, a routing weight function is used to calculate the priority score of each adapter.

[0139] The optimal query engine is determined from multiple query engines based on the priority score. The optimized executable query statement is distributed to the workflow node corresponding to the optimal engine for execution. The execution status is monitored through the heartbeat mechanism. If the execution is not completed within the timeout threshold, a retry or failover is triggered.

[0140] Parse the source data type of the returned raw data by field, perform standardized mapping based on the pre-set type conversion matrix, and generate a JSON object with a unified structure;

[0141] Inject the JSON object into the API response template, append the execution metadata, and output it to the client.

[0142] In this embodiment, different database systems (such as relational databases, NoSQL databases, graph databases, etc.) have different query syntax and execution characteristics. Under the Serverless architecture, how to efficiently and accurately execute cross-database query requests and output the results to the client in a standardized manner has become an urgent problem to be solved. Therefore, the present invention sets an engine adapter set, generates compatibility labels according to the grammatical features of the query statement and the target data source metadata, calculates the priority score of each adapter based on the engine status matrix, determines the optimal engine and distributes the query statement execution, and finally encapsulates the returned data into a standardized API for output.

[0143] Specifically, a unified adapter interface is designed, defining methods that adapters must implement, such as translateQuery (translating generic query statements into statements for the target engine), executeQuery (executing queries on the target engine), and parseResult (parsing query results). Adapter classes are implemented for different database types (such as MySQL, MongoDB, and Neo4j). Each adapter class implements the defined interface methods and is optimized for the query syntax and execution characteristics of a specific database. All implemented adapter classes are instantiated and stored in a collection, forming the engine adapter collection.

[0144] Use syntax analysis tools or regular expressions to analyze the syntactic features of the optimized executable query statement, such as the SQL dialect, keywords, and functions used. Obtain metadata about the target data source from the data source management system, such as the database type, table structure, and field types. Generate a compatibility label based on the query statement's syntactic features and the target data source's metadata. The compatibility label indicates whether the query statement can be converted to the target engine statement by a specific adapter, as well as the complexity and potential risks of the conversion.

[0145] Deploy a monitoring module within the Serverless platform to collect real-time data on each query engine's load, network latency, and historical success rate. Organize the collected engine status data into a matrix, where each row represents an engine and each column represents a status metric (such as load, latency, success rate, etc.). Design a routing weight function to calculate the priority score of each adapter based on the data in the engine status matrix. The routing weight function can consider multiple factors, such as load balancing, network latency optimization, and historical success rate. A higher priority score indicates that the engine corresponding to the adapter is more suitable for executing the current query request.

[0146] The optimal engine is determined from multiple query engines based on priority scores. The optimal engine is the one most suitable for executing the query request in the current state. The optimized, executable query statement is distributed to the workflow node corresponding to the optimal engine for execution. A workflow node is the unit responsible for executing query requests in the Serverless platform. A heartbeat mechanism monitors the execution status of query statements. If a query fails to complete within the timeout threshold, a retry or failover mechanism is triggered to ensure reliable query execution.

[0147] The returned raw data is parsed field by field to determine its source data type, such as string, integer, or floating-point number. Based on a pre-configured type conversion matrix, the parsed data is mapped into a unified structure. The type conversion matrix defines the conversion rules between different data types, ensuring the consistency and readability of the returned data. The mapped data is then encapsulated into a uniformly structured JSON object for easy processing and output.

[0148] The generated JSON object is injected into a predefined API response template. The API response template defines the format and structure of the response data, including data fields, status code, and message. Execution metadata, such as query time, execution engine, and execution status, is appended to the API response to help the client understand the query request's execution status. The encapsulated API response is output to the client, completing the query request processing and return.

[0149] In this embodiment, by selecting the optimal engine to execute query requests, query latency and resource waste are reduced, improving query execution efficiency. By setting up a collection of engine adapters and generating compatibility tags, the system supports multiple database types and query syntaxes, enhancing system compatibility and scalability. By standardizing mapping and API response output, the system can encapsulate data returned by different databases into uniformly structured JSON objects, improving data standardization and readability.

[0150] Reference Figure 3 An embodiment of the present invention provides an intelligent data acquisition API generation system 3, the system 3 specifically comprising:

[0151] The first generating module 301 is used to generate a dynamic drag interface based on the user role information, and obtain the user-defined function input parameters and function output parameters through the dynamic drag interface;

[0152] The second generation module 302 is used to analyze user intentions based on semantic features and operation sequences in historical user interaction data using a Transformer-based dual-channel neural network model, and generate target data topic logic based on function input parameters;

[0153] The third generation module 303 is used to convert the target data subject logic into an executable query statement using an SQL splicing tool, and recommend index optimization solutions to the database engine based on the high-frequency conditions and field combinations in the historical query pattern;

[0154] The fourth generation module 304 is used to monitor the time series characteristics of historical query traffic in real time, predict future traffic peak periods, and dynamically adjust the resource allocation strategy of the Serverless workflow;

[0155] The fifth generation module 305 is used to distribute the optimized executable query statements to Serverless workflow nodes adapted to different query engines for execution, and encapsulate the returned data into a standardized API for output.

[0156] It is understandable that if Figure 1 The contents of the embodiment of the intelligent data acquisition API generation method shown in the figure are applicable to the embodiment of the intelligent data acquisition API generation system. The functions specifically implemented by the embodiment of the intelligent data acquisition API generation system are similar to those in the embodiment of the figure. Figure 1 The embodiment of the intelligent data acquisition API generation method shown is the same as that shown in FIG. Figure 1 The beneficial effects achieved by the embodiment of the intelligent data acquisition API generation method shown are also the same.

[0157] It should be noted that the information interaction, execution process and other contents between the above-mentioned systems are based on the same concept as the embodiment of the method of the present invention. Their specific functions and technical effects can be found in the method embodiment part and will not be repeated here.

[0158] Those skilled in the art can clearly understand that, for the convenience and brevity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In actual applications, the above-mentioned functions can be distributed and completed by different functional units and modules as needed, that is, the internal structure of the system can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated into one processing unit, or each unit can exist physically alone, or two or more units can be integrated into one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

[0159] Reference Figure 4An embodiment of the present invention further provides a computer device 4, comprising: a memory 402, a processor 401, and a computer program 403 stored in the memory 402. When the computer program 403 is executed on the processor 401, the intelligent data acquisition API generation method described in any one of the above methods is implemented.

[0160] The computer device 4 can be a desktop computer, a notebook computer, a palmtop computer, a cloud server or other computing devices. The computer device 4 can include, but is not limited to, a processor 401 and a memory 402. Those skilled in the art will understand that Figure 4 This is merely an example of the computer device 4 and does not constitute a limitation on the computer device 4 . The computer device 4 may include more or fewer components than shown in the figure, or a combination of certain components, or different components. For example, the computer device 4 may also include input and output devices, network access devices, etc.

[0161] The processor 401 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. A general-purpose processor may be a microprocessor or any conventional processor.

[0162] In some embodiments, the memory 402 may be an internal storage unit of the computer device 4, such as a hard disk or memory of the computer device 4. In other embodiments, the memory 402 may also be an external storage device of the computer device 4, such as a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) card, a Flash Card, etc. equipped on the computer device 4. Furthermore, the memory 402 may include both an internal storage unit of the computer device 4 and an external storage device. The memory 402 is used to store an operating system, application programs, a boot loader, data, and other programs, such as the program code of the computer program. The memory 402 may also be used to temporarily store data that has been output or is about to be output.

[0163] An embodiment of the present invention further provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the method for generating an intelligent data acquisition API as described in any one of the above methods is implemented.

[0164] In this embodiment, if the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the process of the above-mentioned method embodiment by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium. When the computer program is executed by a processor, it can implement the steps of each of the above-mentioned method embodiments. The computer program includes computer program code, which can be in source code form, object code form, executable file, or some intermediate form. The computer-readable medium can at least include: any entity or device capable of carrying computer program code to the camera / terminal device, recording medium, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal, and software distribution medium. For example, a USB flash drive, mobile hard drive, magnetic disk, or optical disk. In some jurisdictions, based on legislation and patent practice, computer-readable media cannot be electric carrier signals or telecommunication signals.

[0165] In the above embodiments, the description of each embodiment has its own focus. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant description of other embodiments.

[0166] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0167] In the embodiments disclosed in the present application, it should be understood that the disclosed devices / terminal equipment and methods can be implemented in other ways. For example, the device / terminal equipment embodiments described above are merely schematic. For example, the division of the modules or units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0168] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

Claims

1. A method for generating an intelligent data acquisition API, characterized in that: The method specifically includes: Generate a dynamic drag-and-drop interface based on user role information, and obtain user-defined function input and output parameters through the dynamic drag-and-drop interface; Based on the semantic features and operation sequences in historical user interaction data, a Transformer-based dual-channel neural network model is used to analyze user intent and generate the target data theme logic based on function input and output parameters. Use SQL splicing tools to convert the target data subject logic into executable query statements. At the same time, based on the high-frequency conditions and field combinations in historical query patterns, recommend index optimization solutions to the database engine. By monitoring the time series characteristics of historical query traffic in real time, we can predict future traffic peak periods and dynamically adjust the resource allocation strategy of Serverless workflows. The optimized executable query statements are distributed to Serverless workflow nodes adapted to different query engines for execution, and the returned data is encapsulated as a standardized API for output.

2. The method according to claim 1, characterized in that The method of generating a dynamic drag interface based on user role information and obtaining user-defined function input parameters and function output parameters through the dynamic drag interface specifically includes: Re-obtain the role tag set from the enterprise identity management system, obtain the table field set from the database metadata, and build a role-field permission matrix based on the role tag set and the table field set; Determine the user's current role ID and generate a subset of operable fields based on the role-field permission matrix and the current role ID. When the user drags a field to the input / output parameter area, the metadata constraint relationship between the bound field and the function parameter is determined based on the operable field subset, parameter type constraints are generated, and the validity of the parameter combination is checked through a lightweight syntax tree. Obtain the historical operation records corresponding to the current role ID and the current session semantic features, calculate the rendering priority of the interface components through a weighted formula, and dynamically generate the field arrangement layout within the drag area.

3. The method according to claim 1, characterized in that The method uses a Transformer-based dual-channel neural network model to analyze user intent based on the semantic features and operation sequences in historical user interaction data, and generates target data theme logic based on function input and output parameters. Specifically, it includes: Obtain historical user interaction data, extract user operation sequences and conversation text sets from the historical user interaction data, and generate semantic feature vectors through a pre-trained language model; Input the user operation sequence into the first Transformer encoder channel to generate the operation sequence hidden state; Input the semantic feature vector into the second Transformer encoder channel to generate the semantic hidden state; The feature fusion of the operation sequence hidden state and the semantic hidden state is performed through a learnable weight matrix to obtain a joint representation; Based on the attention mechanism, the matching degree between the joint representation and the preset logic template library is calculated to determine the optimal template. The placeholders are replaced in combination with the current function input and output parameters to generate the target data theme logic.

4. The method according to claim 1, wherein The SQL splicing tool is used to convert the target data subject logic into executable query statements. At the same time, based on the high-frequency conditions and field combinations in the historical query pattern, index optimization solutions are recommended to the database engine, specifically including: Parse the target data subject logic into a directed acyclic graph, and use a graph traversal algorithm to extract the SELECT clause field set, WHERE clause condition set, and JOIN relationship set from the directed acyclic graph; Based on the frequency of occurrence of fields in the WHERE clause condition set, a condition-field frequency matrix is constructed and the index recommendation score is calculated based on the field co-occurrence probability. Generate a joint index creation instruction for the field combination whose index recommendation score is greater than or equal to the preset score threshold, and submit it to the automatic execution queue of the database engine; Based on the joint index creation instruction, the SELECT clause field set, WHERE clause condition set and JOIN relationship set are used as input, and a standardized SQL query statement is generated through syntax tree reconstruction. Parameterized placeholders are injected to obtain an index optimization solution.

5. The method according to claim 1, wherein This approach monitors the time series characteristics of historical query traffic in real time, predicts future traffic peak periods, and dynamically adjusts the resource allocation strategy for Serverless workflows. Specifically, it includes: The time series data of historical query traffic is collected in real time and separated into K modal components and residual terms using the variational mode decomposition algorithm; Each modal component is input into the temporal convolutional network, and the residual term is input into the long short-term memory network. The component value and residual value in the future time window are predicted respectively to obtain the prediction result; Perform modal reorganization on the prediction results to obtain the total traffic prediction value, and calculate the target number of instances based on the preset safety factor and instance capacity; Send the target number of instances to the Serverless platform control interface to trigger scaling operations on workflow nodes. The safety factor is dynamically updated based on real-time monitoring of query latency and error rate data.

6. The method according to claim 5, characterized in that The target number of instances satisfies in, represents the number of target instances, represents the total flow prediction value, t represents the time window, μ represents the safety factor, Q max Indicates the maximum number of concurrent queries per instance, N base Indicates the number of base instances.

7. The method according to any one of claims 1 to 6, characterized in that The optimized executable query statements are distributed to Serverless workflow nodes adapted to different query engines for execution, and the returned data is encapsulated as a standardized API for output, specifically including: Setting an engine adapter set of the query engine, wherein the engine adapter set includes multiple adapters, each adapter corresponding to a database type; Generate a compatibility tag for the engine adapter set based on the grammatical features of the optimized executable query statement and the target data source metadata, wherein the compatibility tag is used to convert the executable query statement into a target engine statement; An engine status matrix is built based on real-time load, network latency, and historical success rates. Based on the engine status matrix, a routing weight function is used to calculate the priority score of each adapter. The optimal query engine is determined from multiple query engines based on the priority score. The optimized executable query statement is distributed to the workflow node corresponding to the optimal engine for execution. The execution status is monitored through the heartbeat mechanism. If the execution is not completed within the timeout threshold, a retry or failover is triggered. Parse the source data type of the returned raw data by field, perform standardized mapping based on the pre-set type conversion matrix, and generate a JSON object with a unified structure; Inject the JSON object into the API response template, append the execution metadata, and output it to the client.

8. An intelligent data acquisition API generation system, characterized in that: The system specifically includes: The first generation module is used to generate a dynamic drag interface based on the user role information, and obtain the user-defined function input parameters and function output parameters through the dynamic drag interface; The second generation module is used to analyze user intent based on the semantic features and operation sequences in historical user interaction data using a Transformer-based dual-channel neural network model, and generate the target data theme logic based on the function input parameters; The third generation module is used to convert the target data subject logic into executable query statements using SQL splicing tools. At the same time, based on the high-frequency conditions and field combinations in historical query patterns, it recommends index optimization solutions to the database engine; The fourth generation module is used to monitor the time series characteristics of historical query traffic in real time, predict future traffic peak periods, and dynamically adjust the resource allocation strategy of the Serverless workflow; The fifth generation module is used to distribute the optimized executable query statements to Serverless workflow nodes adapted to different query engines for execution, and encapsulate the returned data into a standardized API for output.

9. A computer device, characterized in that: include: A memory, a processor, and a computer program stored in the memory, which, when executed on the processor, implements the intelligent data acquisition API generation method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that A computer program is stored thereon, and when the computer program is executed by a processor, the intelligent data acquisition API generation method according to any one of claims 1 to 7 is implemented.

Images