A website service automatic conversion method and system based on network request capture

Abstract

The application provides a website service automatic conversion method and system based on network request capture, and relates to the cross technical field of artificial intelligence and software engineering, the method comprises the following steps: driving a browser automation framework to access a target website, completing identity authentication, and maintaining the session state required for subsequent operation; obtaining all network requests initiated in the browser context, and screening out target API call requests; extracting key information, automatically generating tool function definition and data model definition; automatically generating corresponding MCP server implementation code; binding the tool function and the actual HTTP call logic, so that when the MCP client calls the tool corresponding to the tool function, a request conforming to the API specification of the target website can be automatically constructed, and the processed response is returned; starting the MCP server generated by the MCP server implementation code. According to the application, the maintenance cost can be reduced, and the development efficiency of the MCP tool can be improved.

Description

Technical Field

[0001] This invention relates to the field of interdisciplinary technology of artificial intelligence and software engineering, and in particular to a method and system for automated conversion of website services based on network request capture. Background Technology

[0002] In related technologies, developing an MCP server (tool) for a specific website or web service is a manual, tedious, and error-prone process. It typically involves developers manually analyzing the target website's login mechanism and front-end interaction logic, manually writing adaptation code based on the analysis of request and response data, and encapsulating irregular website APIs into a structured MCP tool. This technology heavily relies on manual analysis, requires repetitive work for each website, has a long development cycle, and any changes to the target website's front-end or API can cause the MCP tool to malfunction, requiring manual re-analysis and modification of the adaptation code. In other words, this technology struggles to improve the development and maintenance efficiency of MCP servers.

[0003] The information disclosed in the background section of this application is intended only to enhance the understanding of the general background of this application and should not be construed as an admission or in any way implying that the information constitutes prior art known to those skilled in the art. Summary of the Invention

[0004] This invention provides a method and system for automated conversion of website services based on network request capture, which can solve the technical problem that related technologies are unable to improve the development and maintenance efficiency of MCP servers.

[0005] According to a first aspect of the present invention, a method for automated conversion of website services based on network request capture is provided, comprising:

[0006] Based on the preset login credentials and the target website login page configuration, the browser automation framework is driven to access the target website. By simulating user input operations and login form submission operations, identity authentication is completed, and the session state required for subsequent operations is maintained.

[0007] In the browser automation framework, the network request listening function is used to obtain all network requests initiated within the browser context, and according to preset rules, the target API call request is selected from all the obtained network requests, and the corresponding response data is recorded completely.

[0008] Extract key information from the target API call request and the response data. The key information includes: endpoint, HTTP method, request parameters, request body data structure, response body data structure, and authentication information. Based on the key information, automatically generate the tool function definition and data model definition required by the corresponding MCP tool.

[0009] Based on the tool function definition and the data model definition, the corresponding MCP server implementation code is automatically generated;

[0010] By binding utility functions to the actual HTTP call logic, when the MCP client calls the tool corresponding to the utility function, it can automatically construct a request that conforms to the target website's API specifications and return a processed response.

[0011] Start the MCP server, which is generated by the MCP server implementation code, so that the MCP server can be discovered and invoked by MCP clients.

[0012] According to the present invention, based on preset login credentials and target website login page configuration, a browser automation framework is driven to access the target website. Identity authentication is completed by simulating user input operations and login form submission, and the session state required for subsequent operations is maintained. The method also includes:

[0013] Get the number of currently valid session credentials, the preset total number of session credentials that should be held, the most recent authentication time, and the preset session validity period threshold;

[0014] The current session normality coefficient is determined based on the number of currently valid session credentials, the preset total number of session credentials that should be held, the most recent authentication consumption time, and the preset session validity period threshold.

[0015] When the current session normality coefficient is less than the preset current session normality coefficient threshold, the automatic re-login mechanism is triggered;

[0016] When the current session normal coefficient is greater than or equal to the preset current session normal coefficient threshold, the session state required for subsequent operations is maintained.

[0017] According to the present invention, the current session normality coefficient is determined based on the number of currently valid session credentials, the preset total number of session credentials to be held, the most recent authentication consumption time, and the preset session validity period threshold, including:

[0018] Set session validity weights and time decay weights;

[0019] The session validity item is determined based on the number of currently valid session credentials and the preset total number of session credentials that should be held;

[0020] The time decay term is determined based on the most recent authentication time and the preset session validity period threshold;

[0021] The normality coefficient of the current session is determined based on the session validity weight, the time decay weight, the session validity item, and the time decay item.

[0022] According to the present invention, filtering out target API call requests from all acquired network requests includes:

[0023] Obtain network request data, wherein the network request data includes: URL path matching length, total URL path length, number of request matching fields, total number of request fields, expected number of matching fields, request method consistency result, and actual delay time;

[0024] Determine the request matching degree based on the network request data;

[0025] When the request matching degree is greater than or equal to the preset request matching degree threshold, the network request is determined to be a target API call request;

[0026] When the request matching degree is less than the preset request matching degree threshold, the network request is determined to be a non-target API call request and is discarded.

[0027] According to the present invention, determining the request matching degree based on the network request data includes: according to the formula: ,

[0028] Determine the request matching degree of the k-th network request. ,in, Match the magnification factor to the preset path. The length of the URL path matching for the k-th network request. Let be the total length of the URL path for the k-th network request. To preset path semantic weights, Preset field structure weights, This represents the number of fields matched in the request for the k-th network request. Let be the total number of fields in the request for the k-th network request. Let be the expected number of fields to match for the k-th network request. As a preset method for timeliness weighting, For the consistency result of the request method of the k-th network request, , To preset the maximum tolerable delay, This represents the actual latency of the k-th network request.

[0029] According to the present invention, key information is extracted from the target API call request and the response data, wherein the key information includes: endpoint, HTTP method, request parameters, request body data structure, response body data structure, and authentication information. Based on the key information, the tool function definition and data model definition required by the corresponding MCP tool are automatically generated. The invention also includes:

[0030] Perform intelligent classification and aggregation, analyze multiple API calls that are related in business logic, and aggregate and encapsulate them into a single more complex MCP tool;

[0031] For the MCP tools after intelligent classification and aggregation, measure the API call complexity and evaluate the tool aggregation benefits to determine the aggregation benefit index;

[0032] The effectiveness of the aggregation is determined based on the aggregation yield index.

[0033] When aggregation is invalid, intelligent classification and aggregation are re-performed, and the aggregation strategy is adjusted.

[0034] According to the present invention, for the MCP tool after intelligent classification and aggregation, the API call complexity is measured and the tool aggregation benefit is evaluated to determine the aggregation benefit index, including:

[0035] Obtain the business scenario, and based on the business scenario, determine the original API business value coefficient and business-sensitive time.

[0036] Obtain raw API data, wherein the raw API data includes: the number of raw API request parameters, the number of raw API response data fields, and the number of raw API logical branches;

[0037] Obtain the average response time of the MCP tool after aggregation and the average response time of a single API call before aggregation;

[0038] The aggregation revenue index is determined based on the original API business value coefficient, the business-sensitive time, the original API data, the average response time of the MCP tool after aggregation, and the average response time of a single API call before aggregation.

[0039] According to the present invention, the aggregation revenue index is determined based on the original API business value coefficient, the business-sensitive time, the original API data, the average response time of the aggregated MCP tool, and the average response time of a single API call before aggregation, including: according to the formula: ,

[0040] Determine the aggregate return index ,in, To pre-determine the comprehensive value weight, To preset the business impact weight, , Preset parameter dimension weights, Preset field dimension weights, To preset the branch dimension weights, To preset the weights of business value dimensions, , Let be the number of original API request parameters for the i-th original API. Let i be the number of fields in the original API response data of the i-th original API. Let i be the number of logical branches of the original API for the i-th original API. Let be the original API business value coefficient of the i-th original API. The average response time of the MCP tool after aggregation. This represents the average response time of a single API call before aggregation. For business-sensitive time, n is the number of original APIs, i≤n, and i and n are both positive integers.

[0041] According to a second aspect of the present invention, a website service automation conversion system based on network request capture is provided, comprising:

[0042] The login and session maintenance module is used to drive the browser automation framework to access the target website based on the preset login credentials and the target website login page configuration. By simulating user input operations and login form submission operations, it completes identity authentication and maintains the session state required for subsequent operations.

[0043] The network request capture and filtering module is used in the browser automation framework to obtain all network requests initiated in the browser context through the network request listening function, and filter out the target API call requests from all the obtained network requests according to preset rules, and record the corresponding response data completely.

[0044] The API analysis model generation module is used to extract key information from the target API call request and the response data. The key information includes: endpoint, HTTP method, request parameters, request body data structure, response body data structure and authentication information. Based on the key information, the module automatically generates the tool function definition and data model definition required by the corresponding MCP tool.

[0045] The code implementation module is used to automatically generate the corresponding MCP server implementation code based on the tool function definition and the data model definition;

[0046] The utility function binding module is used to bind utility functions to the actual HTTP call logic, so that when the MCP client calls the tool corresponding to the utility function, it can automatically construct a request that conforms to the API specification of the target website and return the processed response.

[0047] The client discovery and invocation module is used to start the MCP server generated by the MCP server implementation code, making the MCP server available for discovery and invocation by MCP clients.

[0048] Technical Effects: According to this invention, real-world network interactions can be dynamically captured, accurately reflecting the actual API behavior of a website. This generates more precise tool function definitions and data model definitions, and automatically generates corresponding MCP server implementation code based on these definitions. The generated MCP tools adhere to unified definition specifications, improving the consistency and quality of tool interfaces, reducing integration difficulty, and allowing for rapid updates to the MCP tools when the target website changes, thus reducing maintenance costs and improving development efficiency. When determining request matching degree, the matching degree can be determined based on network request data. During the calculation process, the path semantic matching degree, field structure matching degree, and HTTP correctness and timeliness of the network request can be accurately analyzed. Based on preset path semantic weights, preset field structure weights, and preset method timeliness weights, a weighted sum is applied to the path semantic matching degree item, field structure matching degree item, and HTTP correctness and timeliness combination item to determine the request matching degree, improving the comprehensiveness and accuracy of request matching degree. When determining the aggregation revenue index, the index can be determined based on the original API business value coefficient, business-sensitive time, original API data, average response time of the MCP tool after aggregation, and average response time of a single API call before aggregation. During the calculation process, the comprehensive value of API aggregation can be evaluated based on the API call cost, call difficulty, and call value. The degree of business loss caused by the performance degradation after aggregation can be determined based on the difference in response time before and after the call. Furthermore, the aggregation revenue index is determined based on the comprehensive value of API aggregation and the degree of business loss caused by the performance degradation after aggregation, thus improving the comprehensiveness and accuracy of the aggregation revenue index.

[0049] It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory only, and are not intended to limit the invention. Other features and aspects of the invention will become clearer from the following detailed description of exemplary embodiments with reference to the accompanying drawings. Attached Figure Description

[0050] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other embodiments can be obtained based on these drawings without creative effort.

[0051] Figure 1 An exemplary flowchart illustrates a method for automated conversion of website services based on network request capture according to an embodiment of the present invention;

[0052] Figure 2An exemplary diagram illustrating the determination of whether an automatic re-login mechanism is triggered according to an embodiment of the present invention is shown.

[0053] Figure 3 An exemplary diagram illustrates how, according to an embodiment of the present invention, a target API call request is selected from all acquired network requests;

[0054] Figure 4 An exemplary schematic diagram illustrating intelligent classification and aggregation according to an embodiment of the present invention is shown;

[0055] Figure 5 An exemplary block diagram of a website service automation conversion system based on network request capture according to an embodiment of the present invention is shown. Detailed Implementation

[0056] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0057] The technical solution of the present invention will be described in detail below with reference to specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments.

[0058] Figure 1 An exemplary flowchart illustrates a method for automated conversion of website services based on network request capture according to an embodiment of the present invention, the method comprising:

[0059] Step S1: Based on the preset login credentials and the target website login page configuration, drive the browser automation framework to access the target website, complete the identity authentication by simulating user input operations and submitting login forms, and maintain the session state required for subsequent operations.

[0060] Step S2: In the browser automation framework, the network request listening function is used to obtain all network requests initiated within the browser context, and according to preset rules, the target API call request is selected from all the obtained network requests, and its corresponding response data is recorded completely.

[0061] Step S3: Extract key information from the target API call request and the response data. The key information includes: endpoint, HTTP method, request parameters, request body data structure, response body data structure, and authentication information. Based on the key information, automatically generate the tool function definition and data model definition required by the corresponding MCP tool.

[0062] Step S4: Based on the tool function definition and the data model definition, automatically generate the corresponding MCP server implementation code;

[0063] Step S5: Bind the utility function to the actual HTTP call logic so that when the MCP client calls the tool corresponding to the utility function, it can automatically construct a request that conforms to the API specification of the target website and return the processed response.

[0064] Step S6: Start the MCP server generated by the MCP server implementation code, so that the MCP server can be discovered and invoked by MCP clients.

[0065] The website service automation conversion method based on network request capture according to embodiments of the present invention can dynamically capture real network interactions, accurately reflect the actual API behavior of the website, generate more accurate tool function definitions and data model definitions, and automatically generate corresponding MCP server implementation code based on the tool function definitions and data model definitions. The generated MCP tools follow a unified definition specification, which improves the consistency and quality of tool interfaces, reduces integration difficulty, and allows for rapid updates of MCP tools when the target website changes, reducing maintenance costs and improving the development efficiency of MCP tools.

[0066] According to one embodiment of the present invention, in step S1, based on the preset login credentials and the target website login page configuration, the browser automation framework is driven to access the target website, and identity authentication is completed by simulating user input operations and login form submission operations, and the session state required for subsequent operations is maintained.

[0067] For example, simulating user input and submission of login forms includes handling dynamically loaded form elements, dealing with CAPTCHAs, handling redirects, and managing cookies and session storage.

[0068] Figure 2 An exemplary diagram illustrating the determination of whether an automatic re-login mechanism is triggered according to an embodiment of the present invention is shown.

[0069] According to one embodiment of the present invention, step S1 further includes:

[0070] Step S11: Obtain the current number of valid session credentials, the preset total number of session credentials to be held, the most recent authentication time, and the preset session validity period threshold;

[0071] Step S12: Determine the current session normality coefficient based on the number of currently valid session credentials, the preset total number of session credentials to be held, the most recent authentication consumption time, and the preset session validity period threshold.

[0072] Step S13: When the current session normality coefficient is less than the preset current session normality coefficient threshold, the automatic re-login mechanism is triggered.

[0073] Step S14: When the current session normal coefficient is greater than or equal to the preset current session normal coefficient threshold, maintain the session state required for subsequent operations.

[0074] For example, in the context of browser automation frameworks (such as Selenium and Playwright), before each request is made, the current Document.cookie string is read, and the number of valid credentials that have not expired and have not been marked as HttpOnly / Secure (depending on the business security policy) is filtered out through regular expressions or dictionary parsing. If it is a token mechanism, the number of currently valid tokens in memory or LocalStorage is checked. The number of currently valid session credentials can be obtained in the above way. The preset total number of session credentials that should be held can be determined based on the login logic analysis of the target website. For example, by observing the response headers or page source code returned by the login interface, it can be confirmed which cookies or tokens will be generated after successful login. Their number can be used as a benchmark value. After identity authentication is completed, a high-precision timer (such as Python's time.time() or time.perf_counter()) can be started internally by the system. Before each network request is initiated, the current timestamp is subtracted from the initial timestamp to obtain the number of seconds elapsed, i.e., the most recent authentication time. The preset session validity threshold is a fixed constant preset by the business, usually determined by security policies or business requirements. For example, the session expiration time of a typical web system is 30 minutes, while some payment interfaces with high security requirements may have a 10-minute expiration time. Based on the number of currently valid session credentials, the preset total number of session credentials that should be held, the most recent authentication time, and the preset session validity threshold, the integrity and freshness of the session credentials are evaluated to determine the current session normality coefficient. When the current session normality coefficient is less than the preset current session normality coefficient threshold (which can be set to 0.6), the automatic re-login mechanism is triggered. When the current session normality coefficient is greater than or equal to the preset current session normality coefficient threshold, the current session is maintained to maintain the session state required for subsequent operations.

[0075] According to an embodiment of the present invention, step S12 includes:

[0076] Step S121: Set session validity weight and time decay weight;

[0077] Step S122: Determine the session validity item based on the number of currently valid session credentials and the preset total number of session credentials to be held;

[0078] Step S123: Determine the time decay term based on the most recent authentication consumption time and the preset session validity period threshold;

[0079] Step S124: Determine the current session normality coefficient based on the session validity weight, the time decay weight, the session validity item, and the time decay item.

[0080] For example, the session validity weight and time decay weight can be set to 0.7 and 0.3 respectively. The session validity weight reflects the priority of the proportion of valid cookies / tokens in session health, while the time decay weight reflects the penalty for sessions approaching expiration. The session validity item is determined based on the ratio of the current number of valid session credentials to the preset total number of session credentials that should be held. The larger the session validity item, the higher the proportion of valid credentials, indicating that the session credentials are very complete and unlikely to trigger re-login. The smaller the session validity item, the more valid credentials are missing, which usually occurs when some cookies expire, are actively cleared by the server, or are lost due to network interception, making it very easy to trigger re-login. The time decay item is determined based on the ratio of 1 minus the most recent authentication time and the preset session validity threshold. The smaller the time decay item, the closer the time is to or beyond the validity period. The session validity item and the time decay item are weighted and summed according to the session validity weight and the time decay weight respectively to determine the current session normality coefficient.

[0081] According to an embodiment of the present invention, in step S2, in the browser automation framework, all network requests initiated within the browser context are obtained through the network request listening function, and target API call requests are filtered out from all the obtained network requests according to preset rules, and their corresponding response data are fully recorded.

[0082] For example, in a browser automation framework, the network request listening function can be used to obtain all network requests initiated within the browser context. Based on preset rules (such as URL pattern matching, request method, and request header characteristics), the target API call request can be filtered out from all the obtained network requests, and the corresponding response data can be recorded completely. All obtained network requests include requests initiated through the Fetch API and XMLHttpRequest. The capture method is to listen to network events in the browser developer tools or through the request interception API provided by the browser automation framework.

[0083] Figure 3 An exemplary diagram illustrates how, according to an embodiment of the present invention, a target API call request is selected from all acquired network requests.

[0084] According to an embodiment of the present invention, step S2 includes:

[0085] Step S21: Obtain network request data, wherein the network request data includes: URL path matching length, total URL path length, number of request matching fields, total number of request fields, expected number of matching fields, request method consistency result, and actual delay time;

[0086] Step S22: Determine the request matching degree based on the network request data;

[0087] Step S23: When the request matching degree is greater than or equal to the preset request matching degree threshold, the network request is determined to be a target API call request;

[0088] Step S24: When the request matching degree is less than the preset request matching degree threshold, the network request is determined to be a non-target API call request and is discarded.

[0089] For example, the URL path matching length represents the length of the portion of the request's URL path that matches a preset pattern. This length can be directly extracted from the URL path. For instance, if the target template is ` / api / v1 / user / {id}` and the actual request is ` / api / v1 / user / 123`, the URL path matching length is 16 (including wildcards). The total URL path length represents the total length of the request's URL path. The method for determining the total URL path length is similar to that for determining the URL path matching length, and will not be elaborated here. The number of fields matched in the request refers to the number of fields in the request / response that match the preset field template. This can be counted from the request / response JSON or form. The total number of fields in the request refers to the total number of fields in the request / response, which can also be counted from the request / response JSON or form. The expected number of matched fields refers to the number of fields in the preset field template corresponding to the request. The expected number of fields to be matched (preset by the business side, e.g., the "User Login" template expects to match username and password, a total of 2 fields) is usually preset by the business side. Request method consistency result refers to the consistency between the requested HTTP method and the target API. When the requested HTTP method matches the target API, the request method consistency result is 1; otherwise, the request method consistency result is 0. Actual latency time refers to the actual latency time from request initiation to receiving the response, which can be obtained through browser automation framework recording. Based on the network request data, the request matching degree is determined. When the request matching degree is greater than or equal to the preset request matching degree threshold (which can be set to 0.8), the network request is determined to be a target API call request. When the request matching degree is less than the preset request matching degree threshold, the network request is determined to be a non-target API call request and is discarded.

[0090] According to an embodiment of the present invention, step S22 includes: determining the request matching degree of the k-th network request according to formula (1). , (1),

[0091] in, Match the magnification factor to the preset path. The length of the URL path matching for the k-th network request. Let be the total length of the URL path for the k-th network request. To preset path semantic weights, Preset field structure weights, This represents the number of fields matched in the request for the k-th network request. Let be the total number of fields in the request for the k-th network request. Let be the expected number of fields to match for the k-th network request. As a preset method for timeliness weighting, For the consistency result of the request method of the k-th network request, , To preset the maximum tolerable delay, This represents the actual latency of the k-th network request.

[0092] According to one embodiment of the present invention, This refers to the degree of semantic matching of the path. This is the ratio of the URL path matching length of the k-th network request to the total URL path length of the k-th network request. The larger this ratio, the higher the proportion of the matching part to the total length, the higher the path similarity, and the higher the semantic matching degree of the path. The smaller this ratio, the smaller the matching part, the greater the path difference, and the lower the semantic matching degree of the path. Amplification factor is used to match the preset path. Perform nonlinear amplification. It can be set to 1.3, which allows paths with higher path similarity to achieve a higher degree of path semantic matching.

[0093] According to one embodiment of the present invention, For field structure matching degree item, This is the ratio of the number of matching fields in the k-th network request to the total number of fields in the k-th network request. The larger this ratio, the higher the proportion of matching fields and the more complete the structure; the smaller the ratio, the fewer matching fields and the less complete the structure. The relative deviation between the number of fields matched in the request for the k-th network request and the expected number of fields matched is considered. The greater the relative deviation, the more likely there will be interference from multiple or fewer fields, and the lower the degree of field structure matching.

[0094] According to one embodiment of the present invention, This is a combination of HTTP correctness and timeliness. For the consistency result of the request method of the k-th network request, when the HTTP method of the k-th network request is consistent with the target, =1, otherwise, A value of 0 indicates that if the request method is incorrect, The value is directly 0. Indicates the timeliness of the request. The larger the ratio, the higher the actual latency of the k-th network request, and the lower the timeliness of the request.

[0095] According to one embodiment of the present invention, This means that the request matching degree is determined by weighting and summing the path semantic matching degree item, the field structure matching degree item, and the HTTP correctness and timeliness combination item according to preset path semantic weight, preset field structure weight, and preset method timeliness weight. The sum of the preset path semantic weight, preset field structure weight, and preset method timeliness weight is 1, which can be preset by the business party or system architect. For example, the preset path semantic weight, preset field structure weight, and preset method timeliness weight can be set to 0.5, 0.3, and 0.2, respectively.

[0096] In this way, the request matching degree can be determined based on network request data. During the calculation process, the degree of path semantic matching, field structure matching, and HTTP correctness and timeliness of the network request can be accurately analyzed. Based on preset path semantic weight, preset field structure weight, and preset method timeliness weight, the path semantic matching degree item, field structure matching degree item, and HTTP correctness and timeliness combination item are weighted and summed to determine the request matching degree, thus improving the comprehensiveness and accuracy of the request matching degree.

[0097] According to an embodiment of the present invention, in step S3, key information is extracted from the target API call request and the response data, wherein the key information includes: endpoint, HTTP method, request parameters, request body data structure, response body data structure and authentication information, and based on the key information, the tool function definition and data model definition required by the corresponding MCP tool are automatically generated.

[0098] For example, data model definitions are generated using the syntax of the Pydantic library to ensure type safety and serialization or deserialization, while utility function definitions include function names, descriptions, and parameter lists.

[0099] Figure 4 An exemplary schematic diagram illustrating intelligent classification and aggregation according to an embodiment of the present invention is shown.

[0100] According to one embodiment of the present invention, step S3 further includes:

[0101] Step S31: Perform intelligent classification and aggregation, analyze multiple API calls that are related in business logic, and aggregate and encapsulate them into a single MCP tool with more complex functions.

[0102] Step S32: For the MCP tools after intelligent classification and aggregation, measure the API call complexity and evaluate the tool aggregation benefit to determine the aggregation benefit index.

[0103] Step S33: Determine whether the aggregation is effective based on the aggregation yield index;

[0104] Step S34: When aggregation is invalid, perform intelligent classification and aggregation again, and adjust the aggregation strategy.

[0105] For example, intelligent classification and aggregation refers to analyzing multiple API calls that are related in business logic and aggregating and encapsulating them into a single, more complex MCP tool. For instance, the API calls "Add to Cart" and "Submit Order" can be aggregated into a single "Buy Goods" tool. The MCP tool after intelligent classification and aggregation is then used to measure API call complexity and evaluate the tool's aggregation benefit, determining an aggregation benefit index. Based on the aggregation benefit index, the effectiveness of the aggregation is determined. For example, if the aggregation benefit index is less than a preset threshold based on the business scenario (e.g., set to 0.8), the aggregation is considered invalid; otherwise, it is considered valid. When aggregation is invalid, intelligent classification and aggregation are performed again, and the aggregation strategy is adjusted (e.g., reducing the number of aggregations, replacing high-cost APIs, adjusting aggregation granularity, downgrading to a single API call, or delaying aggregation).

[0106] According to an embodiment of the present invention, step S32 includes:

[0107] Step S321: Obtain the business scenario and determine the original API business value coefficient and business sensitive time based on the business scenario;

[0108] Step S322: Obtain the original API data, wherein the original API data includes: the number of original API request parameters, the number of original API response data fields, and the number of original API logical branches;

[0109] Step S323: Obtain the average response time of the MCP tool after aggregation and the average response time of a single API call before aggregation;

[0110] Step S324: Determine the aggregation revenue index based on the original API business value coefficient, the business sensitive time, the original API data, the average response time of the aggregated MCP tool, and the average response time of a single API call before aggregation.

[0111] For example, the original API business value coefficient can be preset by the business scenario. For instance, the original API business value coefficient for the "Add to Cart" business scenario is 0.3, and the original API business value coefficient for the "Place Order" business scenario is 0.7. Similarly, the business-sensitive time (representing the maximum latency increment the business can tolerate) is preset by the business scenario. For instance, the business-sensitive time for the "Place Order" business scenario is 500ms, and the business-sensitive time for the "Browse" business scenario is 1000ms. The number of original API request parameters can be obtained by extracting the number of fields from the request body / URL (e.g., JSON key-value pairs, query parameters), and the number of original API response data fields can be obtained by extracting the number of fields from the response body / URL. The number of top-level fields (e.g., JSON object keys) is obtained from the data structure. The number of original API logic branches is usually preset by the business team or static analysis tools (e.g., if / else, switch, loop branch counts, etc.). The average response time of the MCP tool after aggregation and the average response time of a single API call before aggregation can be obtained through system testing (e.g., through load testing, log statistics, historical averages). Based on the original API business value coefficient, business-sensitive time, original API data, the average response time of the MCP tool after aggregation, and the average response time of a single API call before aggregation, API call complexity is measured and the tool aggregation benefit is evaluated to determine the aggregation benefit index.

[0112] According to an embodiment of the present invention, step S324 includes: determining the aggregate return index according to formula (2). , (2),

[0113] in, To pre-determine the comprehensive value weight, To preset the business impact weight, , Preset parameter dimension weights, Preset field dimension weights, To preset the branch dimension weights, To preset the weights of business value dimensions, , Let be the number of original API request parameters for the i-th original API. Let i be the number of fields in the original API response data of the i-th original API. Let i be the number of logical branches of the original API for the i-th original API. Let be the original API business value coefficient of the i-th original API. The average response time of the MCP tool after aggregation. This represents the average response time of a single API call before aggregation. For business-sensitive time, n is the number of original APIs, i≤n, and i and n are both positive integers.

[0114] According to one embodiment of the present invention, Preset parameter dimension weights, Preset field dimension weights, To preset the branch dimension weights, To preset the weights for the business value dimensions, reflecting the importance of the technical input, technical output, technical logic, and business value dimensions respectively, they can be set to 0.3, 0.2, 0.2, and 0.3 respectively. Let be the number of original API request parameters for the i-th original API. Let i be the number of fields in the original API response data of the i-th original API. Let i be the number of logical branches of the original API for the i-th original API. , and The larger the value, the higher the technical complexity, the greater the processing burden, the more complex the logic, and the higher the cost of invocation. Let be the original API business value coefficient of the i-th original API. The larger the size, the higher the business value. This indicates the average overall value of an API. For example, APIs with a high original API business value coefficient, low number of request parameters, low number of response data fields, and low number of logical branches have a higher overall value.

[0115] According to one embodiment of the present invention, This is the ratio of the average response time of the MCP tool after aggregation (minus the average response time of a single API call before aggregation) to the business-sensitive time, representing the degree of business loss caused by the performance degradation after aggregation. The larger the value, the more severe the performance degradation, exceeding the tolerance range of the business.

[0116] According to one embodiment of the present invention, The aggregation revenue index is determined based on the average overall value of the APIs and the degree of business loss caused by the performance degradation after aggregation. ,in, To pre-determine the comprehensive value weight, To preset the business impact weight, and It can be configured according to the business scenario, such as for business scenarios that focus on overall value. and They can be set to 0.7 and 0.3 respectively.

[0117] In this way, the aggregation revenue index can be determined based on the original API business value coefficient, business-sensitive time, original API data, average response time of the MCP tool after aggregation, and average response time of a single API call before aggregation. During the calculation process, the comprehensive value of API aggregation can be evaluated based on the API call cost, call difficulty, and call value. The degree of business loss caused by the performance degradation after aggregation can be determined based on the difference in response time before and after the call. Furthermore, the aggregation revenue index is determined based on the comprehensive value of API aggregation and the degree of business loss caused by the performance degradation after aggregation, thus improving the comprehensiveness and accuracy of the aggregation revenue index.

[0118] According to an embodiment of the present invention, in step S4, the corresponding MCP server implementation code is automatically generated based on the tool function definition and the data model definition.

[0119] For example, based on the tool function definition and data model definition, an MCP server skeleton (e.g., a Python project based on FastAPI) can be automatically created using a template or code generator, which is the corresponding MCP server implementation code.

[0120] According to one embodiment of the present invention, in step S5, the utility function is bound to the actual HTTP call logic, so that when the MCP client calls the tool corresponding to the utility function, it can automatically construct a request that conforms to the API specification of the target website and return the processed response.

[0121] For example, each utility function can be bound to the actual HTTP request logic. When the utility is called, it will automatically access the target website according to the original request format (e.g., URL, method, header, body, cookie) of all network requests obtained in step S2.

[0122] According to one embodiment of the present invention, in step S6, the MCP server, which is generated by the MCP server implementation code, is started, so that the MCP server can be discovered and invoked by the MCP client.

[0123] For example, start an MCP server that generates code using an MCP server, have the generated MCP server listen on a port, and provide standard MCP interfaces (such as / tools to list tools, / tools / {name} to execute calls) for AI assistants and other clients to discover and call.

[0124] Furthermore, the present invention can also provide users with configuration files or graphical interfaces for configuring login information, selecting the API mode to be captured, and reviewing, modifying and confirming the automatically generated tool definitions.

[0125] The website service automation conversion method based on network request capture according to embodiments of the present invention can dynamically capture real network interactions, accurately reflect the actual API behavior of the website, generate more precise tool function definitions and data model definitions, and automatically generate corresponding MCP server implementation code based on the tool function definitions and data model definitions. The generated MCP tools follow a unified definition specification, improving the consistency and quality of tool interfaces, reducing integration difficulty, and allowing for rapid updates to the MCP tools when the target website changes, reducing maintenance costs and improving the development efficiency of MCP tools. When determining the request matching degree, the request matching degree can be determined based on network request data. During the calculation process, the path semantic matching degree, field structure matching degree, and HTTP correctness and timeliness of the network request can be accurately analyzed. Based on preset path semantic weights, preset field structure weights, and preset method timeliness weights, a weighted sum is performed on the path semantic matching degree item, the field structure matching degree item, and the HTTP correctness and timeliness combination item to determine the request matching degree, improving the comprehensiveness and accuracy of the request matching degree. When determining the aggregation revenue index, the index can be determined based on the original API business value coefficient, business-sensitive time, original API data, average response time of the MCP tool after aggregation, and average response time of a single API call before aggregation. During the calculation process, the comprehensive value of API aggregation can be evaluated based on the API call cost, call difficulty, and call value. The degree of business loss caused by the performance degradation after aggregation can be determined based on the difference in response time before and after the call. Furthermore, the aggregation revenue index is determined based on the comprehensive value of API aggregation and the degree of business loss caused by the performance degradation after aggregation, thus improving the comprehensiveness and accuracy of the aggregation revenue index.

[0126] Figure 5 An exemplary block diagram of a website service automation conversion system based on network request capture according to an embodiment of the present invention is shown, the system comprising:

[0127] The login and session maintenance module is used to drive the browser automation framework to access the target website based on the preset login credentials and the target website login page configuration. By simulating user input operations and login form submission operations, it completes identity authentication and maintains the session state required for subsequent operations.

[0128] The network request capture and filtering module is used in the browser automation framework to obtain all network requests initiated in the browser context through the network request listening function, and filter out the target API call requests from all the obtained network requests according to preset rules, and record the corresponding response data completely.

[0129] The API analysis model generation module is used to extract key information from the target API call request and the response data. The key information includes: endpoint, HTTP method, request parameters, request body data structure, response body data structure and authentication information. Based on the key information, the module automatically generates the tool function definition and data model definition required by the corresponding MCP tool.

[0130] The code implementation module is used to automatically generate the corresponding MCP server implementation code based on the tool function definition and the data model definition;

[0131] The utility function binding module is used to bind utility functions to the actual HTTP call logic, so that when the MCP client calls the tool corresponding to the utility function, it can automatically construct a request that conforms to the API specification of the target website and return the processed response.

[0132] The client discovery and invocation module is used to start the MCP server generated by the MCP server implementation code, making the MCP server available for discovery and invocation by MCP clients.

[0133] This invention can be a method, apparatus, system, and / or computer program product. The computer program product may include a computer-readable storage medium having computer-readable program instructions loaded thereon for performing various aspects of the invention.

[0134] Those skilled in the art should understand that the embodiments of the present invention described above and shown in the accompanying drawings are merely examples and do not limit the present invention. The objectives of the present invention have been fully and effectively achieved. The functions and structural principles of the present invention have been demonstrated and explained in the embodiments, and any variations or modifications may be made to the implementation of the present invention without departing from the stated principles.

Claims

1. A method for automatically converting website services based on network request capture, characterized in that, include: Based on the preset login credentials and the target website login page configuration, the browser automation framework is driven to access the target website. By simulating user input operations and login form submission operations, identity authentication is completed, and the session state required for subsequent operations is maintained. In the browser automation framework, the network request listening function is used to obtain all network requests initiated within the browser context, and according to preset rules, the target API call request is selected from all the obtained network requests, and the corresponding response data is recorded completely. Extract key information from the target API call request and the response data. The key information includes: endpoint, HTTP method, request parameters, request body data structure, response body data structure, and authentication information. Based on the key information, automatically generate the tool function definition and data model definition required by the corresponding MCP tool. Based on the tool function definition and the data model definition, the corresponding MCP server implementation code is automatically generated; By binding utility functions to the actual HTTP call logic, when the MCP client calls the tool corresponding to the utility function, it can automatically construct a request that conforms to the target website's API specifications and return a processed response. Start the MCP server, which is generated by the MCP server implementation code, so that the MCP server can be discovered and invoked by MCP clients.

2. The method for automated conversion of website services based on network request capture according to claim 1, characterized in that, Based on preset login credentials and the target website's login page configuration, the browser automation framework is driven to access the target website. By simulating user input and form submission, authentication is completed, and the session state required for subsequent operations is maintained. This also includes: Get the number of currently valid session credentials, the preset total number of session credentials that should be held, the most recent authentication time, and the preset session validity period threshold; The current session normality coefficient is determined based on the number of currently valid session credentials, the preset total number of session credentials that should be held, the most recent authentication consumption time, and the preset session validity period threshold. When the current session normality coefficient is less than the preset current session normality coefficient threshold, the automatic re-login mechanism is triggered; When the current session normal coefficient is greater than or equal to the preset current session normal coefficient threshold, the session state required for subsequent operations is maintained.

3. The method for automated conversion of website services based on network request capture according to claim 2, characterized in that, The current session normality coefficient is determined based on the number of currently valid session credentials, the preset total number of session credentials that should be held, the most recent authentication time, and the preset session validity period threshold, including: Set session validity weights and time decay weights; The session validity item is determined based on the number of currently valid session credentials and the preset total number of session credentials that should be held; The time decay term is determined based on the most recent authentication time and the preset session validity period threshold; The normality coefficient of the current session is determined based on the session validity weight, the time decay weight, the session validity item, and the time decay item.

4. The method for automated conversion of website services based on network request capture according to claim 1, characterized in that, Filter the target API call requests from all the network requests obtained, including: Obtain network request data, wherein the network request data includes: URL path matching length, total URL path length, number of request matching fields, total number of request fields, expected number of matching fields, request method consistency result, and actual delay time; Determine the request matching degree based on the network request data; When the request matching degree is greater than or equal to the preset request matching degree threshold, the network request is determined to be a target API call request; When the request matching degree is less than the preset request matching degree threshold, the network request is determined to be a non-target API call request and is discarded.

5. The method for automated conversion of website services based on network request capture according to claim 4, characterized in that, Determining the request matching degree based on the network request data includes: according to the formula: ， Determine the request matching degree of the k-th network request. ,in, Match the magnification factor to the preset path. The length of the URL path matching for the k-th network request. Let be the total length of the URL path for the k-th network request. To preset path semantic weights, Preset field structure weights, This represents the number of fields matched in the request for the k-th network request. Let be the total number of fields in the request for the k-th network request. Let be the expected number of fields to match for the k-th network request. As a preset method for timeliness weighting, For the consistency result of the request method of the k-th network request, , To preset the maximum tolerable delay, This represents the actual latency of the k-th network request.

6. The method for automated conversion of website services based on network request capture according to claim 1, characterized in that, Extract key information from the target API call request and the response data. This key information includes: endpoint, HTTP method, request parameters, request body data structure, response body data structure, and authentication information. Based on this key information, automatically generate the tool function definitions and data model definitions required by the corresponding MCP tool. (The last part, "and also includes:") Perform intelligent classification and aggregation, analyze multiple API calls that are related in business logic, and aggregate and encapsulate them into a single more complex MCP tool; For the MCP tools after intelligent classification and aggregation, measure the API call complexity and evaluate the tool aggregation benefits to determine the aggregation benefit index; The effectiveness of the aggregation is determined based on the aggregation yield index. When aggregation is invalid, intelligent classification and aggregation are re-performed, and the aggregation strategy is adjusted.

7. The method for automated conversion of website services based on network request capture according to claim 6, characterized in that, For the MCP tool after intelligent classification and aggregation, API call complexity is measured and tool aggregation benefit is evaluated to determine the aggregation benefit index, including: Obtain the business scenario, and based on the business scenario, determine the original API business value coefficient and business-sensitive time. Obtain raw API data, wherein the raw API data includes: the number of raw API request parameters, the number of raw API response data fields, and the number of raw API logical branches; Obtain the average response time of the MCP tool after aggregation and the average response time of a single API call before aggregation; The aggregation revenue index is determined based on the original API business value coefficient, the business-sensitive time, the original API data, the average response time of the MCP tool after aggregation, and the average response time of a single API call before aggregation.

8. The method for automated conversion of website services based on network request capture according to claim 7, characterized in that, Based on the original API business value coefficient, the business-sensitive time, the original API data, the average response time of the aggregated MCP tool, and the average response time of a single API call before aggregation, the aggregation revenue index is determined, including: according to the formula: ， Determine the aggregate return index ,in, To pre-determine the comprehensive value weight, To preset the business impact weight, , Preset parameter dimension weights, Preset field dimension weights, To preset the branch dimension weights, To preset the weights of business value dimensions, , Let be the number of original API request parameters for the i-th original API. Let i be the number of fields in the original API response data of the i-th original API. Let i be the number of logical branches of the original API for the i-th original API. Let be the original API business value coefficient of the i-th original API. The average response time of the MCP tool after aggregation. This represents the average response time of a single API call before aggregation. For business-sensitive time, n is the number of original APIs, i≤n, and i and n are both positive integers.

9. A website service automated conversion system based on network request capture, characterized in that, The system is used to perform the method of any one of claims 1-8, comprising: The login and session maintenance module is used to drive the browser automation framework to access the target website based on the preset login credentials and the target website login page configuration. By simulating user input operations and login form submission operations, it completes identity authentication and maintains the session state required for subsequent operations. The network request capture and filtering module is used in the browser automation framework to obtain all network requests initiated in the browser context through the network request listening function, and filter out the target API call requests from all the obtained network requests according to preset rules, and record the corresponding response data completely. The API analysis model generation module is used to extract key information from the target API call request and the response data. The key information includes: endpoint, HTTP method, request parameters, request body data structure, response body data structure and authentication information. Based on the key information, the module automatically generates the tool function definition and data model definition required by the corresponding MCP tool. The code implementation module is used to automatically generate the corresponding MCP server implementation code based on the tool function definition and the data model definition; The utility function binding module is used to bind utility functions to the actual HTTP call logic, so that when the MCP client calls the tool corresponding to the utility function, it can automatically construct a request that conforms to the API specification of the target website and return the processed response. The client discovery and invocation module is used to start the MCP server generated by the MCP server implementation code, making the MCP server available for discovery and invocation by MCP clients.

Images