A business processing method, system and terminal based on a large model middleware engine

Abstract

This application relates to a business processing method, system, and terminal based on a large model middleware engine, belonging to the technical field of data processing. The business processing method uses the middleware engine as the execution entity and includes: receiving business requests sent by the business application side through a unified API entry point; parsing the business requests to identify business function point tags, business parameters, and output formats; filtering a set of associated models matching the business function tags from a large model cluster; selecting the optimal target model from the set of associated models; reconstructing the business parameters according to the interface specifications of the optimal target model and sending a request to the optimal target model; receiving the processing results sent by the optimal target model, encapsulating the processing results according to the output format, and pushing them to the business application side, whereby the business application side presents the processing results. This application has the beneficial effects of reducing the development and maintenance costs of business applications and improving the efficiency and quality of business processing.

Description

Technical Field

[0001] This application relates to the field of data processing technology, and in particular to a business processing method, system and terminal based on a large model middleware engine. Background Technology

[0002] In today's digital age, the demand for intelligent business processing is growing across industries. Large-scale models, with their powerful language understanding, generation, and multi-domain adaptability, have been widely applied in various fields, such as providing natural and fluent dialogue experiences, generating high-quality articles, improving the accuracy of financial credit risk assessment, and assisting in medical diagnosis.

[0003] Currently, the market offers numerous application models from different brands and for various application scenarios. Each model has its own independent application interface methods to solve different business problems. Business applications need to develop specific interfaces and code for different models to send business requests and receive processing results. Furthermore, business applications also need to manage the selection and invocation of models themselves, choosing the appropriate model from multiple models for business processing based on their specific needs.

[0004] Because different large models have different interface specifications, capabilities, and service levels, business applications need to write dedicated adaptation code for each large model. Furthermore, when a large model is updated or replaced, the code needs to be modified and adjusted accordingly, significantly increasing the development and maintenance costs of business applications. In addition, managing the selection and invocation of large models by business applications requires specialized technical knowledge and experience, which is a huge challenge for many enterprises and can easily lead to inappropriate model selection, affecting the efficiency and quality of business processing. Summary of the Invention

[0005] To reduce the development and maintenance costs of business applications and improve the efficiency and quality of business processing, this application provides a business processing method, system, and terminal based on a large model middleware engine.

[0006] Firstly, this application provides a business processing method based on a large model middleware engine, employing the following technical solution:

[0007] A business processing method based on a large-model middleware engine, using the middleware engine as the execution entity, includes:

[0008] Receive business requests sent by the business application side through a unified API entry point;

[0009] The business request is parsed to identify business function point tags, business parameters, and output format;

[0010] The associated model set that matches the business function tag is selected from the large model cluster. The various large models in the large model cluster have their capability tags, service levels and API access credentials pre-registered in the middleware engine.

[0011] Select the optimal target model from the set of associated models;

[0012] The business parameters are reconstructed according to the interface specification of the optimal target model, and a request is sent to the optimal target model;

[0013] The system receives the processing result sent by the optimal target model, encapsulates the processing result according to the output format, and pushes it to the business application side, whereby the business application side presents the processing result.

[0014] By adopting the above technical solution, business applications can send business requests through a unified API entry point, eliminating the need to concern themselves with the specific access methods and details of each model in the large model cluster. This reduces the complexity of integrating business applications with the large model and improves development efficiency. The middleware engine can filter associated model sets based on business function point tags and select the optimal target model. Business applications do not need to manage and select suitable large models themselves; they can leverage the middleware engine's intelligent filtering capabilities to quickly find models that meet business needs, improving the flexibility and adaptability of business processing. The middleware engine encapsulates the processing results according to the output format specified by the business request. Business applications can directly present the processing results without additional format conversion and processing, reducing the development and maintenance costs of business applications. Complete isolation between the large model application interface and specific business application scenarios ensures that updates and iterations within the large model do not affect specific business application scenarios. Furthermore, local application capabilities that satisfy the same business function can better apply to the corresponding large model, thereby improving the efficiency and quality of business processing.

[0015] Optionally, the step of selecting the optimal target model from the set of associated models includes:

[0016] Monitor the real-time status of various large models in the associated model set;

[0017] Identify the application preferences and characteristics of the business application side;

[0018] Based on the real-time status, application preferences, and application characteristics, calculate the model scores for various large models in the associated model set;

[0019] The model with the highest score is selected as the optimal target model.

[0020] By adopting the above technical solutions and monitoring the real-time status of various large models in the associated model set, the middleware engine can avoid models with poor performance and select the model that can quickly respond to and process business requests, thereby greatly improving the timeliness of business processing and reducing user waiting time. Identifying application preferences on the business application side means that the middleware engine can select models based on the specific needs of the business application. Selecting models based on application characteristics allows for better adaptation between the model and the business application, optimizing business processing results. In summary, calculating model scores for various large models in the associated model set based on real-time status, application preferences, and application characteristics provides an objective and quantitative standard for model selection, avoiding the influence of human factors or subjective judgment. This ensures that the most suitable model for the current business needs is selected, thereby fully leveraging the model's advantages, improving the efficiency and accuracy of business processing, and avoiding resource waste caused by selecting an unsuitable model.

[0021] Optionally, the step of calculating the model scores of various large models in the associated model set based on the real-time status, the application preferences, and the application characteristics includes:

[0022] Obtain the real-time status score, the application preference score, and the application feature score;

[0023] Based on the historical success rate of various large models, a first weight is assigned to the real-time state;

[0024] A second weight is assigned to the application preference based on the historical call count of various large models;

[0025] A third weight is assigned to the application features based on the historical matching counts of various large models;

[0026] The model score is obtained based on the scoring model; the scoring model is the model score. .

[0027] By adopting the above technical solutions, the historical call success rate reflects the reliability of the model in past use. If a model has a high historical call success rate, it indicates that it performs stably in actual applications, and its real-time state should be given more importance when selecting a model now. Assigning appropriate weights to the real-time state can improve the reliability of business processing. The number of historical calls reflects the frequency of use of different models by the business application side. If a model is frequently called, it indicates that it better suits the preferences of the business application. Assigning appropriate weights to application preferences can make the selected model more aligned with the habits and needs of the business application, improving the business application's satisfaction with the processing results. The number of historical matches indicates the degree of fit between the model and the characteristics of the business application. A high number of matches means that the model performs well when handling specific business characteristics. Assigning appropriate weights to application features can ensure that the selected model can better leverage its advantages when processing business, improving the efficiency and quality of business processing. Evaluating the model from multiple dimensions avoids the limitations of single-factor evaluation. The real-time state reflects the current operating status of the model, application preferences reflect the subjective needs of the business application, and application features consider the objective characteristics of the business; combining these three aspects allows for a more comprehensive and accurate measurement of the model's applicability and performance.

[0028] Optionally, the steps after pushing to the business application side include:

[0029] Receive the performance score from the business application side;

[0030] Obtain the performance metrics of the model requested in this request;

[0031] Based on the effect score and the performance index, update the relevant weights of the model score corresponding to the model.

[0032] By adopting the above technical solution, the performance score on the business application side directly reflects the business application's satisfaction with the model's processing results, while the model's performance metrics reflect the model's actual performance in processing this business request. Combining these two factors to update the relevant weights of the model score allows the model score to better align with actual business scenarios and needs. Continuously adjusting the model score weights based on actual performance and results ensures that the selected model provides high-quality processing results; stable processing result quality helps improve the reliability of business applications and user satisfaction.

[0033] Optionally, the step of receiving the processing result sent by the optimal target model includes:

[0034] Determine whether the processing result sent by the optimal target model has been received within the set time period;

[0035] If not, the backup model switching strategy will be automatically triggered, and the second-best model will be selected from the associated model set in descending order of score to take over the processing, and the status of the faulty model will be recorded.

[0036] By adopting the above technical solution, when the optimal target model fails to return a processing result within the set time, it is likely that performance problems, failures, or high loads have occurred. Therefore, it is necessary to automatically trigger the backup model switching strategy to quickly select the second-best model from the associated model set to take over the processing, so as to ensure that the business process can continue, reduce the impact of model problems on the business, and ensure the normal operation of the business.

[0037] Optionally, the business processing method further includes:

[0038] Regularly calculate the confidence levels of various large models;

[0039] The large models with confidence levels below the first threshold are disabled, and a manual review process is triggered to update the registration information;

[0040] Large models with confidence levels below the second threshold are restricted from use, and a model optimization process is triggered; the second threshold is greater than the first threshold.

[0041] By adopting the above technical solutions and periodically calculating the confidence levels of various large models, it is possible to promptly identify those large models with confidence levels below the first threshold. Disabling these models and triggering a manual review process can prevent the use of unreliable models for business processing, thereby ensuring the accuracy of business processing results and reducing business risks. For large models with confidence levels below the second threshold but above the first threshold, usage can be restricted, and a model optimization process can be triggered. Even if the confidence level of these models decreases, they still have some usability. By limiting their usage and simultaneously optimizing them, model resources can be fully utilized while ensuring the basic reliability of business processing.

[0042] Optionally, the steps for calculating the confidence scores of various large models include:

[0043] Obtain historical call data for various large models within a preset time window. The historical call data includes at least: call success rate, average response time, processing result accuracy, error type and frequency, and resource consumption indicators.

[0044] Different evaluation dimensions and corresponding weight coefficients are set for each indicator in the historical call data. Among them, the weight coefficients of call success rate and processing result accuracy are higher than the weight coefficients of average response time and resource consumption indicators.

[0045] Each indicator is standardized and converted into a uniform confidence contribution score;

[0046] The confidence contribution scores of each indicator are weighted and summed according to their respective weight coefficients to obtain the preliminary confidence scores for each type of large model.

[0047] The initial confidence level is dynamically adjusted based on a time decay factor, whereby the time decay factor indicates that the influence of recent historical call data on the confidence level is greater than that of long-term data.

[0048] The adjusted confidence level is used as the final confidence level for each type of large model.

[0049] By adopting the above technical solution and collecting and analyzing historical call data across multiple dimensions and throughout the entire lifecycle, the overall performance and reliability of the large model can be comprehensively reflected, avoiding the bias caused by a single indicator. Differentiated weights are assigned to different indicators, highlighting the dominant role of core business needs (such as the accuracy of processing results and call success rate) in confidence assessment, while also considering efficiency (average response time) and cost (resource consumption) factors, making confidence calculation more aligned with actual business value orientation. The introduction of a time decay factor allows the confidence score to dynamically track the latest trends in model performance, promptly capturing performance fluctuations caused by iterative upgrades, data drift, or hardware aging, avoiding interference from outdated data in the current assessment. The final calculated confidence score provides an objective, dynamic, and business-specific quantitative basis for subsequent decisions on disabling, restricting, or optimizing models, helping the middleware engine continuously select high-quality, highly adaptable models, thereby ensuring the long-term stability of business processing efficiency, accuracy, and cost controllability.

[0050] Secondly, this application provides a business processing system based on a large model middleware engine, which adopts the following technical solution:

[0051] A business processing system based on a large model middleware engine includes:

[0052] On the business application side, it is used to send business requests and present processing results through a unified API entry point;

[0053] The middleware engine receives the business request, parses the business request, identifies business function point tags, business parameters and output format; filters the associated model set that matches the business function tags from the large model cluster; selects the optimal target model from the associated model set; reconstructs the business parameters according to the interface specification of the optimal target model, and sends a request to the optimal target model.

[0054] The large model cluster contains various large models that pre-register capability tags, service levels, and API access credentials with the middleware engine; the optimal target model in the large model cluster responds to the sent request, processes it, and sends the processing result.

[0055] The middleware engine is also used to receive the processing results sent by the optimal target model, encapsulate the processing results according to the output format, and push them to the business application side.

[0056] Thirdly, this application provides a terminal that adopts the following technical solution:

[0057] A terminal, comprising:

[0058] The memory stores business processing programs based on the large model middleware engine;

[0059] A processor is used to execute the program stored in the memory to implement the steps of the above-described business processing method based on a large model middleware engine.

[0060] In summary, this application has at least the following beneficial effects:

[0061] Business applications send business requests through a unified API entry point, eliminating the need to concern themselves with the specific access methods and details of each model in the large model cluster. This reduces the complexity of integrating business applications with the large model and improves development efficiency. The middleware engine can filter related model sets based on business function tags and select the optimal target model. Business applications no longer need to manage and select suitable large models themselves; they can leverage the middleware engine's intelligent filtering capabilities to quickly find models that meet business needs, improving the flexibility and adaptability of business processing. The middleware engine encapsulates the processing results according to the output format specified in the business request. Business applications can directly present the processing results without additional format conversion and processing, reducing the development and maintenance costs of business applications. Complete isolation between the large model application interface and specific business application scenarios ensures that updates and iterations within the large model do not affect specific business application scenarios. Furthermore, local application capabilities that satisfy the same business function can be applied to the corresponding large model with better results, thereby improving the efficiency and quality of business processing. Attached Figure Description

[0062] Figure 1 This is a first flowchart of an embodiment of the method of this application;

[0063] Figure 2 This is a second flowchart of an embodiment of the method of this application;

[0064] Figure 3 This is a third flowchart of an embodiment of the method of this application;

[0065] Figure 4 This is the fourth flowchart of an embodiment of the method of this application;

[0066] Figure 5This is the fifth flowchart of an embodiment of the method of this application;

[0067] Figure 6 This is the sixth flowchart of an embodiment of the method of this application;

[0068] Figure 7 This is a schematic diagram of information interaction in an embodiment of the system of this application. Detailed Implementation

[0069] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the appendices in the embodiments of the present invention will be described below. Figure 1 -Appendix Figure 6 The technical solutions in the embodiments of the present invention are clearly and completely described herein. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0070] The first embodiment of this application discloses a business processing method based on a large-model middleware engine, with the middleware engine as the execution entity. (See also...) Figure 1 The business processing method may include S110-S160:

[0071] S110 receives business requests sent by the business application side through a unified API entry point;

[0072] S120 parses business requests and identifies business function point tags, business parameters, and output formats;

[0073] S130: Select the associated model set that matches the business function tag from the large model cluster. Various large models in the large model cluster are pre-registered with capability tags, service levels and API access credentials in the middleware engine.

[0074] S140, Select the optimal target model from the set of related models;

[0075] S150, reconstruct the business parameters according to the interface specification of the optimal target model, and send a request to the optimal target model;

[0076] S160 receives the processing result sent by the optimal target model, encapsulates the processing result according to the output format, and pushes it to the business application side, where the business application side presents the processing result.

[0077] Specifically, as the core execution entity for business processing, the middleware engine needs to build a dynamically expandable function and interface management system during its initialization phase. First, a microservice architecture is adopted to design a registration mechanism for business application function points. A function point management interface is exposed through a RESTful API, allowing business applications to submit function point definitions (such as function name, unique label, input parameter JSON schema, output format template, etc.) in JSON format. The middleware engine stores this information in a "function point registry" in a distributed cache (such as Redis) and a relational database (such as MySQL). Simultaneously, it establishes an association index between function points and basic data (such as user profile data, industry knowledge base, etc.), and implements incremental updates of basic data through scheduled tasks (such as Quartz) or event-triggered mechanisms (such as Kafka message queues). For loading large model API interface data, the middleware engine provides a unified model registration portal, requiring the access party to submit model metadata (capability tags such as "text generation" and "image recognition", service level SLA such as response timeout, API access credentials such as API Key, interface documentation URL, etc.), and to verify connectivity by calling the model provider's health check interface (such as / health) via HTTPS protocol. After successful verification, the interface data is stored in the "model registry", and a multi-dimensional index (such as a tag tree structure) is built based on capability tags to support subsequent fast filtering.

[0078] In addition, the middleware engine needs to initialize the association model between business applications and large models, and build a matching rule base by training historical interaction data (such as business function call records and model response results). For example, a collaborative filtering algorithm can be used to generate a "function point-model" recommendation matrix, which is stored in a graph database (such as Neo4j) to improve the efficiency of association queries.

[0079] In steps S110 and S120, the business application sends requests through a unified API gateway (such as Spring Cloud Gateway) provided by the middleware engine. The gateway layer first performs identity authentication (such as OAuth 2.0-based token verification) and traffic control (such as token bucket algorithm rate limiting). After successful authentication, the request is routed to the business processing module. The request format must follow predefined specifications, such as using a JSON structure, including fields such as appId (application identifier), functionTag (business function tag), businessParams (business parameters), and outputFormat (output format requirements such as "JSON" or "HTML").

[0080] After receiving the request, the middleware engine calls the parser module to perform word segmentation and semantic matching on the functionTag (such as text classification based on the BERT model), queries the corresponding input parameter specifications from the "Function Point Registry", and verifies the completeness and format correctness of businessParams through a JSONSchema validation tool (such as Ajv) (such as whether required fields are missing and whether the data types match). If the validation fails, it returns a standardized error code (such as PARAM_ERROR) and detailed prompts.

[0081] Meanwhile, the parser extracts the outputFormat field, associates it with the output template in the "Function Point Registry", provides a format basis for subsequent result encapsulation, and stores the parsed key information (function point label, validated business parameters, output format) in the request context (such as ThreadLocal) for easy calling in subsequent steps.

[0082] Reference Figure 2 S140, the steps for selecting the optimal target model from the set of associated models include S210-S240:

[0083] S210 monitors the real-time status of various large models in the associated model set;

[0084] S220 identifies application preferences and characteristics on the business application side;

[0085] S230, calculates the model scores of various large models in the associated model set based on real-time status, application preferences and application characteristics;

[0086] S240: Determine the largest model with the highest model score as the optimal target model.

[0087] Reference Figure 3 S230, the steps for calculating the model scores of various large models in the associated model set based on real-time status, application preferences, and application characteristics include S310-S350:

[0088] S310, obtain the real-time status score, the application preference score, and the application feature score;

[0089] S320 assigns the first weight to the real-time status based on the historical success rate of various large models.

[0090] S330 assigns a second weight to application preferences based on the historical call count of various large models;

[0091] S340 assigns a third weight to the application features based on the historical matching counts of various large models;

[0092] S350, based on the scoring model, obtain the model score; the scoring model is the model score. .

[0093] Specifically, in steps S130 and S140, based on the parsed business function point tags, the middleware engine filters the set of related models from the "model registry". In practice, a tag matching algorithm is used. First, models with completely consistent capability tags are filtered out through exact matching (e.g., when the function point tag is "legal document generation", models with tags including "law" and "text generation" are matched). Then, semantically related models are supplemented through fuzzy matching (e.g., cosine similarity calculation) to generate a candidate model list.

[0094] Subsequently, the middleware engine calls the model status monitoring module to evaluate model availability in real time through multi-dimensional metrics: monitoring network connectivity by periodically sending PING requests (e.g., once per second) and recording response times; calling the model provider's monitoring interface (e.g., the `model_inference_latency_seconds` metric exposed by Prometheus) to obtain performance data such as real-time inference latency and GPU utilization; and parsing the error codes returned by the model (e.g., 503 Service Unavailable) to calculate the anomaly rate. These metrics are collected via Telegraf and stored in a time-series database (e.g., InfluxDB). The monitoring module calculates a status score every 500 milliseconds (e.g., using a 0-100 scale, with connectivity accounting for 40%, latency for 30%, and anomaly rate for 30%) and updates the score to the "model status table".

[0095] To identify application preferences on the business application side, the middleware engine extracts historical call records for the appId from the "Application Call Log Table" (such as model IDs, call counts, and average scores called in the past 30 days), and calculates preference scores using a weighted average method: for example, higher scores are assigned to the top 3 models in terms of call count (such as 100 points for the 1st place, 80 points for the 2nd place, and 60 points for the 3rd place), and scores are assigned to the remaining models according to the proportion of call counts.

[0096] Application feature recognition is based on application metadata associated with appId (such as industry type "finance" "healthcare", user scale "enterprise" "personal") and business parameter features (such as input text length, whether it contains sensitive information). The application features are classified into preset categories (such as "high concurrency" "high precision") through decision tree algorithms (such as XGBoost), and feature scores are generated according to feature matching degree (such as the model score for matching "finance industry" features increases by 20 points).

[0097] Regarding weight calculation, the first weight (real-time status) is determined based on the historical call success rate of the large model (e.g., the percentage of successful calls in the past 1000 calls). A higher success rate results in a larger weight (e.g., a 99% success rate corresponds to a weight of 0.6, and 80% corresponds to 0.3). This weight is dynamically adjusted using a linear function, which is: .

[0098] The second weight (application preference) is based on the number of historical calls and is processed using log normalization (e.g., second weight = log(number of calls + 1) / max_log, where max_log is the maximum log value of the number of calls for all models).

[0099] The third weight (application feature) is calculated based on the number of historical matches (such as the number of successful matches between the model and the application feature). The formula is: third weight = number of matches / total number of matches. Ensure that the sum of the three weights is 1 (e.g., through normalization, weight = weight / (first weight + second weight + third weight)).

[0100] The scoring model employs a linear weighted summation formula. The middleware engine inputs the state score, first weight, preference score, second weight, feature score, and third weight into a predefined scoring function: Model Scoring. For example, if a model has a state score of 80 and a first weight of 0.4, a preference score of 90 and a second weight of 0.3, and a feature score of 85 and a third weight of 0.3, then... .

[0101] To improve computational efficiency, the scoring model is deployed in an in-memory computing framework (such as Spark Streaming). It computes scores in parallel for each model in the associated model set, sorts the results in descending order, and selects the model with the highest score as the optimal target model. If there are models with the same score, a secondary sort is performed by comparing historical average response times (shorter models take precedence) or service levels (higher SLA levels take precedence) to ensure the selection of a unique optimal model.

[0102] In steps S150 and S160, after determining the optimal target model, the middleware engine calls the interface adaptation module to reconstruct the business parameters according to the interface specifications (such as the request method POST, parameter position body, and data format XML or JSON) stored in the "model registry". For example, if the optimal model requires the parameters to be {"prompt": "...", "temperature":0.7}, while the business parameters are {"question": "...", "creativity": "medium"}, then the question field is mapped to prompt according to predefined mapping rules (stored in the "parameter mapping table"), and the creativity field is converted to temperature according to the enumerated value ("medium" corresponds to 0.7). During the conversion process, the parameter values ​​are dynamically filled using a template engine (such as FreeMarker), and the format of the reconstructed parameters is validated again using JSON Schema.

[0103] Subsequently, the middleware engine sends a request to the optimal model via an HTTP client (such as OkHttp), sets a timeout (based on the SLA during model registration, such as 5 seconds), and enables asynchronous callback listening (such as based on CompletableFuture). If the processing result is received within the set time, the result encapsulation module is invoked to render the result according to the output format required by the business application (such as an HTML template). For example, the Thymeleaf template engine is used to populate the JSON result returned by the model into an HTML table, and application-side style tags (such as CSS class names) are added. Finally, the encapsulated result is pushed to the business application via WebSocket or HTTP response, and the business application parses and renders it using a front-end framework (such as Vue.js).

[0104] If the processing result of the optimal target model is not received within the set time period (e.g., request timeout, connection rejection), the middleware engine automatically triggers the backup model switching strategy. First, it reads the model list sorted in descending order of score from the associated model set, excludes the currently faulty model, selects the second-best model, repeats the request reconstruction and sending process, and records the status of the faulty model (fault type, occurrence time, error stack) through a logging framework (e.g., Logback), writes the fault information to the "Model Fault Table", and triggers an alarm notification, such as sending it to the operations and maintenance personnel via email.

[0105] Once the backup model returns the processing result, it is also formatted and pushed to the business application side. In addition, the middleware engine includes a performance rating entry point when pushing the result (e.g., adding a feedbackUrl field to the response). The business application side can submit a rating (e.g., 1-5 stars) and evaluation text for the processing result through this URL. The middleware engine stores the rating data in the "Performance Rating Table", associating it with information such as the model ID and function tag of this request, providing a basis for subsequent weight updates.

[0106] Reference Figure 4 The steps after pushing to the business application side include S410-S430:

[0107] S410, receives performance scores from the service application side;

[0108] S420, retrieve the performance metrics of the model requested in this request;

[0109] S430 updates the relevant weights of the model score based on the effect score and performance indicators.

[0110] Specifically, the middleware engine periodically (e.g., daily at midnight) initiates a weight update task. It retrieves the performance score (e.g., average score_avg) from the "Performance Score Table" for the business application side and the model performance metrics (e.g., response time latency, resource consumption gpu_usage) for the current request from the time-series database. First, the performance score and performance metrics are standardized (e.g., the score is mapped to 0-100, and the response time is normalized to 0-100 using the maximum value). Then, a multiple linear regression model is used to analyze the correlation between the score and each metric (e.g., the correlation coefficient between the performance score and the state score), dynamically adjusting the first, second, and third weights. For example, if the correlation between a model's performance score and application preference significantly increases (correlation coefficient rises from 0.3 to 0.5), the range of the second weight is increased (e.g., from 0.2-0.4 to 0.3-0.5). The updated weights are stored in the "Weight Configuration Table" and take effect in the next model score calculation.

[0111] Reference Figure 5 The business processing methods also include S510-S530:

[0112] S510 periodically calculates the confidence level of various large models;

[0113] S520 disables large models with confidence levels below the first threshold and triggers a manual review process to update registration information;

[0114] S530 restricts the use of large models with confidence levels below the second threshold and triggers the model optimization process; the second threshold is greater than the first threshold.

[0115] Reference Figure 6 In S510, the steps for calculating the confidence scores of various large models include S610-S660:

[0116] S610, obtains historical call data of various large models within a preset time window;

[0117] S620 sets different evaluation dimensions and corresponding weight coefficients for various indicators in historical data;

[0118] S630 standardizes each indicator and converts them into a unified confidence contribution score.

[0119] S640: The confidence contribution scores of each indicator are weighted and summed according to their weight coefficients to obtain the preliminary confidence scores of various large models.

[0120] S650 dynamically adjusts the initial confidence level based on the time decay factor. The time decay factor indicates that the influence of recent historical call data on the confidence level is higher than that of long-term data.

[0121] S660 uses the adjusted confidence level as the final confidence level for each type of large model.

[0122] Specifically, historical call data includes at least: call success rate, average response time, processing result accuracy, error type and frequency, and resource consumption indicators; among them, the weighting coefficients of call success rate and processing result accuracy are higher than the weighting coefficients of average response time and resource consumption indicators.

[0123] To ensure the reliability of large models, the middleware engine performs a confidence assessment process periodically (e.g., hourly). First, it extracts historical call data for a preset time window (e.g., the past 7 days) from the "Model Call Log Table" and "Effect Score Table" using data warehouse tools (e.g., Hive). This includes call success rate (number of successful calls / total number of calls), average response time (total response time / number of successful calls), processing result accuracy (number of manually labeled correct results / total number of labels), error type and frequency (e.g., percentage of 4xx errors, percentage of 5xx errors), and resource consumption metrics (e.g., average GPU memory usage).

[0124] Subsequently, evaluation dimensions and weighting coefficients were set for each indicator: call success rate (weight 0.3) and processing result accuracy (weight 0.3) were used as core indicators, average response time (weight 0.2) and resource consumption indicators (weight 0.1) were used as secondary indicators, and error type and frequency (weight 0.1) were used as auxiliary indicators. Each indicator was standardized; for example, call success rate was directly taken as its original value (0-1), average response time was converted to a positive score (higher values ​​indicate better) by subtracting 1 from (time / maximum time), processing result accuracy was taken as the manually labeled accuracy rate (0-1), and serious errors were assigned a higher negative weight within the error types.

[0125] The standardized indicators are weighted and summed with weighted coefficients to obtain the initial confidence level. Then, a time decay factor is introduced to reduce the weight of long-term data (e.g., 7 days ago) and increase the weight of recent data (e.g., within 1 day). This adjustment yields the final confidence level. The time decay factor can be an exponential decay function. t is the attenuation coefficient, t_now is the current time point for calculating the confidence level, and t is the actual time when the historical data occurred.

[0126] If the final confidence level is lower than the first threshold (e.g., 0.4), the middleware engine updates the model status to "disabled," removes it from the associated model set, and triggers a manual review process through the workflow engine (e.g., Activiti), sending a pending task to the model administrator (e.g., generating a work order in the OA system). The administrator needs to check the model registration information (e.g., whether the API credentials have expired) and update it. If the confidence level is lower than the second threshold (e.g., 0.6, higher than the first threshold), the model status is updated to "restricted" (e.g., limiting the number of daily calls or allowing only non-core business calls), and the model optimization process is triggered. The optimization interface of the model provider is called via API (e.g., / retrain) or an optimization suggestion report is sent (based on historical error types, such as "suggesting to reduce the temperature parameter to improve the accuracy of the results") to promote model iteration.

[0127] Based on the above method embodiments, the second embodiment of this application discloses a business processing system based on a large model middleware engine. The business processing system based on a large model middleware engine in this embodiment can implement any of the above-described business processing methods based on a large model middleware engine, and the specific working process of each module in the business processing system based on a large model middleware engine can be referred to the corresponding process in the above method embodiments.

[0128] For ease of understanding, the following example is provided: (Refer to...) Figure 7 A business processing system based on a large model middleware engine, comprising:

[0129] On the business application side, it is used to send business requests and present processing results through a unified API entry point;

[0130] The middleware engine receives business requests, parses them, identifies business function point tags, business parameters, and output formats; filters the associated model set that matches the business function tags from the large model cluster; selects the optimal target model from the associated model set; reconstructs the business parameters according to the interface specification of the optimal target model; and sends a request to the optimal target model.

[0131] The large model cluster has various large models pre-registered with the middleware engine for capability tags, service levels, and API access credentials; the optimal target model in the large model cluster responds to the sent request, processes it, and sends the processing result.

[0132] The middleware engine is also used to receive the processing results sent by the optimal target model, encapsulate the processing results according to the output format, and push them to the business application side.

[0133] A third embodiment of this application provides a terminal. As one implementation of this terminal, the terminal may include: a memory and a processor; wherein...

[0134] The memory is used to store business processing programs based on the large model middleware engine;

[0135] The processor is used to execute programs stored in memory to implement the steps of the business processing method based on the large model middleware engine described above.

[0136] The memory can communicate with the processor via a communication bus, which can be an address bus, a data bus, a control bus, etc.

[0137] Additionally, the memory may include random access memory (RAM) or non-volatile memory (NVM), such as at least one disk storage device.

[0138] Furthermore, the processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc.

[0139] The above are all preferred embodiments of this application and are not intended to limit the scope of protection of this application. Any feature disclosed in this specification (including the abstract and drawings) may be replaced by other equivalent or similar features unless specifically stated otherwise. That is, unless specifically stated otherwise, each feature is only one example of a series of equivalent or similar features.

Claims

1. A business processing method based on a large model middleware engine, characterized in that, Using a middleware engine as the execution entity, including: Receive business requests sent by the business application side through a unified API entry point; The business request is parsed to identify business function point tags, business parameters, and output format; The associated model set that matches the business function point tag is selected from the large model cluster. The various large models in the large model cluster have their capability tags, service levels and API access credentials pre-registered in the middleware engine. Select the optimal target model from the set of associated models; The business parameters are reconstructed according to the interface specification of the optimal target model, and a request is sent to the optimal target model; The system receives the processing result sent by the optimal target model, encapsulates the processing result according to the output format, and pushes it to the business application side, whereby the business application side presents the processing result. The steps for selecting the optimal target model from the set of associated models include: Monitor the real-time status of various large models in the associated model set; Identify the application preferences and characteristics of the business application side; Obtain the real-time status score, the application preference score, and the application feature score; Based on the historical success rate of various large models, a first weight is assigned to the real-time state; A second weight is assigned to the application preference based on the historical call count of various large models; A third weight is assigned to the application features based on the historical matching counts of various large models; The model score is obtained based on the scoring model; the scoring model is: Model Score = First Weight x State Score + Second Weight x Preference Score + Third Weight x Feature Score; The model with the highest model score is selected as the optimal target model. The steps after pushing to the business application side include: Receive the performance score from the business application side; Obtain the performance metrics of the model requested in this request; Update the relevant weights of the model score corresponding to the model based on the effect score and the performance index; The business processing method also includes: Regularly calculate the confidence levels of various large models; The large models with confidence levels below the first threshold are disabled, and a manual review process is triggered to update the registration information; Large models with confidence levels below the second threshold are restricted from use, and a model optimization process is triggered; the second threshold is greater than the first threshold.

2. The business processing method based on a large model middleware engine according to claim 1, characterized in that, The steps for receiving the processing results sent by the optimal target model include: Determine whether the processing result sent by the optimal target model has been received within the set time period; If not, the backup model switching strategy will be automatically triggered, and the second-best model will be selected from the associated model set in descending order of score to take over the processing, and the status of the faulty model will be recorded.

3. The business processing method based on a large model middleware engine according to claim 1, characterized in that, The steps for calculating the confidence scores of various large models include: Obtain historical call data for various large models within a preset time window. The historical call data includes at least: call success rate, average response time, processing result accuracy, error type and frequency, and resource consumption indicators. Different evaluation dimensions and corresponding weight coefficients are set for each indicator in the historical call data. Among them, the weight coefficients of call success rate and processing result accuracy are higher than the weight coefficients of average response time and resource consumption indicators. Each indicator is standardized and converted into a uniform confidence contribution score; The confidence contribution scores of each indicator are weighted and summed according to their respective weight coefficients to obtain the preliminary confidence scores for each type of large model. The initial confidence level is dynamically adjusted based on a time decay factor, whereby the time decay factor indicates that the influence of recent historical call data on the confidence level is greater than that of long-term data. The adjusted confidence level is used as the final confidence level for each type of large model.

4. A business processing system based on a large model middleware engine, characterized in that, The method for performing business processing based on a large model middleware engine as described in any one of claims 1-3 includes: On the business application side, it is used to send business requests and present processing results through a unified API entry point; The middleware engine receives the business request, parses the business request, identifies business function point tags, business parameters, and output format; filters the associated model set that matches the business function point tags from the large model cluster; selects the optimal target model from the associated model set; reconstructs the business parameters according to the interface specification of the optimal target model, and sends a request to the optimal target model. The large model cluster contains various large models that pre-register capability tags, service levels, and API access credentials with the middleware engine; the optimal target model in the large model cluster responds to the sent request, processes it, and sends the processing result. The middleware engine is also used to receive the processing results sent by the optimal target model, encapsulate the processing results according to the output format, and push them to the business application side.

5. A terminal, characterized in that, include: The memory stores business processing programs based on the large model middleware engine; A processor for executing a program stored in the memory to implement the steps of the business processing method based on a large model middleware engine as described in any one of claims 1-3.

Images