Multi-model service intelligent scheduling optimization method and system based on edge computing

Abstract

The invention discloses a multi-model service intelligent scheduling optimization method and system based on edge computing, and relates to the technical field of data processing. The method comprises the following steps: carrying out API (Application Program Interface) connection through edge calculation, and constructing a unified access platform; configuring a multi-model scheduling strategy group; performing model scheduling detection based on the unified access platform to generate a scheduling detection result; generating a switching instruction according to the scheduling detection result, activating the multi-model scheduling strategy group to perform switching control, and determining multiple pieces of model service data; and generating a gateway access key, packaging the plurality of model service data according to the gateway access key through the unified access platform, and generating multi-model response data. The technical problems of inflexible model service scheduling, low response speed and low resource utilization efficiency in the prior art are solved, and the technical effects of improving the model service scheduling flexibility, accelerating the response speed and optimizing the resource utilization efficiency are achieved.

Description

Technical Field

[0001] The present invention relates to the technical field of data processing, and particularly to an intelligent scheduling optimization method and system for multi-model services based on edge computing. Background Art

[0002] With the rapid development of technologies such as the Internet of Things, big data, and cloud computing, the generation and transmission of massive data have become the norm, which poses higher requirements for the scheduling efficiency, response speed, and resource utilization ability of model services. Model services, as key components in intelligent applications, are responsible for performing intelligent tasks such as prediction, classification, and recognition based on input data, and their performance is directly related to the stability of the entire intelligent system and the user experience. However, most traditional model service scheduling methods adopt a centralized architecture, where data needs to be transmitted from various terminals or sensors to the central server for processing and then the results are returned. This architecture not only increases data transmission latency, reduces the system response speed, but also faces problems such as uneven resource allocation and inflexible scheduling. Especially when facing large-scale and high-concurrency data processing requirements, the centralized architecture often appears inadequate and cannot meet the real-time, efficient, and intelligent service needs. Summary of the Invention

[0003] The present application provides an intelligent scheduling optimization method and system for multi-model services based on edge computing, which solves the technical problems of inflexible model service scheduling, slow response speed, and low resource utilization efficiency in the prior art.

[0004] In the first aspect of the present application, there is provided an intelligent scheduling optimization method for multi-model services based on edge computing, the method comprising: Performing API connection through edge computing to construct a unified access platform; configuring a multi-model scheduling policy group; performing model scheduling detection based on the unified access platform to generate a scheduling detection result; generating a switching instruction according to the scheduling detection result, activating the multi-model scheduling policy group for switching control to determine multiple model service data; generating a gateway access key, and encapsulating the multiple model service data according to the gateway access key through the unified access platform to generate multi-model response data.

[0005] In the second aspect of the present application, there is provided an intelligent scheduling optimization system for multi-model services based on edge computing, the system comprising: A connection module for API connection through edge computing to build a unified access platform; a configuration module for configuring a multi-model scheduling policy group; a detection module for performing model scheduling detection based on the unified access platform to generate a scheduling detection result; a control module for generating a switching instruction according to the scheduling detection result, activating the multi-model scheduling policy group for switching control to determine multiple model service data; a packaging module for generating a gateway access key and encapsulating the multiple model service data according to the gateway access key through the unified access platform to generate multi-model response data.

[0006] One or more technical solutions provided in this application have at least the following technical effects or advantages: First, perform API connection through edge computing to build a unified access platform. Then, configure a multi-model scheduling policy group. Next, perform model scheduling detection based on the unified access platform to generate a scheduling detection result. Further, generate a switching instruction according to the scheduling detection result, activate the multi-model scheduling policy group for switching control to determine multiple model service data. Finally, generate a gateway access key and encapsulate the multiple model service data according to the gateway access key through the unified access platform to generate multi-model response data. This solves the technical problems of inflexible model service scheduling, slow response speed, and low resource utilization efficiency in the prior art, and achieves the technical effects of improving the flexibility of model service scheduling, accelerating the response speed, and optimizing the resource utilization efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0007] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the following drawings are only some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0008] Figure 1 Schematic flowchart of the intelligent scheduling optimization method for multi-model services based on edge computing provided by the embodiment of the present application; Figure 2 Schematic structural diagram of the intelligent scheduling optimization system for multi-model services based on edge computing provided by the embodiment of the present application.

[0009] Description of the reference numerals: connection module 11, configuration module 12, detection module 13, control module 14, packaging module 15. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0010] This application provides an intelligent scheduling optimization method and system for multi-model services based on edge computing, which solves the technical problems of inflexible model service scheduling, slow response speed, and low resource utilization efficiency in the prior art.

[0011] Next, the technical solutions in the embodiments of the present application will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments in the present application without creative efforts shall fall within the protection scope of the present application.

[0012] It should be noted that the terms "include" and "have" are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or server that includes a series of steps or units does not necessarily have to be limited to those steps or units clearly listed, but may include other steps or modules that are not clearly listed or are inherent to these processes, methods, products, or devices.

[0013] Embodiment 1, as Figure 1 shown, the present application provides an intelligent scheduling optimization method for multi-model services based on edge computing. Among them, the method includes: Conduct API connection through edge computing to build a unified access platform.

[0014] Connect the model services on each edge node through the API (Application Programming Interface) to build a unified access platform, so that users or upper-layer applications can access and call different model services through this platform without caring about the underlying complex network architecture and model distribution.

[0015] Furthermore, conducting API connection through edge computing to build a unified access platform, the method includes: Unify the input parameter structures of multiple model services, deploy a dynamic parameter mapping engine; traverse the API endpoints for data storage to build a service description meta-database; create a protocol conversion matrix, and perform protocol compatibility detection on the dynamic parameter mapping engine and the service description meta-database according to the protocol conversion matrix to build a multi-protocol API adaptation layer; dynamically access the API interfaces of multiple heterogeneous model service providers by the multi-protocol API adaptation layer to form the unified access platform.

[0016] The system uniformly converts the input parameter structures of multiple model services. For the input formats, data types, and parameter requirements of different models, a dynamic parameter mapping engine is deployed to achieve standardized processing of parameters, ensuring that each model can run compatibly under a unified call framework. The dynamic parameter mapping engine extracts the field names, data types, format requirements, and default value information of input parameters by parsing the API documentation, interface descriptions, or sample requests of model services, and converts the input parameters of different model services according to predefined parameter mapping rules to make them conform to the standardized input format. For parameters that require additional processing, such as missing value filling, data unit conversion, or format transformation, the dynamic parameter mapping engine automatically performs adaptation through a built-in rule library to ensure that all model services can receive request data according to a unified interface specification.

[0017] After completing the parameter standardization process, the system automatically scans all connected model services, accesses their public API endpoints one by one, and collects the basic information of the APIs, including but not limited to interface addresses, request methods (such as GET, POST), input parameters, return data structures, authentication methods, and response times, etc.; the collected API information is stored in the service description meta-database after being structured. The database contains a complete model service information table for subsequent dynamic calls, load balancing, and compatibility detection.

[0018] To adapt to the communication protocols used by different APIs, the system further creates a protocol conversion matrix. This matrix contains mapping rules for common communication protocols such as RESTful API, gRPC, WebSocket, etc., and defines the conversion methods between various protocols. During the operation of the system, the protocol conversion matrix automatically detects the communication protocol type of each model service API through a protocol parsing module, and matches the dynamic parameter mapping engine with the service description meta-database according to a preset protocol compatibility detection process to ensure that all API calls conform to the unified standards of the platform. On this basis, the system constructs a multi-protocol API adaptation layer based on the protocol conversion matrix. This adaptation layer integrates protocol parsing, format conversion, request forwarding, and response parsing functions, can automatically identify the API communication method, and perform transparent conversion between different protocols, enabling edge computing nodes to seamlessly connect to various heterogeneous model services without additional modification or encapsulation of each API. Finally, the edge computing node realizes dynamic access to the API interfaces of multiple heterogeneous model service providers through the multi-protocol API adaptation layer, automatically adapts to the communication protocols and data formats of different models, thus forming a unified access platform.

[0019] Configure a multi-model scheduling policy group.

[0020] On top of the unified access platform, multiple model scheduling policy groups are configured. These model scheduling policy groups have predefined different scheduling rules based on factors such as business requirements, resource status, and model performance. They can select the optimal model for service according to indicators such as the real-time load, prediction accuracy, and response time of the model.

[0021] Furthermore, the method for configuring the multi-model scheduling policy group includes: Traverse the edge computing nodes for real-time monitoring to obtain multiple real-time data, and dynamically generate a model scheduling priority weight matrix based on the multiple real-time data; verify the logical consistency between multiple policies through a rule engine and establish a policy conflict detection mechanism; execute multiple model service policies according to the model scheduling priority weight matrix to generate policy execution results; analyze the policy execution results according to the policy conflict detection mechanism to generate optimized scheduling parameters; optimize the multiple model service policies based on the optimized scheduling parameters to determine the multi-model scheduling policy group.

[0022] The system traverses each model service deployed on the edge computing nodes and conducts real-time monitoring on them, collecting multiple key performance indicators including service response time, computing load, data throughput, task queue length, prediction accuracy, etc. At the same time, combined with external environmental factors (such as network latency, available computing power, etc.), multiple real-time data are formed.

[0023] Based on the collected real-time data, the system dynamically generates a model scheduling priority weight matrix using a multi-dimensional decision-making method. First, weights are assigned to different performance indicators, and the indicator weight ratio is adjusted according to business requirements. For example, in latency-sensitive applications, the weight of the response time will increase, while in high-precision tasks, the weight of prediction accuracy will account for a larger proportion. Then, the system uses a weighted scoring mechanism to calculate the comprehensive score of each model, generates a priority ranking according to the scores, and constructs a model scheduling priority weight matrix. This matrix can reflect the relative priorities of different models in the current system state and provide a decision-making basis for subsequent intelligent scheduling. On this basis, the system verifies the logical consistency of multiple scheduling policies through a rule engine and establishes a policy conflict detection mechanism. The rule engine presets a series of scheduling rules and constraints, such as: avoiding the repeated execution of the same task on multiple models, ensuring the minimum resource guarantee for critical tasks, and avoiding low-priority tasks from occupying high-priority computing resources. The system uses a conflict detection algorithm based on graph analysis to analyze the relationships between different scheduling policies and identify potential policy conflicts, such as resource competition, circular dependencies, policy coverage, etc. When a conflict is detected, the system can automatically adjust the model weights and modify the policy execution order to ensure the coordinated execution of multi-model scheduling policies.

[0024] Next, the system executes multiple model service policies according to the model scheduling priority weight matrix and distributes computing tasks among different models through a distributed task scheduling mechanism. The execution results of each scheduling policy are recorded and monitored by a distributed tracing system to form multiple policy execution paths. Based on the execution path data, the system evaluates the scheduling effect of each model, calculates the changes in key performance indicators, and generates policy execution results. Subsequently, based on the policy conflict detection mechanism, the system deeply analyzes the policy execution results and calculates optimized scheduling parameters in combination with the actual operation situation. For example, when certain policies lead to excessive computing loads, the system can dynamically adjust the task allocation ratio to appropriately divert the computing tasks of high-load models to low-load models; when the prediction accuracy of certain models is lower than expected, the system can adjust the model combination method, increase multi-model fusion calculations, and improve the overall inference accuracy. Finally, based on the optimized scheduling parameters, the system dynamically adjusts multiple model service policies and determines the optimal multi-model scheduling policy group, which can flexibly adjust the scheduling method according to real-time business requirements, including load balancing mode, performance priority mode, resource saving mode, etc., to adapt to different application scenarios. For example, in a high-concurrency scenario, the system can automatically enable the load balancing policy and preferentially select the model with the shortest response time for inference calculation; in a low-power consumption scenario, the system can preferentially select models with lower energy consumption and reduce the invocation frequency of high-energy consumption models.

[0025] Furthermore, to execute multiple model service policies according to the model scheduling priority weight matrix and generate policy execution results, the method includes: Performing distributed tracing on the execution of the multiple model service policies based on the model scheduling priority weight matrix to obtain multiple policy execution paths; conducting policy evaluation according to the multiple policy execution paths to generate multiple policy effect information; and performing feedback control based on the multiple policy effect information to generate the policy execution results.

[0026] Based on the model scheduling priority weight matrix, the system distributes computing tasks and executes multiple model service policies according to the execution priorities of different models. During the execution process, the system uses distributed tracing technology to perform full-link tracking on the execution of tasks in different model services, that is, assigns a unique identifier to each task and records the complete path of the task from request initiation to result return. The system collects key information during the task flow, including the time when the task enters the model queue, computing duration, task execution status, data flow path, resource occupancy, etc., to form multiple policy execution paths. These paths can not only reflect the specific execution situation of tasks in each model but also reveal the impact of different scheduling policies on computing resources.

[0027] After obtaining multiple policy execution paths, the system conducts policy evaluation on these paths to analyze the execution effects of different scheduling policies. During the evaluation process, key metrics such as policy hit rate, policy switching latency, and policy execution exception rate are mainly concerned. Among them, the policy hit rate is used to measure the matching degree between the actually executed scheduling policy and the preset policy, and its calculation method is the ratio of the number of successfully matched policies to the total number of executed policies. The higher the hit rate, the stronger the scheduling stability of the system. Otherwise, it may be necessary to adjust the model weights or optimize the policy adaptation logic. The policy switching latency measures the time required to switch to the backup model after the main model experiences an exception or load change. This latency is determined by the time difference between the failure of the main model and the stable operation of the backup model. A higher switching latency may lead to service interruption. Therefore, the performance can be optimized by asynchronously loading the backup model or using a pre-judgment mechanism to switch in advance. The policy execution exception rate measures the frequency of exceptions triggered during the actual operation of the policy, including issues such as policy conflicts, execution failures, or execution deviations. Its calculation method is the ratio of the number of exception triggers to the total number of executed policies. If the exception rate is relatively high, it may indicate resource competition, model response failure, or the need to optimize the policy execution logic in the system. The probability of exceptions can be reduced by detecting policy conflicts, optimizing task allocation, or introducing a multi-level fallback mechanism. By comprehensively evaluating the policy hit rate, policy switching latency, and policy execution exception rate, multiple policy effect information is generated.

[0028] Based on multiple policy effect information, feedback control is performed to optimize the scheduling policy and generate the final policy execution result. After the system executes the scheduling policy, it will track and evaluate the policy execution path, and perform dynamic adjustment based on core metrics such as policy hit rate, policy switching latency, and policy execution exception rate. Among them, the policy execution exception rate is used to measure the frequency of exceptions occurring during the policy operation. When this indicator exceeds 5%, the system will automatically trigger the policy rollback mechanism to ensure scheduling stability. The policy rollback mechanism detects the execution path of the abnormal policy, determines the specific cause of the exception trigger, and selects the optimal alternative policy for replacement in combination with historical data to ensure the reliability and continuity of the scheduling logic. During the process of triggering the rollback, the system first analyzes the exception category based on the exception log to determine whether it is caused by model service failure, resource competition, or policy conflict, and then selects the appropriate rollback method according to the exception type, such as switching to the stable policy of the previous version, recalculating the weights to adjust the scheduling priority, or directly enabling the standby model for service switching.

[0029] Based on the unified access platform, model scheduling detection is performed to generate a scheduling detection result.

[0030] Through the unified access platform, scheduling detection of the current model service is carried out, including monitoring the running status, performance metrics, and user feedback of each model to obtain real-time scheduling information. Based on this information, a scheduling detection result is generated to provide a basis for subsequent switching decisions.

[0031] Furthermore, based on the unified access platform, model scheduling detection is performed to generate a scheduling detection result. The method includes: Traverse the multiple model service policies to determine the primary model service data; extract the running characteristics of the primary model service data through a sliding time window to determine the dynamic running data set; perform anomaly recognition based on the dynamic running data set, and perform anomaly marking according to the recognition result to determine the abnormal scheduling event; perform confidence weighting based on the abnormal scheduling event to generate the scheduling detection result.

[0032] First, the system traverses multiple model service policies to determine the primary model service data currently being executed, including core information such as the input parameters, calculation status, output results, and running logs of the model. Then, the sliding time window technique is used to extract the running characteristics of the primary model service data, and key running parameters are extracted from historical and real-time data, such as model response time, prediction error, calculation load, and resource consumption. Based on these parameters, a dynamic running data set is constructed, which can reflect the long-term running trend and short-term state changes of the model, providing accurate data support for anomaly detection. Subsequently, the system performs anomaly recognition based on the dynamic running data set, using statistical analysis, rule matching, or machine learning methods to detect possible abnormal situations, such as model response timeouts, excessive calculation result deviations, abnormal resource occupancy, or frequent error logs. For the detected anomalies, the system classifies and marks them, determines the abnormal scheduling event, and analyzes it in combination with the number of occurrences, impact range, and historical records of the anomaly. Next, the system uses the confidence weighting method to evaluate based on the abnormal scheduling event. By calculating the confidence level of the anomaly, the severity and possible impact range of the anomaly are judged, so as to decide whether model switching or scheduling optimization is required. Finally, the system comprehensively analyzes multiple anomaly indicators to generate a scheduling detection result, and uses this result to optimize the model scheduling strategy, such as adjusting the model weights, switching to a standby model, or optimizing the calculation resource allocation, to ensure the continuous and stable operation and intelligent optimization scheduling of the multi-model service.

[0033] Generate a switching instruction according to the scheduling detection result, activate the multi-model scheduling policy group for switching control, and determine multiple model service data.

[0034] Based on the scheduling detection result, the intelligent scheduling system generates switching instructions. These instructions activate the previously configured multi-model scheduling policy group, and select the optimal model service for switching according to predefined rules and algorithms; after the switching is completed, determine and obtain the model service data generated by these model services.

[0035] Furthermore, a handover instruction is generated according to the scheduling detection result to activate the multi-model scheduling policy group for handover control, and multiple model service data are determined. The method includes: Set multi-level handover trigger conditions based on the model scheduling priority weight matrix; use the main model service data as an index according to the abnormal scheduling event to index the dynamic handover policy library, and determine the target handover trigger condition, where the target handover trigger condition is included in the multi-level handover trigger conditions; generate a handover instruction according to the target handover trigger condition, traverse the multi-model scheduling policy group according to the handover instruction for automatic handover, and determine the backup model service data; when the single scheduling request of the backup model service data fails, trigger the automatic retry mechanism for handover control to determine the multiple model service data.

[0036] The system sets multi-level handover trigger conditions based on the model scheduling priority weight matrix to adopt appropriate handover strategies in different operating states, realizing intelligent scheduling and optimal utilization of resources. Specifically, under the first-level handover trigger condition, when it is detected that the service success rate is lower than 80% for 30 consecutive seconds, the system will immediately trigger the handover mechanism to ensure business continuity; under the second-level handover trigger condition, when the resource consumption rate exceeds the set threshold and it is predicted that a resource bottleneck may be reached within the next 30 minutes, the system will start the planned handover mechanism. This mechanism, through dynamic resource monitoring and predictive analysis, can identify upcoming resource shortages in advance, such as the impending exhaustion of CPU and GPU computing resources, or storage and bandwidth approaching full load. When the system meets the planned handover conditions, it will gradually switch to a backup model or computing node with better resource utilization during the off-peak period of the business to optimize the overall resource scheduling; under the third-level handover trigger condition, when the historical same-period failure rate exceeds 60%, the system will execute a preventive handover strategy based on long-term data trend analysis. This strategy is mainly applicable to preventive adjustments before the peak period. For example, if a certain model service has failed multiple times during the same period in the past, or a certain node has high instability in a specific scenario.

[0037] When the system detects an abnormal scheduling event, it uses the current main model service data as an index to query the dynamic handover policy library and obtain the target handover trigger condition that matches the current abnormal situation. The target handover trigger condition is included in the multi-level handover trigger conditions to ensure the accuracy and executability of the handover strategy. For example, if a certain computing node cannot provide stable services due to resource exhaustion, the system will search for the best available node in the dynamic handover policy library and select the optimal backup model service for handover when specific constraints (such as response time, computing resource load, etc.) are met.

[0038] After determining the target switching trigger condition, the system generates a switching instruction based on it and executes the automatic switching process by traversing the multi-model scheduling policy group. Specifically, according to the preset scheduling logic, the system selects the optimal candidate service from the available list of standby model services and completes the traffic migration seamlessly to ensure the continuity and stability of the business. During the switching process, if a single scheduling request for the standby model service fails, the system will immediately trigger the automatic retry mechanism for switching control. The automatic retry mechanism first determines the error type code parameter and classifies the abnormal scenarios based on different error types. For example, for errors caused by short-term network fluctuations, the system may perform a quick retry, while for situations of long-term unavailability, it may directly trigger higher-level fault handling strategies such as rollback or circuit breaker. At the same time, the system dynamically adjusts the retry policy, such as setting the upper limit threshold of the retry times, and when the consecutive failure times exceed the threshold, it triggers the abnormal alarm and service circuit breaker mechanism to prevent the system from falling into an ineffective retry loop and further improve the overall service stability and intelligent management ability.

[0039] Furthermore, when a single scheduling request for the standby model service data fails, the automatic retry mechanism is triggered for switching control to determine the multiple model service data. The method includes: When a single scheduling request for the standby model service data fails, determine the error type code parameter; distinguish the scenarios according to the error type code parameter, determine multiple abnormal scenario information, construct a retry policy decision tree based on the multiple abnormal scenario information; set the upper limit threshold of the retry times, perform switching control through the retry policy decision tree, and generate an abnormal alarm when the consecutive failure times exceed the upper limit threshold of the retry times; perform service circuit breaker through the abnormal alarm to obtain the multiple model service data.

[0040] When a single scheduling request for the standby model service data fails, the system determines the specific reason for the failure through the error type code parameter. These error types may include connection timeout, service unavailable, or result verification failure, etc. Through the classification of the error types, the system can further determine whether the fault belongs to a temporary problem or a long-term irrecoverable problem. Next, the system distinguishes different abnormal scenarios according to the error type code parameter. Specifically, the possible abnormal scenarios include but are not limited to connection timeout, service unavailable, and result verification failure, etc. Each scenario will trigger specific processing logics. For example, for connection timeout, the request may be retried several times, while for service unavailable, the switching of the standby model will be immediately triggered. According to different abnormal scenarios, the system constructs a retry policy decision tree, and each node in the tree represents a decision path, and the appropriate policy is selected for processing according to the current scenario type.

[0041] To prevent the waste of system resources caused by infinite retries, the system will set an upper threshold for the number of retries. Each time a request fails, the system will perform switching control according to the path of the decision tree and record the number of failures. Once the consecutive failure count exceeds the set upper threshold for the number of retries, the system will automatically generate an exception alert to notify the management for further handling. At this time, the system will respond to the exception alert and take appropriate measures, such as adjusting the scheduling strategy, switching to other alternative services, or optimizing the current service configuration. If the problem still persists after multiple retries and alert responses, the system will stop calling the currently unavailable alternative model service through the service circuit breaker mechanism and instead select other more stable model services to obtain multiple model service data, thereby ensuring the reliability and stability of the overall service.

[0042] Generate a gateway access key, and encapsulate multiple model service data through the unified access platform according to the gateway access key to generate multi-model response data.

[0043] The system generates a gateway access key through a security mechanism. The gateway access key is an independent key pair assigned to each access client, aiming to ensure the security of communication and authentication. After generating the gateway access key, the unified access platform will encapsulate multiple model service data through this key to generate multi-model response data.

[0044] Furthermore, the process of generating the gateway access key includes the following steps: Use hierarchical encryption technology to assign to each access client, determine multiple independent key pairs, and construct a key management layer; perform API signature verification based on the multiple independent key pairs to generate edge security verification information; adapt the heterogeneous model return data according to the edge security verification information to generate an adapted format; match the multiple independent key pairs with the adapted format, and extract access to the key management layer according to the matching result to determine the gateway access key.

[0045] The system uses hierarchical encryption technology to perform secure allocation for each accessing client; through this technology, each client will obtain an independent key pair, and these key pairs are used to ensure the confidentiality and integrity of the communication between the client and the platform. Among them, these independent key pairs are centrally stored in a key management layer, and the key management layer is responsible for managing the life cycle of all key pairs, including operations such as generation, storage, update, and revocation. Next, based on the multiple independent key pairs, the system will perform API signature verification. In each API call, the system will use the private key part in the key pair to sign the request. The role of API signature is to ensure that the source of the request is an authenticated client, and at the same time, it can also verify whether the request has been tampered with during the transmission process. The signature verification result will generate an edge security verification information, and this information is used to further confirm the legitimacy of the client and the integrity of the request. Once the verification is completed, the system will adapt the data returned from the heterogeneous model according to the edge security verification information. The data formats returned by the heterogeneous models may be different, and the system needs to adapt the data according to the security verification information and the data format standard to make it conform to the unified format required by the platform. Through this adaptation process, the system can ensure that data from different sources can be smoothly processed and transmitted in the unified access platform. Finally, the system matches the multiple independent key pairs with the adapted data format; according to the matching result, the key management layer will extract the gateway access key related to the current request.

[0046] Furthermore, the unified access platform encapsulates multiple model service data according to the gateway access key to generate multi-model response data. The method includes: The unified access platform receives a client request for parsing to generate request parsing data, performs double verification on the request parsing data to generate a data verification result; initiates response monitoring according to the data verification result to generate multiple service response status information; performs multi-level processing on the data returned by the heterogeneous model to generate a heterogeneous model processing result; encapsulates the multiple service response status information according to the heterogeneous model processing result and the gateway access key to generate the multi-model response data.

[0047] The unified access platform receives requests from the client and parses them. During the parsing process, the requests sent by the client are converted into a standard format that the platform can handle, generating request parsing data. Subsequently, the platform performs double verification on this request parsing data. Identity verification ensures that the source of the request is legal, while permission verification guarantees that the requesting user has the authorization to access the corresponding model service. After completing the double verification, the platform generates a data verification result, and based on this result, it determines whether the request is valid and then decides whether to continue with the next step of processing. According to the data verification result, the platform activates the response monitoring mechanism. The response monitoring mechanism is responsible for real-time tracking and recording the response status of each service, including key information such as response time and whether the service has successfully responded. The platform generates multiple service response status information based on this monitoring information to provide comprehensive monitoring of the health status of each service. Then, the platform performs multi-level processing on the data returned from heterogeneous model services. Since multiple model services come from different providers, the data they return may have differences in format, protocol, or structure. Therefore, the platform needs to uniformly process this data, including data format conversion, abnormal data elimination, and optimization of model results, etc., to ensure that all data can be seamlessly integrated and meet the unified output standard. Finally, based on the processing results of heterogeneous models and combined with the previously extracted gateway access key, the platform encapsulates the multiple service response status information. The encapsulation process packs all the processed and optimized data and encrypts or signs the encapsulated data using the gateway access key to ensure the security and integrity of the data. After completing the encapsulation, the platform generates multi-model response data and sends it back to the client to ensure that the client can receive a data response that has been verified, processed, and optimized.

[0048] In summary, the embodiments of the present application have at least the following technical effects: First, an API connection is established through edge computing to build a unified access platform. Then, a multi-model scheduling policy group is configured. Next, model scheduling detection is performed based on the unified access platform to generate a scheduling detection result. Further, a switching instruction is generated according to the scheduling detection result to activate the multi-model scheduling policy group for switching control and determine multiple model service data. Finally, a gateway access key is generated, and the multiple model service data is encapsulated through the unified access platform according to the gateway access key to generate multi-model response data. This solves the technical problems of inflexible model service scheduling, slow response speed, and low resource utilization efficiency in the prior art, and achieves the technical effects of improving the flexibility of model service scheduling, accelerating the response speed, and optimizing resource utilization efficiency.

[0049] Embodiment 2, based on the same inventive concept as the multi-model service intelligent scheduling optimization method based on edge computing in the foregoing embodiment, as Figure 2 shown, the present application provides a multi-model service intelligent scheduling optimization system based on edge computing, wherein the system includes: The connection module 11 is used to perform API connection through edge computing to build a unified access platform; the configuration module 12 is used to configure a multi-model scheduling policy group; the detection module 13 is used to perform model scheduling detection based on the unified access platform to generate a scheduling detection result; the control module 14 is used to generate a switching instruction according to the scheduling detection result, activate the multi-model scheduling policy group for switching control, and determine multiple model service data; the encapsulation module 15 is used to generate a gateway access key, and encapsulate the multiple model service data according to the gateway access key through the unified access platform to generate multi-model response data.

[0050] Furthermore, the connection module 11 is used to execute the following method: Unify the input parameter structures of multiple model services, and deploy a dynamic parameter mapping engine; traverse the API endpoints for data storage to build a service description meta-database; create a protocol conversion matrix, and perform protocol compatibility detection on the dynamic parameter mapping engine and the service description meta-database according to the protocol conversion matrix to build a multi-protocol API adaptation layer; dynamically access the API interfaces of multiple heterogeneous model service providers through the multi-protocol API adaptation layer to form the unified access platform.

[0051] Furthermore, the configuration module 12 is used to execute the following method: Traverse the edge computing nodes for real-time monitoring to obtain multiple real-time data, and dynamically generate a model scheduling priority weight matrix according to the multiple real-time data; verify the logical consistency between multiple policies through a rule engine to establish a policy conflict detection mechanism; execute multiple model service policies according to the model scheduling priority weight matrix to generate a policy execution result; analyze the policy execution result according to the policy conflict detection mechanism to generate optimized scheduling parameters; optimize the multiple model service policies based on the optimized scheduling parameters to determine the multi-model scheduling policy group.

[0052] Furthermore, the configuration module 12 is used to execute the following method: Perform distributed tracing on the multiple model service policies based on the model scheduling priority weight matrix to obtain multiple policy execution paths; perform policy evaluation according to the multiple policy execution paths to generate multiple policy effect information; perform feedback control based on the multiple policy effect information to generate the policy execution result.

[0053] Furthermore, the detection module 13 is used to execute the following method: Traverse the multiple model service policies to determine the primary model service data; extract the running characteristics of the primary model service data through a sliding time window to determine the dynamic running data set; perform anomaly recognition based on the dynamic running data set, perform anomaly identification according to the recognition result to determine the anomaly scheduling event; perform confidence weighting according to the anomaly scheduling event to generate the scheduling detection result.

[0054] Further, the control module 14 is used to execute the following method: Set multi-level switching trigger conditions based on the model scheduling priority weight matrix; use the primary model service data as an index according to the anomaly scheduling event to index the dynamic switching policy library to determine the target switching trigger condition, and the target switching trigger condition is included in the multi-level switching trigger conditions; generate a switching instruction according to the target switching trigger condition, traverse the multi-model scheduling policy group according to the switching instruction for automatic switching to determine the backup model service data; when a single scheduling request for the backup model service data fails, trigger an automatic retry mechanism for switching control to determine the multiple model service data.

[0055] Further, the control module 14 is used to execute the following method: When a single scheduling request for the backup model service data fails, determine the error type code parameter; perform scenario differentiation according to the error type code parameter to determine multiple anomaly scenario information, construct a retry policy decision tree according to the multiple anomaly scenario information; set an upper threshold for the number of retries, perform switching control through the retry policy decision tree, and generate an anomaly alarm when the consecutive failure times exceed the upper threshold for the number of retries; perform service fusing through the anomaly alarm to obtain the multiple model service data.

[0056] Further, the encapsulation module 15 is used to execute the following method: Use a hierarchical encryption technology to allocate to each access client to determine multiple independent key pairs and construct a key management layer; perform API signature verification based on the multiple independent key pairs to generate edge security verification information; adapt the heterogeneous model return data according to the edge security verification information to generate an adaptation format; match the multiple independent key pairs with the adaptation format, and perform access extraction on the key management layer according to the matching result to determine the gateway access key.

[0057] Further, the encapsulation module 15 is used to execute the following method: Receive a client request through the unified access platform for parsing to generate request parsing data, perform dual verification on the request parsing data to generate a data verification result; initiate response monitoring according to the data verification result to generate multiple service response status information; perform multi-level processing on the heterogeneous model return data to generate a heterogeneous model processing result; encapsulate the multiple service response status information according to the heterogeneous model processing result in combination with the gateway access key to generate the multi-model response data.

[0058] It should be noted that the above-mentioned sequence of embodiments of the present application is only for description and does not represent the superiority or inferiority of the embodiments. And the above describes specific embodiments of this specification. The processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired result. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0059] The above are only the preferred embodiments of the present application and are not intended to limit the present application. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present application shall be included within the protection scope of the present application.

[0060] This specification and the drawings are only exemplary descriptions of the present application and are considered to have covered any and all modifications, variations, combinations, or equivalents within the scope of the present application. Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the present application and its equivalent technologies, the present application is intended to include these changes and modifications.

Claims

1. A multi-model service intelligent scheduling optimization method based on edge computing, characterized in that: The method comprises: Use edge computing to connect APIs and build a unified access platform; Configure multi-model scheduling policy groups; Performing model scheduling detection based on the unified access platform to generate scheduling detection results; Generate a switching instruction according to the scheduling detection result, activate the multi-model scheduling strategy group to perform switching control, and determine multiple model service data; Generate a gateway access key, encapsulate multiple model service data according to the gateway access key through the unified access platform, and generate multi-model response data.

2. The multi-model service intelligent scheduling optimization method based on edge computing according to claim 1 is characterized in that: Through edge computing, API connections are established to build a unified access platform. The methods include: Convert the input parameter structures of multiple model services uniformly and deploy a dynamic parameter mapping engine; Traverse API endpoints for data storage and build a service description metadata database; Creating a protocol conversion matrix, performing protocol compatibility detection between the dynamic parameter mapping engine and the service description metadata database according to the protocol conversion matrix, and constructing a multi-protocol API adaptation layer; The edge computing nodes are dynamically connected to the API interfaces of multiple heterogeneous model service providers through the multi-protocol API adaptation layer to form the unified access platform.

3. The multi-model service intelligent scheduling optimization method based on edge computing according to claim 2 is characterized in that: Configure a multi-model scheduling policy group. The methods include: Traversing the edge computing nodes for real-time monitoring to obtain a plurality of real-time data, and dynamically generating a model scheduling priority weight matrix according to the plurality of real-time data; Verify the logical consistency between multiple strategies through the rule engine and establish a strategy conflict detection mechanism; Execute multiple model service strategies according to the model scheduling priority weight matrix to generate strategy execution results; Analyze the policy execution results according to the policy conflict detection mechanism to generate optimized scheduling parameters; The multiple model service strategies are optimized based on the optimized scheduling parameters to determine the multi-model scheduling strategy group.

4. The multi-model service intelligent scheduling optimization method based on edge computing according to claim 3 is characterized in that: Executing multiple model service strategies according to the model scheduling priority weight matrix and generating strategy execution results, the method includes: Execute the multiple model service strategies based on the model scheduling priority weight matrix to perform distributed tracing and obtain multiple strategy execution paths; Performing strategy evaluation according to the multiple strategy execution paths to generate multiple strategy effect information; Feedback control is performed based on the plurality of policy effect information to generate the policy execution result.

5. The multi-model service intelligent scheduling optimization method based on edge computing according to claim 4 is characterized in that: Performing model scheduling detection based on the unified access platform to generate scheduling detection results, the method includes: Traversing the plurality of model service strategies to determine main model service data; Extracting operation features of the main model service data through a sliding time window to determine a dynamic operation data set; Perform anomaly recognition based on the dynamic operation data set, perform anomaly identification according to the recognition result, and determine an abnormal scheduling event; Confidence weighting is performed according to the abnormal scheduling event to generate the scheduling detection result.

6. The multi-model service intelligent scheduling optimization method based on edge computing according to claim 5 is characterized in that: Generate a switching instruction according to the scheduling detection result, activate the multi-model scheduling strategy group to perform switching control, and determine multiple model service data, the method comprising: Setting a multi-level switching trigger condition based on the model scheduling priority weight matrix; According to the abnormal scheduling event, the main model service data is used as an index to index the dynamic switching policy library to determine the target switching trigger condition, and the target switching trigger condition is included in the multi-level switching trigger condition; Generate a switching instruction according to the target switching trigger condition, traverse the multi-model scheduling strategy group according to the switching instruction to perform automatic switching, and determine the standby model service data; When a single scheduling request for the standby model service data fails, an automatic retry mechanism is triggered to perform switching control to determine the multiple model service data.

7. The multi-model service intelligent scheduling optimization method based on edge computing according to claim 6 is characterized in that: When a single scheduling request of the standby model service data fails, an automatic retry mechanism is triggered to perform switching control, and the plurality of model service data are determined, the method comprising: When a single scheduling request for the standby model service data fails, determining an error type code parameter; Differentiate scenarios according to the error type code parameters, determine multiple abnormal scenario information, and construct a retry strategy decision tree according to the multiple abnormal scenario information; Setting an upper threshold of the number of retries, performing switching control through the retry strategy decision tree, and generating an abnormal alarm when the number of consecutive failures exceeds the upper threshold of the number of retries; The service is disconnected through the abnormal alarm to obtain the multiple model service data.

8. The multi-model service intelligent scheduling optimization method based on edge computing according to claim 1, characterized in that: The process of generating a gateway access key includes: Use layered encryption technology to assign multiple independent key pairs to each access client and build a key management layer; Performing API signature verification based on the multiple independent key pairs to generate edge security verification information; Adapting the heterogeneous model return data according to the edge security verification information to generate an adaptation format; The multiple independent key pairs are matched with the adaptation format, and access and extract the key management layer according to the matching result to determine the gateway access key.

9. The multi-model service intelligent scheduling optimization method based on edge computing according to claim 8, characterized in that: The method includes encapsulating multiple model service data according to the gateway access key through the unified access platform to generate multi-model response data, and comprising: Receiving a client request through the unified access platform for parsing, generating request parsing data, performing double verification on the request parsing data, and generating a data verification result; Initiate response monitoring according to the data verification result and generate multiple service response status information; Perform multi-level processing on the heterogeneous model return data to generate a heterogeneous model processing result; The plurality of service response status information are data-encapsulated according to the heterogeneous model processing result and in combination with the gateway access key to generate the multi-model response data.

10. The multi-model service intelligent scheduling optimization system based on edge computing is characterized by: The system for implementing the multi-model service intelligent scheduling optimization method based on edge computing according to any one of claims 1 to 9 comprises: The connection module is used to connect to the API through edge computing and build a unified access platform; Configuration module, used to configure multi-model scheduling strategy group; A detection module, used to perform model scheduling detection based on the unified access platform and generate scheduling detection results; A control module, used to generate a switching instruction according to the scheduling detection result, activate the multi-model scheduling strategy group to perform switching control, and determine multiple model service data; The encapsulation module is used to generate a gateway access key, encapsulate multiple model service data according to the gateway access key through the unified access platform, and generate multi-model response data.

Images