Intelligent Scheduling Optimization Method and System for Multi-Model Services Based on Edge Computing

Abstract

The present invention discloses an intelligent scheduling optimization method and system for multi-model services based on edge computing, which relates to the technical field of data processing. The method includes: performing API connection through edge computing to construct a unified access platform; configuring a multi-model scheduling policy group; performing model scheduling detection based on the unified access platform to generate a scheduling detection result; generating a switching instruction according to the scheduling detection result, activating the multi-model scheduling policy group for switching control to determine multiple model service data; generating a gateway access key, and encapsulating the multiple model service data through the unified access platform according to the gateway access key to generate multi-model response data. It solves the technical problems of inflexible model service scheduling, slow response speed and low resource utilization efficiency in the prior art, and achieves the technical effects of improving the flexibility of model service scheduling, accelerating the response speed and optimizing the resource utilization efficiency.

Description

Technical Field

[0001] The present invention relates to the technical field of data processing, and particularly to an intelligent scheduling optimization method and system for multi-model services based on edge computing. Background Art

[0002] With the rapid development of technologies such as the Internet of Things, big data, and cloud computing, the generation and transmission of massive data have become the norm, which poses higher requirements for the scheduling efficiency, response speed, and resource utilization ability of model services. Model services, as key components in intelligent applications, are responsible for performing intelligent tasks such as prediction, classification, and recognition based on input data, and their performance is directly related to the stability of the entire intelligent system and the user experience. However, most traditional model service scheduling methods adopt a centralized architecture, where data needs to be transmitted from various terminals or sensors to a central server for processing and then the results are returned. This architecture not only increases data transmission latency, reduces the system response speed, but also faces problems such as uneven resource allocation and inflexible scheduling. Especially when facing large-scale and high-concurrency data processing requirements, the centralized architecture often appears inadequate and cannot meet the real-time, efficient, and intelligent service needs. Summary of the Invention

[0003] This application provides an intelligent scheduling optimization method and system for multi-model services based on edge computing, which solves the technical problems of inflexible model service scheduling, slow response speed, and low resource utilization efficiency in the prior art.

[0004] In the first aspect of this application, an intelligent scheduling optimization method for multi-model services based on edge computing is provided. The method includes:

[0005] Conduct API connection through edge computing to build a unified access platform; configure a multi-model scheduling policy group; perform model scheduling detection based on the unified access platform to generate a scheduling detection result; generate a switching instruction according to the scheduling detection result, activate the multi-model scheduling policy group for switching control to determine multiple model service data; generate a gateway access key, and encapsulate the multiple model service data according to the gateway access key through the unified access platform to generate multi-model response data.

[0006] In the second aspect of this application, an intelligent scheduling optimization system for multi-model services based on edge computing is provided. The system includes:

[0007] A connection module for API connection through edge computing to build a unified access platform; a configuration module for configuring a multi-model scheduling policy group; a detection module for performing model scheduling detection based on the unified access platform to generate a scheduling detection result; a control module for generating a switching instruction according to the scheduling detection result, activating the multi-model scheduling policy group for switching control, and determining multiple model service data; a packaging module for generating a gateway access key, and encapsulating the multiple model service data through the unified access platform according to the gateway access key to generate multi-model response data.

[0008] One or more technical solutions provided in this application have at least the following technical effects or advantages:

[0009] First, perform API connection through edge computing to build a unified access platform. Next, configure a multi-model scheduling policy group. Then, perform model scheduling detection based on the unified access platform to generate a scheduling detection result. Further, generate a switching instruction according to the scheduling detection result, activate the multi-model scheduling policy group for switching control, and determine multiple model service data. Finally, generate a gateway access key, and encapsulate the multiple model service data through the unified access platform according to the gateway access key to generate multi-model response data. This solves the technical problems of inflexible model service scheduling, slow response speed, and low resource utilization efficiency in the prior art, and achieves the technical effects of improving the flexibility of model service scheduling, accelerating the response speed, and optimizing the resource utilization efficiency. Description of the Drawings

[0010] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the following drawings are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0011] Figure 1 It is a schematic flowchart of an intelligent scheduling optimization method for multi-model services based on edge computing provided by an embodiment of this application;

[0012] Figure 2 It is a schematic structural diagram of an intelligent scheduling optimization system for multi-model services based on edge computing provided by an embodiment of this application.

[0013] Description of the reference numerals: connection module 11, configuration module 12, detection module 13, control module 14, packaging module 15. Detailed Embodiments

[0014] The present application provides an intelligent scheduling optimization method and system for multi-model services based on edge computing, which solves the technical problems of inflexible model service scheduling, slow response speed, and low resource utilization efficiency in the prior art.

[0015] Next, the technical solutions in the embodiments of the present application will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present application without creative efforts shall fall within the scope of protection of the present application.

[0016] It should be noted that the terms "include" and "have" are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or server that includes a series of steps or units does not necessarily have to be limited to those steps or units clearly listed, but may include other steps or modules that are not clearly listed or are inherent to these processes, methods, products, or devices.

[0017] Embodiment 1, as Figure 1 shown, the present application provides an intelligent scheduling optimization method for multi-model services based on edge computing, where the method includes:

[0018] Perform API connection through edge computing to build a unified access platform.

[0019] Connect the model services on each edge node through an API (Application Programming Interface) to build a unified access platform, enabling users or upper-layer applications to access and call different model services through this platform without caring about the underlying complex network architecture and model distribution.

[0020] Furthermore, performing API connection through edge computing to build a unified access platform, the method includes:

[0021] Unify the input parameter structures of multiple model services, deploy a dynamic parameter mapping engine; traverse the API endpoints for data storage to build a service description meta-database; create a protocol conversion matrix, and perform protocol compatibility detection on the dynamic parameter mapping engine and the service description meta-database according to the protocol conversion matrix to build a multi-protocol API adaptation layer; dynamically access the API interfaces of multiple heterogeneous model service providers through the multi-protocol API adaptation layer to form the unified access platform.

[0022] The system uniformly converts the input parameter structures of multiple model services. For the input formats, data types, and parameter requirements of different models, a dynamic parameter mapping engine is deployed to achieve standardized processing of parameters, ensuring that each model can operate compatibly under a unified call framework. The dynamic parameter mapping engine extracts the field names, data types, format requirements, and default value information of input parameters by parsing the API documentation, interface descriptions, or sample requests of model services, and converts the input parameters of different model services according to predefined parameter mapping rules to make them conform to the standardized input format. For parameters that require additional processing, such as missing value filling, data unit conversion, or format transformation, the dynamic parameter mapping engine automatically performs adaptation through a built-in rule library to ensure that all model services can receive request data according to a unified interface specification.

[0023] After completing the parameter standardization process, the system automatically scans all the connected model services, accesses their public API endpoints one by one, and collects the basic information of the APIs, including but not limited to interface addresses, request methods (such as GET, POST), input parameters, return data structures, authentication methods, and response times, etc.; the collected API information is stored in the service description meta-database after being structured. The database contains a complete model service information table for subsequent dynamic calls, load balancing, and compatibility detection.

[0024] To adapt to the communication protocols used by different APIs, the system further creates a protocol conversion matrix. This matrix contains mapping rules for common communication protocols such as RESTful API, gRPC, WebSocket, etc., and defines the conversion methods between each protocol. During the operation of the system, the protocol conversion matrix automatically detects the communication protocol type of each model service API through a protocol parsing module, and matches the dynamic parameter mapping engine with the service description meta-database according to a preset protocol compatibility detection process to ensure that all API calls conform to the unified standards of the platform. On this basis, the system constructs a multi-protocol API adaptation layer based on the protocol conversion matrix. This adaptation layer integrates protocol parsing, format conversion, request forwarding, and response parsing functions, can automatically identify the API communication method, and perform transparent conversion between different protocols, enabling edge computing nodes to seamlessly connect to various heterogeneous model services without additional modification or encapsulation of each API. Finally, the edge computing node realizes dynamic access to the API interfaces of multiple heterogeneous model service providers through the multi-protocol API adaptation layer, automatically adapts to the communication protocols and data formats of different models, thus forming a unified access platform.

[0025] Configure a multi-model scheduling policy group.

[0026] On top of the unified access platform, multiple model scheduling policy groups are configured. These model scheduling policy groups have predefined different scheduling rules based on factors such as business requirements, resource status, and model performance, and can select the optimal model for service according to indicators such as the real-time load, prediction accuracy, and response time of the model.

[0027] Furthermore, the method for configuring the multi-model scheduling policy group includes:

[0028] Traverse the edge computing nodes for real-time monitoring to obtain multiple real-time data, and dynamically generate a model scheduling priority weight matrix based on the multiple real-time data; verify the logical consistency between multiple policies through a rule engine and establish a policy conflict detection mechanism; execute multiple model service policies according to the model scheduling priority weight matrix to generate policy execution results; analyze the policy execution results according to the policy conflict detection mechanism to generate optimized scheduling parameters; optimize the multiple model service policies based on the optimized scheduling parameters to determine the multi-model scheduling policy group.

[0029] The system traverses each model service deployed on the edge computing nodes and conducts real-time monitoring on them, collecting multiple key performance indicators including service response time, computing load, data throughput, task queue length, prediction accuracy, etc. At the same time, combined with external environmental factors (such as network latency, available computing power, etc.), multiple real-time data are formed.

[0030] Based on the collected real-time data, the system dynamically generates a model scheduling priority weight matrix using a multi-dimensional decision-making method. First, weights are assigned to different performance indicators, and the indicator weight ratio is adjusted according to business requirements. For example, in latency-sensitive applications, the weight of the response time will be increased, while in high-precision tasks, the weight ratio of prediction accuracy will be larger. Then, the system uses a weighted scoring mechanism to calculate the comprehensive scores of each model, generates a priority ranking according to the scores, and constructs a model scheduling priority weight matrix. This matrix can reflect the relative priorities of different models in the current system state and provide a decision-making basis for subsequent intelligent scheduling. On this basis, the system verifies the logical consistency of multiple scheduling policies through a rule engine and establishes a policy conflict detection mechanism. The rule engine presets a series of scheduling rules and constraints, such as: avoiding the repeated execution of the same task on multiple models, ensuring the minimum resource guarantee for critical tasks, and avoiding low-priority tasks from occupying high-priority computing resources. The system uses a conflict detection algorithm based on graph analysis to analyze the relationships between different scheduling policies and identify potential policy conflicts, such as resource competition, circular dependencies, policy coverage, etc. When conflicts are detected, the system can automatically adjust the model weights and modify the policy execution order to ensure the coordinated execution of multi-model scheduling policies.

[0031] Next, the system executes multiple model service policies according to the model scheduling priority weight matrix, and distributes computing tasks among different models through a distributed task scheduling mechanism. The execution result of each scheduling policy is recorded and monitored by a distributed tracing system to form multiple policy execution paths. Based on the execution path data, the system evaluates the scheduling effect of each model, calculates the changes in key performance indicators, and generates the policy execution result. Subsequently, based on the policy conflict detection mechanism, the system deeply analyzes the policy execution result and calculates optimized scheduling parameters in combination with the actual operation situation. For example, when certain policies lead to too high computing load, the system can dynamically adjust the task allocation ratio to appropriately divert the computing tasks of high-load models to low-load models; when the prediction accuracy of certain models is lower than expected, the system can adjust the model combination method, increase multi-model fusion calculation, and improve the overall inference accuracy. Finally, based on the optimized scheduling parameters, the system dynamically adjusts multiple model service policies and determines the optimal multi-model scheduling policy group, which can flexibly adjust the scheduling method according to real-time business requirements, including load balancing mode, performance priority mode, resource saving mode, etc., to adapt to different application scenarios. For example, in a high-concurrency scenario, the system can automatically enable the load balancing policy and preferentially select the model with the shortest response time for inference calculation; in a low-power consumption scenario, the system can preferentially select models with lower energy consumption and reduce the call frequency of high-energy consumption models.

[0032] Furthermore, executing multiple model service policies according to the model scheduling priority weight matrix to generate a policy execution result, the method includes:

[0033] Performing distributed tracing by executing the multiple model service policies based on the model scheduling priority weight matrix to obtain multiple policy execution paths; performing policy evaluation according to the multiple policy execution paths to generate multiple policy effect information; and performing feedback control based on the multiple policy effect information to generate the policy execution result.

[0034] Based on the model scheduling priority weight matrix, the system distributes computing tasks and executes multiple model service policies according to the execution priorities of different models. During the execution process, the system uses distributed tracing technology to perform full-link tracking on the execution situation of tasks in different model services, that is, assigns a unique identifier to each task and records the complete path of the task from request initiation to result return. The system collects key information during the task flow, including the time when the task enters the model queue, computing time consumption, task execution status, data flow path, resource occupancy, etc., to form multiple policy execution paths. These paths can not only reflect the specific execution situation of tasks in each model but also reveal the impact of different scheduling policies on computing resources.

[0035] After obtaining multiple policy execution paths, the system evaluates these paths to analyze the execution effects of different scheduling policies. During the evaluation process, key metrics such as policy hit rate, policy switching latency, and policy execution exception rate are mainly concerned. Among them, the policy hit rate is used to measure the matching degree between the actually executed scheduling policy and the preset policy, and the calculation method is the ratio of the number of successfully matched policies to the total number of executed policies. The higher the hit rate, the stronger the scheduling stability of the system. Otherwise, it may be necessary to adjust the model weights or optimize the policy adaptation logic. The policy switching latency measures the time required to switch to the backup model after the main model encounters an exception or load change. This latency is determined by the time difference from the failure of the main model to the stable operation of the backup model. A higher switching latency may lead to service interruption. Therefore, the performance can be optimized by asynchronously loading the backup model or using a pre-judgment mechanism to switch in advance. The policy execution exception rate measures the frequency of exceptions triggered during the actual operation of the policy, including issues such as policy conflicts, execution failures, or execution deviations. The calculation method is the ratio of the number of exception triggers to the total number of executed policies. If the exception rate is high, it may indicate resource competition, model response failure, or the need to optimize the policy execution logic in the system. The probability of exceptions can be reduced by detecting policy conflicts, optimizing task allocation, or introducing a multi-level fallback mechanism. By comprehensively evaluating the policy hit rate, policy switching latency, and policy execution exception rate, multiple policy effect information is generated.

[0036] Based on multiple policy effect information, feedback control is performed to optimize the scheduling policy and generate the final policy execution result. After the system executes the scheduling policy, it will track and evaluate the policy execution path, and perform dynamic adjustment based on core metrics such as policy hit rate, policy switching latency, and policy execution exception rate. Among them, the policy execution exception rate is used to measure the frequency of exceptions occurring during the operation of the policy. When this indicator exceeds 5%, the system will automatically trigger the policy rollback mechanism to ensure scheduling stability. The policy rollback mechanism detects the execution path of the abnormal policy, determines the specific cause of the exception trigger, and selects the optimal alternative policy for replacement in combination with historical data to ensure the reliability and continuity of the scheduling logic. During the process of triggering the rollback, the system first analyzes the exception category based on the exception log to determine whether it is caused by model service failure, resource competition, or policy conflict, and then selects the appropriate rollback method according to the exception type, such as switching to the stable policy of the previous version, recalculating the weights to adjust the scheduling priority, or directly enabling the standby model for service switching.

[0037] Based on the unified access platform, model scheduling detection is performed to generate a scheduling detection result.

[0038] Through the unified access platform, scheduling detection of the current model service is carried out, including monitoring the running status, performance metrics, and user feedback of each model to obtain real-time scheduling information. Based on this information, a scheduling detection result is generated to provide a basis for subsequent switching decisions.

[0039] Furthermore, based on the unified access platform, model scheduling detection is performed to generate a scheduling detection result. The method includes:

[0040] Traverse the multiple model service policies to determine the primary model service data; extract the running characteristics of the primary model service data through a sliding time window to determine the dynamic running data set; perform anomaly recognition based on the dynamic running data set, and perform anomaly identification according to the recognition result to determine the abnormal scheduling event; perform confidence weighting based on the abnormal scheduling event to generate the scheduling detection result.

[0041] First, the system traverses multiple model service policies to determine the primary model service data currently being executed, including core information such as the input parameters, calculation status, output results, and running logs of the model. Then, the sliding time window technique is used to extract the running characteristics of the primary model service data, and key running parameters are extracted from historical and real-time data, such as model response time, prediction error, calculation load, and resource consumption. Based on these parameters, a dynamic running data set is constructed, which can reflect the long-term running trend and short-term state changes of the model and provide accurate data support for anomaly detection. Subsequently, the system performs anomaly recognition based on the dynamic running data set, and uses statistical analysis, rule matching, or machine learning methods to detect possible abnormal situations, such as model response timeouts, excessive calculation result deviations, abnormal resource occupancy, or frequent error logs. For the detected anomalies, the system classifies and identifies them, determines the abnormal scheduling event, and analyzes it in combination with the number of occurrences of the anomaly, the scope of influence, and historical records. Next, the system uses the confidence weighting method to evaluate based on the abnormal scheduling event. By calculating the confidence level of the anomaly, the severity and possible scope of influence of the anomaly are judged, so as to decide whether model switching or scheduling optimization is required. Finally, the system comprehensively analyzes multiple anomaly indicators to generate a scheduling detection result, and uses this result to optimize the model scheduling strategy, such as adjusting model weights, switching to a standby model, or optimizing the calculation resource allocation, to ensure the continuous and stable operation and intelligent optimized scheduling of the multi-model service.

[0042] Generate a switching instruction according to the scheduling detection result, activate the multi-model scheduling policy group for switching control, and determine multiple model service data.

[0043] Based on the scheduling detection result, the intelligent scheduling system generates switching instructions, which activate the previously configured multi-model scheduling policy group. According to predefined rules and algorithms, the optimal model service is selected for switching; after the switching is completed, the model service data generated by these model services is determined and obtained.

[0044] Furthermore, generating a switching instruction according to the scheduling detection result, activating the multi-model scheduling strategy group for switching control, and determining multiple model service data, the method includes:

[0045] Multi-level switching trigger conditions are set based on the model scheduling priority weight matrix; according to the abnormal scheduling event, the main model service data is used as an index to index the dynamic switching policy library to determine the target switching trigger condition, and the target switching trigger condition is included in the multi-level switching trigger condition; a switching instruction is generated according to the target switching trigger condition, and the multi-model scheduling policy group is traversed according to the switching instruction to perform automatic switching and determine the backup model service data; when a single scheduling request of the backup model service data fails, an automatic retry mechanism is triggered to perform switching control and determine the multiple model service data.

[0046] The system sets multiple levels of handover triggers based on a model scheduling priority weight matrix, enabling intelligent scheduling and optimized resource utilization by implementing appropriate handover strategies under different operating conditions. Specifically, under the first-level handover trigger, if the service success rate is monitored to be below 80% for 30 consecutive seconds, the system immediately triggers a handover mechanism to ensure business continuity. Under the second-level handover trigger, if the resource consumption rate exceeds a set threshold and a resource bottleneck is predicted within the next 30 minutes, the system initiates a planned handover mechanism. This mechanism uses dynamic resource monitoring and predictive analysis to proactively identify impending resource constraints, such as CPU and GPU computing resources running out, or storage and bandwidth reaching capacity. When the planned handover conditions are met, the system gradually switches to backup models or compute nodes with better resource utilization during off-peak hours to optimize overall resource scheduling. Under the third-level handover trigger, if the historical failure rate for the same period exceeds 60%, the system initiates a preventative handover strategy based on long-term data trend analysis. This strategy is primarily used for pre-peak adjustments, such as when a model service has experienced multiple failures during the same time period or when a node exhibits high instability under specific scenarios.

[0047] When the system detects an abnormal scheduling event, it uses the current primary model service data as an index to query the dynamic switching policy library and obtain the target switching trigger conditions that match the current abnormal situation. The target switching trigger conditions are included in the multi-level switching trigger conditions to ensure the accuracy and enforceability of the switching policy. For example, if a computing node cannot provide stable service due to resource exhaustion, the system will search the dynamic switching policy library for the best available node and select the optimal backup model service for switching if specific constraints (such as response time and computing resource load) are met.

[0048] After determining the target switching trigger condition, the system generates a switching instruction based on it and executes the automatic switching process by traversing the multi-model scheduling policy group. Specifically, according to the preset scheduling logic, the system selects the optimal candidate service from the available list of standby model services and completes the traffic migration seamlessly to ensure the continuity and stability of the business. During the switching process, if a single scheduling request for the standby model service fails, the system will immediately trigger the automatic retry mechanism for switching control. The automatic retry mechanism first determines the error type code parameter and classifies the abnormal scenarios based on different error types. For example, for errors caused by short-term network fluctuations, the system may perform a quick retry, while for long-term unavailability, it may directly trigger a higher-level fault handling strategy such as rollback or fuse. At the same time, the system dynamically adjusts the retry policy, such as setting an upper threshold for the number of retries, and when the number of consecutive failures exceeds the threshold, it triggers an abnormal alarm and the service fuse mechanism to prevent the system from falling into an infinite retry loop and further improve the overall service stability and intelligent management ability.

[0049] Furthermore, when a single scheduling request for the standby model service data fails, the automatic retry mechanism is triggered for switching control to determine the multiple model service data. The method includes:

[0050] When a single scheduling request for the standby model service data fails, determine the error type code parameter; distinguish scenarios according to the error type code parameter to determine multiple abnormal scenario information, and construct a retry policy decision tree based on the multiple abnormal scenario information; set an upper threshold for the number of retries, perform switching control through the retry policy decision tree, and generate an abnormal alarm when the number of consecutive failures exceeds the upper threshold for the number of retries; perform service fuse through the abnormal alarm to obtain the multiple model service data.

[0051] When a single scheduling request for the standby model service data fails, the system determines the specific cause of the failure through the error type code parameter. These error types may include connection timeout, service unavailable, or result verification failure, etc. Through the classification of error types, the system can further determine whether the fault is a temporary problem or a long-term irrecoverable problem. Next, the system distinguishes different abnormal scenarios according to the error type code parameter. Specifically, possible abnormal scenarios include but are not limited to connection timeout, service unavailable, and result verification failure, etc. Each scenario will trigger specific processing logic. For example, for connection timeout, the request may be retried several times, while for service unavailable, the switch to the standby model will be triggered immediately. According to different abnormal scenarios, the system constructs a retry policy decision tree, and each node in the tree represents a decision path, and the appropriate policy is selected for processing according to the current scenario type.

[0052] To prevent the waste of system resources caused by infinite retries, the system will set an upper threshold for the number of retries. Each time a request fails, the system will perform switching control according to the path of the decision tree and record the number of failures. Once the consecutive failure count exceeds the set upper threshold for the number of retries, the system will automatically generate an exception alert to notify the management staff for further processing. At this time, the system will respond to the exception alert and take appropriate measures, such as adjusting the scheduling strategy, switching to other backup services, or optimizing the current service configuration. If the problem still persists after multiple retries and alert responses, the system will stop calling the currently unavailable backup model service through the service circuit breaker mechanism and instead select other more stable model services to obtain multiple model service data, thus ensuring the reliability and stability of the overall service.

[0053] Generate a gateway access key, and encapsulate multiple model service data through the unified access platform according to the gateway access key to generate multi-model response data.

[0054] The system generates a gateway access key through a security mechanism. The gateway access key is an independent key pair assigned to each access client, aiming to ensure the security of communication and authentication. After generating the gateway access key, the unified access platform will encapsulate multiple model service data through this key to generate multi-model response data.

[0055] Furthermore, the process of generating the gateway access key includes the following steps:

[0056] Use hierarchical encryption technology to assign to each access client, determine multiple independent key pairs, and construct a key management layer; perform API signature verification based on the multiple independent key pairs to generate edge security verification information; adapt the heterogeneous model return data according to the edge security verification information to generate an adapted format; match the multiple independent key pairs with the adapted format, and access and extract the key management layer according to the matching result to determine the gateway access key.

[0057] The system uses hierarchical encryption technology to securely allocate each accessing client; through this technology, each client will obtain an independent key pair, and these key pairs are used to ensure the confidentiality and integrity of the communication between the client and the platform. Among them, these independent key pairs are centrally stored in a key management layer, and the key management layer is responsible for managing the life cycle of all key pairs, including operations such as generation, storage, update, and revocation. Next, based on the multiple independent key pairs, the system will perform API signature verification. In each API call, the system will use the private key part in the key pair to sign the request. The role of API signature is to ensure that the source of the request is an authenticated client, and at the same time, it can also verify whether the request has been tampered with during transmission. The signature verification result will generate an edge security verification information, which is used to further confirm the legitimacy of the client and the integrity of the request. Once the verification is completed, the system will adapt the data returned from the heterogeneous model according to the edge security verification information. The data formats returned by the heterogeneous models may vary, and the system needs to adapt the data according to the security verification information and the data format standard to make it conform to the unified format required by the platform. Through this adaptation process, the system can ensure that data from different sources can be smoothly processed and transmitted in the unified access platform. Finally, the system matches the multiple independent key pairs with the adapted data format; according to the matching result, the key management layer will extract the gateway access key related to the current request.

[0058] Furthermore, through the unified access platform, the multiple model service data is encapsulated according to the gateway access key to generate multi-model response data. The method includes:

[0059] The unified access platform receives the client request for parsing to generate request parsing data, performs double verification on the request parsing data to generate a data verification result; initiates response monitoring according to the data verification result to generate multiple service response status information; performs multi-level processing on the data returned by the heterogeneous model to generate a heterogeneous model processing result; encapsulates the multiple service response status information according to the heterogeneous model processing result and the gateway access key to generate the multi-model response data.

[0060] The unified access platform receives requests from the client and parses them. During the parsing process, the requests sent by the client are converted into a standard format that the platform can handle, generating request parsing data. Subsequently, the platform performs dual verification on this request parsing data. Identity verification ensures that the source of the request is legitimate, while permission verification guarantees that the requesting user has the authorization to access the corresponding model service. After completing the dual verification, the platform generates a data verification result, and based on this result, it determines whether the request is valid and then decides whether to continue with the next step of processing. According to the data verification result, the platform activates the response monitoring mechanism. The response monitoring mechanism is responsible for tracking and recording the response status of each service in real time, including key information such as response time and whether the service has successfully responded. The platform generates multiple service response status information based on this monitoring information to provide comprehensive monitoring of the health status of each service. Then, the platform performs multi-level processing on the data returned from heterogeneous model services. Since multiple model services come from different providers, the data they return may have differences in format, protocol, or structure. Therefore, the platform needs to uniformly process this data, including data format conversion, abnormal data elimination, and optimization of model results, to ensure that all data can be seamlessly integrated and meet the unified output standard. Finally, based on the processing results of heterogeneous models and combined with the previously extracted gateway access key, the platform performs data encapsulation on multiple service response status information. The encapsulation process packages all the processed and optimized data and encrypts or signs the encapsulated data using the gateway access key to ensure the security and integrity of the data. After completing the encapsulation, the platform generates multi-model response data and sends it back to the client to ensure that the client can receive the verified, processed, and optimized data response.

[0061] In summary, the embodiments of the present application at least have the following technical effects:

[0062] First, an API connection is established through edge computing to build a unified access platform. Then, a multi-model scheduling policy group is configured. Next, model scheduling detection is performed based on the unified access platform to generate a scheduling detection result. Further, a switching instruction is generated according to the scheduling detection result to activate the multi-model scheduling policy group for switching control and determine multiple model service data. Finally, a gateway access key is generated, and the multiple model service data is encapsulated through the unified access platform according to the gateway access key to generate multi-model response data. This solves the technical problems of inflexible model service scheduling, slow response speed, and low resource utilization efficiency in the prior art, and achieves the technical effects of improving the flexibility of model service scheduling, accelerating the response speed, and optimizing the resource utilization efficiency.

[0063] Embodiment 2 is based on the same inventive concept as the multi-model service intelligent scheduling optimization method based on edge computing in the foregoing embodiment, as Figure 2As shown in the figure, the present application provides an intelligent scheduling optimization system for multi-model services based on edge computing. The system includes:

[0064] A connection module 11, which is used to perform API connection through edge computing to build a unified access platform; a configuration module 12, which is used to configure a multi-model scheduling policy group; a detection module 13, which is used to perform model scheduling detection based on the unified access platform to generate a scheduling detection result; a control module 14, which is used to generate a switching instruction according to the scheduling detection result, activate the multi-model scheduling policy group for switching control, and determine multiple model service data; an encapsulation module 15, which is used to generate a gateway access key, and encapsulate the multiple model service data through the unified access platform according to the gateway access key to generate multi-model response data.

[0065] Furthermore, the connection module 11 is used to execute the following method:

[0066] Unify the input parameter structures of multiple model services, deploy a dynamic parameter mapping engine; traverse the API endpoints for data storage to build a service description meta-database; create a protocol conversion matrix, and perform protocol compatibility detection on the dynamic parameter mapping engine and the service description meta-database according to the protocol conversion matrix to build a multi-protocol API adaptation layer; dynamically access the edge computing nodes to the API interfaces of multiple heterogeneous model service providers through the multi-protocol API adaptation layer to form the unified access platform.

[0067] Furthermore, the configuration module 12 is used to execute the following method:

[0068] Traverse the edge computing nodes for real-time monitoring to obtain multiple real-time data, dynamically generate a model scheduling priority weight matrix according to the multiple real-time data; verify the logical consistency between multiple policies through a rule engine to establish a policy conflict detection mechanism; execute multiple model service policies according to the model scheduling priority weight matrix to generate a policy execution result; analyze the policy execution result according to the policy conflict detection mechanism to generate optimized scheduling parameters; optimize the multiple model service policies based on the optimized scheduling parameters to determine the multi-model scheduling policy group.

[0069] Furthermore, the configuration module 12 is used to execute the following method:

[0070] Perform distributed tracing on the multiple model service policies based on the model scheduling priority weight matrix to obtain multiple policy execution paths; perform policy evaluation according to the multiple policy execution paths to generate multiple policy effect information; perform feedback control based on the multiple policy effect information to generate the policy execution result.

[0071] Further, the detection module 13 is used to execute the following method:

[0072] Traverse the multiple model service policies to determine the main model service data; extract the running characteristics of the main model service data through a sliding time window to determine the dynamic running data set; perform anomaly identification based on the dynamic running data set, perform anomaly marking according to the identification result to determine the anomaly scheduling event; perform confidence weighting according to the anomaly scheduling event to generate the scheduling detection result.

[0073] Further, the control module 14 is used to execute the following method:

[0074] Set multi-level switching trigger conditions based on the model scheduling priority weight matrix; use the main model service data as an index according to the anomaly scheduling event to index the dynamic switching policy library to determine the target switching trigger condition, and the target switching trigger condition is included in the multi-level switching trigger conditions; generate a switching instruction according to the target switching trigger condition, and traverse the multi-model scheduling policy group according to the switching instruction for automatic switching to determine the backup model service data; when a single scheduling request for the backup model service data fails, trigger an automatic retry mechanism for switching control to determine the multiple model service data.

[0075] Further, the control module 14 is used to execute the following method:

[0076] When a single scheduling request for the backup model service data fails, determine the error type code parameter; perform scene classification according to the error type code parameter to determine multiple anomaly scene information, and construct a retry policy decision tree according to the multiple anomaly scene information; set the upper limit threshold of the retry times, perform switching control through the retry policy decision tree, and generate an anomaly alarm when the continuous failure times exceed the upper limit threshold of the retry times; perform service fusing through the anomaly alarm to obtain the multiple model service data.

[0077] Further, the encapsulation module 15 is used to execute the following method:

[0078] Use hierarchical encryption technology to allocate to each access client to determine multiple independent key pairs and construct a key management layer; perform API signature verification based on the multiple independent key pairs to generate edge security verification information; adapt the heterogeneous model return data according to the edge security verification information to generate an adaptation format; match the multiple independent key pairs with the adaptation format, and perform access extraction on the key management layer according to the matching result to determine the gateway access key.

[0079] Further, the encapsulation module 15 is used to execute the following method:

[0080] Receive a client request through the unified access platform for parsing to generate request parsing data, perform double verification on the request parsing data to generate a data verification result; initiate response monitoring according to the data verification result to generate multiple service response status information; perform multi-level processing on the heterogeneous model return data to generate a heterogeneous model processing result; encapsulate the multiple service response status information according to the heterogeneous model processing result in combination with the gateway access key to generate the multi-model response data.

[0081] It should be noted that the above sequence of embodiments of the present application is only for description and does not represent the superiority or inferiority of the embodiments. And the above description of specific embodiments of this specification has been made. The processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired result. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0082] The above are only the preferred embodiments of the present application and are not intended to limit the present application. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present application shall be included within the protection scope of the present application.

[0083] This specification and the drawings are only exemplary descriptions of the present application and are considered to have covered any and all modifications, variations, combinations, or equivalents within the scope of the present application. Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the present application and its equivalent technologies, the present application is intended to include these changes and modifications.

Claims

1. An intelligent scheduling optimization method for multi-model services based on edge computing, characterized in that The method includes: Conduct API connection through edge computing to build a unified access platform; Configure a multi-model scheduling policy group; Based on the unified access platform, conduct model scheduling detection to generate a scheduling detection result; Generate a switching instruction according to the scheduling detection result, activate the multi-model scheduling policy group for switching control, and determine multiple model service data; Generate a gateway access key, and encapsulate the multiple model service data through the unified access platform according to the gateway access key to generate multi-model response data; The method of conducting API connection through edge computing to build a unified access platform includes: Unify the conversion of the input parameter structures of multiple model services and deploy a dynamic parameter mapping engine; Traverse the API endpoints for data storage to build a service description metadata database; Create a protocol conversion matrix, and conduct protocol compatibility detection on the dynamic parameter mapping engine and the service description metadata database according to the protocol conversion matrix to build a multi-protocol API adaptation layer; Dynamically connect edge computing nodes to the API interfaces of multiple heterogeneous model service providers through the multi-protocol API adaptation layer to form the unified access platform.

2. The intelligent scheduling optimization method for multi-model services based on edge computing according to claim 1, characterized in that, The method of configuring a multi-model scheduling policy group includes: Traverse the edge computing nodes for real-time monitoring to obtain multiple real-time data, and dynamically generate a model scheduling priority weight matrix according to the multiple real-time data; Verify the logical consistency between multiple policies through a rule engine to establish a policy conflict detection mechanism; Execute multiple model service policies according to the model scheduling priority weight matrix to generate a policy execution result; Analyze the policy execution result according to the policy conflict detection mechanism to generate optimized scheduling parameters; Optimize the multiple model service policies based on the optimized scheduling parameters to determine the multi-model scheduling policy group.

3. The intelligent scheduling optimization method for multi-model services based on edge computing according to claim 2, characterized in that, The method of executing multiple model service policies according to the model scheduling priority weight matrix to generate a policy execution result includes: Execute the multiple model service policies based on the model scheduling priority weight matrix for distributed tracing to obtain multiple policy execution paths; Conduct policy evaluation according to the multiple policy execution paths to generate multiple policy effect information; Conduct feedback control based on the multiple policy effect information to generate the policy execution result.

4. The intelligent scheduling optimization method for multi-model services based on edge computing according to claim 3, characterized in that The method of conducting model scheduling detection based on the unified access platform to generate a scheduling detection result includes: Traverse the multiple model service policies to determine the main model service data; Extract the running characteristics of the main model service data through a sliding time window to determine a dynamic running data set; Conduct anomaly identification based on the dynamic running data set, conduct anomaly marking according to the identification result, and determine an abnormal scheduling event; Conduct confidence weighting according to the abnormal scheduling event to generate the scheduling detection result.

5. The intelligent scheduling optimization method for multi-model services based on edge computing according to claim 4, characterized in that The method of generating a switching instruction according to the scheduling detection result, activating the multi-model scheduling policy group for switching control, and determining multiple model service data includes: Set multi-level switching trigger conditions based on the model scheduling priority weight matrix; Index the dynamic switching policy library using the main model service data according to the abnormal scheduling event, and determine the target switching trigger condition, where the target switching trigger condition is included in the multi-level switching trigger conditions; Generate a switching instruction according to the target switching trigger condition, and automatically switch by traversing the multi-model scheduling policy group according to the switching instruction to determine the standby model service data; When a single scheduling request for the standby model service data fails, trigger an automatic retry mechanism for switching control to determine the multiple model service data.

6. The intelligent scheduling optimization method for multi-model services based on edge computing according to claim 5, characterized in that When a single scheduling request for the standby model service data fails, trigger an automatic retry mechanism for switching control to determine the multiple model service data. The method includes: When a single scheduling request for the standby model service data fails, determine the error type code parameter; Perform scenario differentiation according to the error type code parameter to determine multiple abnormal scenario information, and construct a retry policy decision tree according to the multiple abnormal scenario information; Set an upper threshold for the number of retries, perform switching control through the retry policy decision tree, and generate an abnormal alarm when the consecutive failure times exceed the upper threshold for the number of retries; Perform service fusing through the abnormal alarm to obtain the multiple model service data.

7. The intelligent scheduling optimization method for multi-model services based on edge computing according to claim 1, characterized in that The process of generating a gateway access key. The method includes: Use hierarchical encryption technology to allocate to each access client to determine multiple independent key pairs and construct a key management layer; Perform API signature verification based on the multiple independent key pairs to generate edge security verification information; Adapt the heterogeneous model return data according to the edge security verification information to generate an adapted format; Match the multiple independent key pairs with the adapted format, and extract access to the key management layer according to the matching result to determine the gateway access key.

8. The intelligent scheduling optimization method for multi-model services based on edge computing according to claim 7, wherein The unified access platform encapsulates multiple model service data according to the gateway access key to generate multi-model response data. The method includes: The unified access platform receives a client request for parsing to generate request parsing data, and performs double verification on the request parsing data to generate a data verification result; Initiate response monitoring according to the data verification result to generate multiple service response status information; Perform multi-level processing on the heterogeneous model return data to generate a heterogeneous model processing result; Package the multiple service response status information according to the heterogeneous model processing result in combination with the gateway access key to generate the multi-model response data.

9. An intelligent scheduling optimization system for multi-model services based on edge computing, characterized in that, For implementing the intelligent scheduling optimization method of multi-model services based on edge computing according to any one of claims 1-8, the system includes: A connection module for establishing an API connection through edge computing to construct a unified access platform; A configuration module for configuring a multi-model scheduling policy group; A detection module for performing model scheduling detection based on the unified access platform to generate a scheduling detection result; A control module for generating a switching instruction according to the scheduling detection result, activating the multi-model scheduling policy group for switching control to determine multiple model service data; An encapsulation module is used to generate a gateway access key, and encapsulate multiple model service data according to the gateway access key through the unified access platform to generate multi-model response data.

Images