Model service configuration-oriented management method and device, medium, equipment and product

Abstract

A kind of management method, device, medium, equipment and product for model service configuration, involve computer technical field and artificial intelligence technical field, the management method includes: according to the service request received, in first cache, the first model service configuration matched with service request is inquired by gateway instance, first cache is held by gateway instance, first model service configuration is used to support gateway instance and is routed to corresponding model service end for service request, model service end is used to process service request to obtain the processing result of service request;In response to not inquiring first model service configuration in first cache by gateway instance, in second cache, the first model service configuration matched with service request is inquired, different gateway instances share second cache, so as to ensure that when first cache is unavailable, model service configuration for processing service request can be obtained from second cache, the reliability and stability of service request processing scheme are enhanced.

Description

Technical Field

[0001] This article relates to at least the fields of computer technology and AI (Artificial Intelligence) technology, and specifically to a management method, apparatus, medium, device and product for model service configuration. Background Technology

[0002] With the rapid development of AI (Artificial Intelligence) services, different service providers have launched various AI-based model services, such as services that rely on large language models.

[0003] In related technologies, gateways, serving as a unified entry point for different model services, are typically deployed in a distributed architecture to achieve high availability and load balancing. In such an architecture, the gateway needs to route business requests to the appropriate model service, which then calls the corresponding model service to process the request. Managing the configuration of the model service supporting this routing becomes a key challenge in the entire business processing. Summary of the Invention

[0004] This content section is provided to briefly introduce the ideas, which will be described in detail in the examples section later. This content section is not intended to identify key or essential features of the claimed content, nor is it intended to limit the scope of the claimed content.

[0005] Firstly, a management method for model service configuration is provided, including: Receive business requests through the gateway instance; According to the business request, the gateway instance queries the first cache for a first model service configuration that matches the business request. The first cache is held by the gateway instance. The first model service configuration is used to support the gateway instance in routing the business request to the corresponding model service. The model service is used to process the business request to obtain the processing result of the business request. If the first model service configuration is not found in the first cache through the gateway instance, the first model service configuration matching the business request is queried in the second cache, and different gateway instances share the second cache.

[0006] Secondly, a management method for model service configuration is provided, including: The first receiving module is used to receive service requests through the gateway instance; The first query module is used to query the first model service configuration that matches the business request in the first cache through the gateway instance. The first cache is held by the gateway instance. The first model service configuration is used to support the gateway instance to route the business request to the corresponding model server. The model server is used to process the business request to obtain the processing result of the business request. The second query module is used to query the first model service configuration that matches the business request in the second cache when the first model service configuration is not found in the first cache through the gateway instance. The second cache is shared by different gateway instances.

[0007] Thirdly, a computer-readable medium is provided having a computer program stored thereon, wherein the computer program, when executed by a processing device, implements the steps of the management method described in the first aspect.

[0008] Fourthly, an electronic device is provided, comprising: A storage device on which computer programs are stored; A processing device for executing the computer program in the storage device to implement the steps of the management method described in the first aspect.

[0009] Fifthly, a computer program product is provided, comprising a computer program, wherein when the computer program is executed by a processor, it implements the steps of the management method described in the first aspect.

[0010] Using the above method, the gateway instance queries the first cache held by the gateway instance to find the first model service configuration that matches the business request based on the business request. In response to the first cache not being found by the gateway instance, the first model service configuration that matches the business request is queried in the second cache. Different gateway instances share the second cache. Thus, by configuring the first and second caches and providing a degradation strategy for model service configuration, it is ensured that when the first cache is unavailable, the model service configuration for processing the business request can be obtained from the backup second cache, thereby enhancing the reliability and stability of the business request processing solution.

[0011] Other features and advantages will be described in detail in the following examples section. Attached Figure Description

[0012] The above and other features, advantages, and aspects of this document will become more apparent when viewed in conjunction with the accompanying drawings and the following examples. Throughout the drawings, the same or similar reference numerals denote the same or similar elements. It should be understood that the drawings are schematic, and the originals and elements are not necessarily drawn to scale. In the drawings: Figure 1This is a schematic diagram illustrating an implementation environment based on certain circumstances; Figure 2 This is a flowchart illustrating a management method for model-oriented service configuration based on certain scenarios; Figure 3 This is an architecture diagram illustrating a management approach for model-service configuration, based on certain scenarios. Figure 4 This is a sequence diagram illustrating a management approach for model-oriented service configuration based on certain scenarios; Figure 5 This is a block diagram of a management device for model-service configuration, shown under certain circumstances. Figure 6 This is a schematic diagram of the structure of an electronic device shown under certain circumstances. Detailed Implementation

[0013] The following description will be given in more detail with reference to the accompanying drawings. While certain scenarios are shown in the drawings, it should be understood that this document can be implemented in various forms and should not be construed as limited to the scenarios described herein. Rather, these scenarios are provided to provide a more thorough and complete understanding of this document. It should be understood that the accompanying drawings and the scenarios depicted are for illustrative purposes only and are not intended to limit the scope of this document.

[0014] It should be understood that the steps described in the method may be performed in different orders and / or in parallel. Furthermore, the method may include additional steps and / or omit the steps shown. The scope of this document is not limited in this respect.

[0015] The term "comprising" and its variations can be open-ended, meaning "including but not limited to". The term "based on" can mean "at least partially based on". The term "one case" means "at least one case"; the term "another case" means "at least one additional case"; the term "some cases" means "at least some cases". Definitions of other terms will be given in the following description.

[0016] It should be noted that the concepts of "first" and "second" are only used to distinguish different devices, modules or units, and are not used to limit the order of the functions performed by these devices, modules or units or their interdependencies.

[0017] It should be noted that the modifiers “one” and “multiple” can be illustrative rather than restrictive, and those skilled in the art should understand that, unless otherwise expressly indicated in the context, they should be understood as “one or more”.

[0018] The names of messages or information exchanged between multiple devices are for illustrative purposes only and are not intended to limit the scope of these messages or information.

[0019] It is understandable that the data involved in the technical solution (including but not limited to the data itself, the acquisition or use of the data) should comply with the requirements of relevant regulations.

[0020] As the background technology indicates, gateways, serving as a unified entry point for different model services, are typically deployed in a distributed architecture to achieve high availability and load balancing. In such an architecture, the gateway needs to route business requests to the appropriate model service, which then calls the corresponding model service to process the request. Managing the configuration of the model service supporting this routing becomes a key challenge in the entire business processing. In related technologies, the configuration of model services is maintained through a single cache. However, if this cache becomes inaccessible due to certain failures, the distributed nodes in the gateway cannot obtain the model service configuration, causing the inability to route business requests and thus interrupting their processing. Furthermore, due to the distributed architecture, maintaining the consistency of model service configurations accessed by the distributed nodes in the gateway is also an important research area.

[0021] In view of this, this document provides a management method, apparatus, storage medium, electronic device and program product for model service configuration, which will be explained and described below with reference to the accompanying drawings.

[0022] Figure 1 This is a schematic diagram illustrating an implementation environment based on certain circumstances. See [link / reference]. Figure 1 The implementation environment includes: terminal 101, artificial intelligence gateway 102, and multiple servers 103. Terminal 101 and artificial intelligence gateway 102, as well as artificial intelligence gateway 102 and servers 103, can be directly or indirectly connected via wired or wireless communication, without limitation.

[0023] In one scenario, a business system can be installed on terminal 101. This business system can submit business requests to AI gateway 102, which then routes the requests to at least one of multiple servers 103. AI gateway 102 can receive processing results from at least one server 103 and return these results to the business system. Server 103 can be a backend server of AI gateway 102, providing at least one backend service to the business system; this backend service may refer to a model service.

[0024] In one scenario, the AI ​​gateway 102 may include a configuration center and multiple gateway instances. These gateway instances can achieve purposes such as high availability and load balancing. For example, the configuration center can provide a visual configuration interface to distribute and modify model service configurations, while each gateway instance can route business requests. Each gateway instance can be deployed on hardware resources such as physical servers, cloud hosts, containers, or virtual machines.

[0025] In one scenario, terminal 101 can be at least one of the following devices: smartphone, smartwatch, desktop computer, laptop, virtual reality terminal, augmented reality terminal, wireless terminal, and laptop computer. Terminal 101 has communication capabilities and can access wired or wireless networks. Terminal 101 can refer to one of multiple terminals, and those skilled in the art will understand that the number of such terminals can be more or less.

[0026] In one scenario, each server 103 can be a physical server, or it can be a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN (Content Delivery Network), and big data and artificial intelligence platforms.

[0027] Figure 2 This is a flowchart illustrating a management method for model-service configuration based on certain scenarios. This management method for model-service configuration can be... Figure 1 The AI ​​gateway 102 shown can be executed, and may include multiple gateway instances and a configuration center. Figure 3 This is an architecture diagram illustrating a management method for model-oriented service configuration based on certain scenarios, combined with... Figure 2 and Figure 3 This will provide explanations and clarifications to the text. (Refer to...) Figure 2 The management method for model service configuration may include steps S210, S220 and S230.

[0028] In step S210, a service request is received through the gateway instance.

[0029] Among them, the business request can be from the business party (e.g. Figure 1 The request submitted by terminal 101 shown requires the support of the model service for processing.

[0030] In step S220, the gateway instance queries the first model service configuration that matches the business request in the first cache according to the business request. The first cache is held by the gateway instance. The first model service configuration is used to support the gateway instance in routing the business request to the corresponding model service. The model service is used to process the business request to obtain the processing result of the business request.

[0031] The first cache includes at least one model service configuration, and each model service configuration may include routing configuration and model service information.

[0032] The routing configuration supports matching business requests with model service configurations. For example, using matching based on model identifiers, if the model field carried in the business request is the same as the model field in the routing configuration of the model service, then this model service configuration is the first model service configuration to match the business request. Similarly, using matching based on tenant identifiers, if the tenant field carried in the business request is the same as the tenant field in the routing configuration of the model service, then this model service configuration is the first model service configuration to match the business request. In schemes that support matching based on multiple conditions, the priorities of various conditions can be set, and matching is performed according to the priority.

[0033] Model service information is bound to routing configuration. It ensures that business requests are routed to the corresponding model service. This information can include service groups, backend addresses, weights, and health policies. The backend address can be understood as the address of the model service. Model service information allows gateway instances to route requests from available nodes (nodes providing model services, such as...). Figure 1 From the servers shown, select suitable nodes; these nodes can be considered as model servers for handling business requests. For example, after determining the routing configuration, business requests are routed to the model servers in the corresponding service group. This service group may include multiple model servers deployed in a distributed manner. After determining the service group, further steps can be taken to read all backend addresses (e.g., IP (Internet Protocol) addresses) under that service group. Based on the health policy, determine the healthy nodes among the nodes corresponding to all backend addresses. These healthy nodes can serve as model servers for handling business requests.

[0034] In step S230, in response to the failure to find the first model service configuration in the first cache through the gateway instance, the first model service configuration matching the business request is queried in the second cache, and different gateway instances share the second cache.

[0035] The second cache includes at least one model service configuration. For an explanation and description of the model service configuration, please refer to the above content. Furthermore, the matching logic for querying the first model service configuration that matches the business request in the second cache is the same as the matching logic for querying the first model service configuration that matches the business request in the first cache. This matching logic can also be explained in the above content.

[0036] The model service configuration in the first or second cache can be stored in the form of key-value pairs, where the key is a unique identifier and the value can be the routing configuration and model service information corresponding to that identifier.

[0037] In particular, if the first cache is unavailable or if the first model service configuration that matches the business request does not exist in the first cache, the first model service configuration cannot be found in the first cache.

[0038] By configuring a first cache and a second cache and providing a degradation strategy for model service configuration, the above method ensures that when the first cache is unavailable (e.g., due to network failure), the model service configuration for processing business requests can be obtained from the backup second cache, thereby enhancing the reliability and stability of the business request processing scheme.

[0039] In some cases, the query performance of the first cache is higher than that of the second cache. For example, the first cache could be a local cache within the gateway instance. Furthermore, such as... Figure 3 As shown, the local cache can be a local memory cache. The second cache, however, can be a remote cache relative to the gateway instance, such as... Figure 3 As shown, a remote cache can be a cache shared by all gateway instances in a shared infrastructure.

[0040] Reference Figure 3 The system prioritizes querying the local memory cache held by the gateway instance. If the first model service configuration cannot be found in the local memory cache, a fallback strategy is adopted to query the first model service configuration that matches the business request from the remote cache.

[0041] Since the query performance of the first cache is higher than that of the second cache, querying the first model service configuration that matches the business request from the first cache first can improve the query efficiency of the model service configuration.

[0042] In some cases, the first cache includes the local cache of the gateway instance, and the second cache includes a remote cache. In these cases, the above management method may further include the following steps: in response to querying the first model service configuration in the second cache through the gateway instance, synchronizing the first model service configuration to the first cache of the gateway instance.

[0043] Since the query performance of the local cache of the gateway instance is higher than that of the remote cache, after the gateway instance finds the first model service configuration in the second cache, it synchronizes the first model service configuration to the first cache of the gateway instance, so as to provide a data foundation for subsequent business requests to quickly match the model service configuration.

[0044] Figure 4 This is a sequence diagram illustrating a model-service-oriented configuration management approach based on certain scenarios. The implementation environment for this approach may include a configuration center, a second cache, a message queue, and multiple gateway instances. Figure 4 Only one gateway instance is shown; the other gateway instances implement the same functionality. Figure 4 The gateway instances shown are identical. The configuration center can deploy message generation threads, and each gateway instance can deploy corresponding local memory caches and background processes. The following is based on... Figure 4 Describe the updates to the first and second caches.

[0045] In some cases, as can be seen from the above, gateway instances can be deployed in a distributed architecture, that is, there can be multiple gateway instances. In these cases, the above management method may also include the following steps: updating the second cache in response to a change operation in the configuration center; publishing a change message to the message queue through the configuration center, the change message being used to synchronously update the first cache held by the gateway instance.

[0046] For example, the configuration center can provide an API (Application Programming Interface) for submitting change operations, such as... Figure 3 The interface shown is used to directly trigger an update to the second cache. Specifically, this change operation modifies the model service configuration in the second cache. The operation carries the identifier corresponding to the model service configuration to be changed, the specific model service configuration, and the change type. Based on the carried identifier, model service configuration, and change type, the second cache is updated. The change type can include add, delete, and modify types, and different change types can correspond to different APIs. Figure 3 The interface shown may include sub-interfaces for different change types.

[0047] A change message can include a subscription type, a message type, and a message body. The subscription type indicates the broadcast object of the change message, which can be a gateway instance that has subscribed to the corresponding subscription type. The message type includes broadcast types, which indicate the message distribution method. A broadcast type means that the configuration center distributes the change message via broadcast, rather than the gateway instance obtaining and consuming the change message through polling. The message body can include the change type and the second model service configuration. The change type in the message body is similar to the change type carried by the change operation mentioned above, and can include add, delete, and modify types. The second model service configuration refers to the object being changed. Corresponding to the add, delete, and modify types, when the change type is add, the second model service configuration is the model service configuration that needs to be added in the cache; when the change type is delete, the second model service configuration is the model service configuration that needs to be deleted in the cache; and when the change type is modify, the second model service configuration is the model service configuration whose configuration item needs to be modified in the cache. The configuration item here can be the routing configuration and model service information in the example above. For explanations and descriptions of routing configuration and model service information, please refer to the above content. Using the above method, after the configuration center receives the change operation submitted by the operation user, it first updates the second cache. This step ensures that even if subsequent steps such as publishing change messages or synchronizing the first cache fail, the model service configuration has already been updated in the shared second cache. In addition, by publishing change messages from the configuration center to the message queue, the middleware of the message queue uses broadcast to distribute change messages. Compared with the solution of the gateway instance polling the configuration center or the message queue, this can reduce the pressure on the configuration center and network bandwidth, provide the possibility of expanding the number of gateway nodes, and improve the real-time performance of change message distribution, so as to quickly realize the synchronous update of the first cache on the gateway instance side.

[0048] Continue to refer to Figure 4 After receiving the update success message from the second cache, the configuration center can send the update information to the message producer. This update information can include the broadcast object, broadcast type, change type, and second model service configuration mentioned above. The message producer then generates the corresponding update message based on the broadcast object, broadcast type, change type, and second model service configuration and publishes it to the message queue. Finally, the update message in the message queue is broadcast. (Continue to refer to...) Figure 3 The message generation thread deployed in the configuration center has the same function as the message producer, and is used to at least realize the generation and publication of update messages.

[0049] Continue to refer to Figure 3In some cases, the gateway instance's cache management component registers a message consumption thread at startup to identify update messages it needs to process and update the local memory cache based on those update messages. As an example, broadcast objects can be characterized by topic types (i.e., representing the type of subscription). If the topic type that the gateway instance needs to process is the same as the topic type in the change message, then the update message is determined to be the update message that the gateway instance needs to process. (Continue to refer to...) Figure 4 After the gateway instance successfully updates the first cache it holds based on the update message, the first cache can return a message indicating that the update was successful to the gateway instance.

[0050] In some cases, the same change message may contain multiple second model service configurations, and these multiple second model service configurations may have the same change type. Therefore, packaging multiple second model service configurations with the same change type into a single change message for broadcast can improve the efficiency of change broadcasting. Furthermore, for the gateway instance, it can obtain the change information of multiple second model service configurations with a single parsing, thereby improving the efficiency of updating the first cache on the gateway instance side.

[0051] In some cases, the message type may also include non-broadcast types. In these cases, the gateway instance needs to actively poll the message queue to obtain and consume messages.

[0052] In some cases, the first cache includes the local cache of the gateway instance, and the second cache includes a remote cache. The management method further includes: receiving a polling request through the configuration center, the polling request being sent by the gateway instance according to a preset period; in response to receiving the polling request through the configuration center, obtaining all model service configurations in the second cache, and returning all model service configurations in the second cache to the gateway instance; and updating the first cache held by the gateway instance based on all the model service configurations.

[0053] The preset period can be dynamically adjusted, for example, based on the actual load of the gateway instance. For instance, the configuration center can provide an interface to which the gateway instance can send polling requests. (See also...) Figure 3 This interface could be, for example, Figure 3 The interface shown may include sub-interfaces for responding to polling requests. See also... Figure 3 In a gateway instance, a polling process can be deployed to initiate polling requests and update the local memory cache based on all the model service configurations obtained.

[0054] By using the above method, on the basis of the configuration center actively broadcasting update messages, an active polling initiated by the gateway instance according to a preset period is superimposed. This polling is used to fully synchronize all model service configurations in the second cache to the first cache. Thus, even if the update message is lost or the update based on the update message fails, the high consistency of the model service configuration in the distributed architecture (i.e., the distributed environment composed of multiple gateway instances) can be strictly maintained within a preset period.

[0055] Continue to refer to Figure 4 In some cases, the step of updating the first cache held by the gateway instance based on all the model service configurations can be implemented by atomically updating the first cache held by the gateway instance based on all the model service configurations.

[0056] Atomic update refers to a replacement operation that is uninterrupted and without intermediate states. The entire update operation is indivisible; it either executes completely successfully or fails entirely, with no intermediate, incomplete states. Specifically, atomic update of the first cache means replacing all model service configurations in the first cache in a single, complete process, based on all model service configurations returned by the configuration center to the gateway instance. (Continue to refer to...) Figure 4 The first cache held by the gateway instance can be atomically updated by a background thread in the gateway instance based on all the model service configurations.

[0057] As an example, after receiving all model service configurations, the gateway instance can create a brand new cache instance representing the memory cache, then populate all model service configurations into this cache instance, and directly replace the first cache corresponding to the gateway instance with the populated result. That is, the populated result is directly used as the new first cache for subsequent access by the gateway instance. Through this replacement, atomic updates are achieved.

[0058] By using the above method, the first cache held by the gateway instance can be updated atomically, which simplifies the update operation, eliminates the need to record the update progress, and avoids the writing of dirty data caused by step-by-step updates.

[0059] In some cases, continue to refer to Figure 4 After atomically updating the first cache, the first cache can return a message of successful update to the background thread in the gateway instance, so as to notify the gateway instance to complete the update of the full model service configuration in the remote cache on the gateway instance side.

[0060] In some cases, the third model service configuration in the second cache is set with an expiration time. The third model service configuration is any model service configuration in the second cache. The management method further includes: in response to the expiration time of the third model service configuration being reached, deleting the third model service configuration from the second cache.

[0061] When writing the third model service configuration to the second cache, you can set an expiration time for the third model service configuration, that is, set TTL (Time To Live).

[0062] By combining the above method with the strategy of periodic full synchronization (i.e., updating the first cache held by the gateway instance based on polling requests) and the automatic update of the second cache based on TTL, it is possible to clear out any expired data that may exist in the second cache and also to keep the first cache consistent with the source data source (i.e., the second cache).

[0063] Based on the same concept, this paper provides a management device for model service configuration. Figure 5 This is a block diagram of a management device for model-service configuration, shown under certain circumstances. (Refer to...) Figure 3 The management device 500 for model service configuration includes: The first receiving module 501 is used to receive service requests through the gateway instance; The first query module 502 is used to query a first model service configuration that matches the business request in a first cache according to the business request through the gateway instance. The first cache is held by the gateway instance. The first model service configuration is used to support the gateway instance to route the business request to the corresponding model server. The model server is used to process the business request to obtain the processing result of the business request. The second query module 503 is used to query the first model service configuration that matches the business request in the second cache in response to the fact that the first model service configuration is not found in the first cache through the gateway instance. The second cache is shared by different gateway instances.

[0064] In some cases, the first cache includes the local cache of the gateway instance, the second cache includes a remote cache, and the management device 500 for model service configuration further includes: The synchronization module is used to synchronize the first model service configuration to the first cache of the gateway instance in response to querying the first model service configuration in the second cache through the gateway instance.

[0065] In some cases, the gateway instance includes multiple instances, and the management device 500 for model service configuration further includes: The first update module is used to update the second cache in response to changes made in the configuration center; The publishing module is used to publish change messages to the message queue through the configuration center. The change messages are used to synchronously update the first cache held by the gateway instance.

[0066] In some cases, the change message includes: Subscription type, which indicates the broadcast object of the change message; Message types, including broadcast types; The message body includes the change type and the second model service configuration.

[0067] In some cases, the first cache includes the local cache of the gateway instance, the second cache includes a remote cache, and the management device 500 for model service configuration further includes: The second receiving module is used to receive polling requests through the configuration center, and the polling requests are sent through the gateway instance according to a preset period. The third receiving module is used to, in response to receiving the polling request through the configuration center, obtain all model service configurations in the second cache, and return all model service configurations in the second cache to the gateway instance; The second update module is used to update the first cache held by the gateway instance based on all the model service configurations.

[0068] In some cases, the third model service configuration in the second cache has an expiration time, and the third model service configuration is any model service configuration in the second cache. The management device 500 for model service configuration further includes: The deletion module is used to delete the third model service configuration from the second cache through the configuration center in response to the expiration time of the third model service configuration.

[0069] Among them, regarding Figure 5 The implementation principles of each module in the model service configuration management device 500 shown can be referred to the above content, and the model service configuration management device 500 has the same technical effect as the above-described model service configuration management method.

[0070] Based on the same concept, a computer-readable medium is provided that stores a computer program thereon, wherein when the computer program is executed by a processing device, it implements the steps of the above-described method, and the computer-readable medium has the technical effects that can be achieved by implementing the above-described related methods, and the technical effects can be referred to the above content.

[0071] Based on the same concept, a computer program product is provided, including a computer program, wherein when the computer program is executed by a processor, it implements the steps of the above-described method, and the computer program product has the technical effects that can be achieved by implementing the above-described related methods, and the technical effects can be referred to the above content.

[0072] Based on the same concept, an electronic device is provided, comprising: A storage device on which computer programs are stored; A processing device is used to execute the computer program in the storage device to implement the steps of the above method, and the electronic device has the technical effects that can be achieved by implementing the above-mentioned related methods, and the technical effects can be referred to the above content.

[0073] The following is for reference. Figure 6 It shows an electronic device suitable for implementing the above method (e.g. Figure 1 The diagram shows the structure of the terminal or server in the diagram. The terminal device may include, but is not limited to, mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Tablet Personal Computers), PMPs (Portable Media Players), in-vehicle terminals (such as in-vehicle navigation terminals), and fixed terminals such as digital TVs (Televisions), desktop computers, etc. Figure 6 The electronic device shown is merely an example and should not be construed as limiting its functionality or scope of use.

[0074] like Figure 6 As shown, electronic device 600 may include a processing unit (e.g., a central processing unit, a graphics processing unit, etc.) 601, which can perform various appropriate actions and processes according to a program stored in read-only memory (ROM) 602 or a program loaded from storage device 608 into random access memory (RAM) 603. The random access memory 603 also stores various programs and data required for the operation of electronic device 600. The processing unit 601, ROM 602, and RAM 603 are interconnected via bus 604. An input / output (I / O) interface 605 is also connected to bus 604.

[0075] Typically, the following devices can be connected to the input / output interface 605: input devices 606 including, for example, a touchscreen, touchpad, keyboard, mouse, camera, microphone, accelerometer, gyroscope, etc.; output devices 607 including, for example, a liquid crystal display (LCD), speaker, vibrator, etc.; storage devices 608 including, for example, magnetic tape, hard disk, etc.; and communication devices 609. Communication device 609 allows electronic device 600 to communicate wirelessly or wiredly with other devices to exchange data. Although Figure 6 An electronic device 600 with various devices is shown; however, it should be understood that it is not required to implement or possess all of the devices shown. More or fewer devices may be implemented or possessed alternatively.

[0076] In particular, depending on certain circumstances, the processes described in the flowchart above can be implemented as computer software programs. For example, a computer program product is provided, comprising a computer program carried on a non-transitory computer-readable medium, the computer program containing program code for performing the methods shown in the flowchart. This computer program can be downloaded and installed from a network via communication device 609, or installed from storage device 608, or installed from read-only memory 602. When the computer program is executed by processing device 601, it performs the functions defined in the above-described methods.

[0077] It should be noted that the aforementioned computer-readable medium may be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium may be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM, or flash memory), optical fiber, portable compact disc read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In one case, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In another case, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. The transmitted data signal can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. The computer-readable signal medium can also be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wires, optical fibers, RF (Radio Frequency), etc., or any suitable combination thereof.

[0078] In some scenarios, terminals and servers can communicate using any currently known or future-developed network protocol, such as HTTP (Hypertext Transfer Protocol), and can interconnect with digital data communication (e.g., communication networks) of any form or medium. Examples of communication networks include Local Area Networks (LANs), Wide Area Networks (WANs), the Internet (e.g., the Internet), and peer-to-peer networks (e.g., ad-hoc peer-to-peer networks), as well as any currently known or future-developed networks.

[0079] The aforementioned computer-readable medium may be included in the aforementioned electronic device; or it may exist independently and not assembled into the electronic device.

[0080] The aforementioned computer-readable medium carries one or more programs that, when executed by the electronic device, cause the electronic device to: receive a service request through a gateway instance; query a first model service configuration matching the service request in a first cache according to the service request through the gateway instance, the first cache being held by the gateway instance, the first model service configuration being used to support the gateway instance in routing the service request to a corresponding model server, the model server being used to process the service request to obtain a processing result of the service request; and, in response to not finding the first model service configuration in the first cache through the gateway instance, query a second cache matching the service request, the second cache being shared by different gateway instances.

[0081] Computer program code for performing the above operations can be written in one or more programming languages ​​or a combination thereof. These programming languages ​​include, but are not limited to, object-oriented programming languages, as well as conventional procedural programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0082] The flowcharts and block diagrams in the accompanying figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products under various scenarios. Each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing the specified logical function. It should also be noted that, in some alternative cases, the functions indicated in the blocks may occur in a different order than those indicated in the figures. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0083] The modules mentioned above can be implemented in software or hardware. In some cases, the name of a module does not necessarily limit the module itself; for example, the first receiving module can also be described as "the module that receives service requests through a gateway instance".

[0084] The functions described above can be performed, at least in part, by one or more hardware logic components. For example, exemplary types of hardware logic components that can be used, without limitation, include: Field-Programmable Gate Arrays (FPGAs), Application-Specific Integrated Circuits (ASICs), Application-Specific Standard Parts (ASSPs), Systems on Chips (SoCs), Complex Programmable Logic Devices (CPLDs), and so on.

[0085] In this context, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0086] The above description is merely illustrative and explains the technical principles employed. Those skilled in the art should understand that the scope of this document is not limited to the specific combinations of the above-described technical features, but should also cover any combination of the above-described technical features or their equivalents without departing from the above concept.

[0087] Furthermore, while the operations are described in a specific order, this should not be construed as requiring these operations to be performed in the specific order shown or in a sequential order. Multitasking and parallel processing may be advantageous in certain contexts. Similarly, while some specific implementation details are included in the above discussion, these should not be interpreted as limiting the scope of this paper. Certain features described in the context of a single example can also be implemented in combination in a single example. Conversely, various features described in the context of a single example can also be implemented individually or in any suitable sub-combination in multiple examples.

[0088] Although this document has been described using language specific to structural features and / or methodological logic, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described above. Rather, the specific features and actions described above are merely illustrative examples of implementing the claims. Regarding the aforementioned apparatus, the specific manner in which the various modules perform their operations has already been described in detail in the section concerning the method, and will not be elaborated upon here.

Claims

1. A management method for model service configuration, comprising: Receive business requests through the gateway instance; According to the business request, the gateway instance queries the first cache for a first model service configuration that matches the business request. The first cache is held by the gateway instance. The first model service configuration is used to support the gateway instance in routing the business request to the corresponding model service. The model service is used to process the business request to obtain the processing result of the business request. If the first model service configuration is not found in the first cache through the gateway instance, the first model service configuration matching the business request is queried in the second cache, and different gateway instances share the second cache.

2. The management method according to claim 1, wherein the first cache includes the local cache of the gateway instance, the second cache includes the remote cache, and the management method further includes: In response to the first model service configuration being found in the second cache through the gateway instance, the first model service configuration is synchronized to the first cache of the gateway instance.

3. The management method according to claim 1 or 2, wherein the gateway instance includes multiple instances, and the management method further includes: In response to changes made in the configuration center, update the second cache; The configuration center publishes a change message to the message queue, which is used to synchronously update the first cache held by the gateway instance.

4. The management method according to claim 3, wherein the change message includes: Subscription type, which indicates the broadcast object of the change message; Message types, including broadcast types; The message body includes the change type and the second model service configuration.

5. The management method according to claim 3, wherein the first cache includes the local cache of the gateway instance, the second cache includes the remote cache, and the management method further includes: The configuration center receives polling requests, which are then sent by the gateway instance according to a preset period. In response to receiving the polling request through the configuration center, all model service configurations in the second cache are obtained, and all model service configurations in the second cache are returned to the gateway instance; The gateway instance updates the first cache held by the gateway instance based on all the model service configurations.

6. The management method according to claim 5, wherein the third model service configuration in the second cache is set to have an expiration time, and the third model service configuration is any model service configuration in the second cache, the management method further includes: In response to the expiration time of the third model service configuration, the third model service configuration is deleted from the second cache through the configuration center.

7. A management device for model service configuration, comprising: The first receiving module is used to receive service requests through the gateway instance; The first query module is used to query the first model service configuration that matches the business request in the first cache through the gateway instance. The first cache is held by the gateway instance. The first model service configuration is used to support the gateway instance to route the business request to the corresponding model server. The model server is used to process the business request to obtain the processing result of the business request. The second query module is used to query the first model service configuration that matches the business request in the second cache when the first model service configuration is not found in the first cache through the gateway instance. The second cache is shared by different gateway instances.

8. A computer-readable medium having a computer program stored thereon, wherein, When the computer program is executed by the processing device, it implements the steps of the management method according to any one of claims 1-6.

9. An electronic device, comprising: A storage device on which computer programs are stored; A processing device for executing the computer program in the storage device to implement the steps of the management method according to any one of claims 1-6.

10. A computer program product comprising a computer program, wherein, When executed by a processor, the computer program implements the steps of the management method according to any one of claims 1-6.

Images