Service gateway configuration method, apparatus, device, and computer-readable storage medium

Abstract

The present disclosure relates to a service gateway configuration method, device and equipment and a computer readable storage medium. The method comprises: in response to a service gateway configuration instruction, obtaining a service instance list according to a preset label based on a label selector; constructing a target port list based on a service role type corresponding to each service instance and a port corresponding to each service role type; constructing a service address list of each service role type based on an IP address and the target port list and storing the list in a service gateway. The present disclosure constructs a target port list by obtaining all ports corresponding to different service role types to achieve multi-port setting according to service role types, and constructs a service address list of each service role type based on an IP address and the target port list and stores the list in a service gateway to respond to service processing requests, so that the service gateway can select multiple service instances with different ports to process service requests, further improving the effectiveness and diversity of service processing.

Description

Technical Field

[0001] This disclosure relates to the field of computer technology, and in particular to a service gateway configuration method, apparatus, device, and computer-readable storage medium. Background Technology

[0002] With the development of large model inference (such as LLM (Large Language Model) inference) systems, inference services have gradually evolved from single service instances to distributed, multi-service role type collaborative architectures, including service role types such as Prefill and Decode.

[0003] In related technologies, service gateways either automatically discover a single port or pre-determine the IP (Internet Protocol) address to the corresponding port for access. However, accessing all service instances of a service node through this unified port would result in all service role types such as Prefill and Decode accessing through a single unified port, which cannot meet the processing needs of distributed scenarios. Alternatively, specifying the port corresponding to the IP address requires pre-specification, which is inflexible and inefficient. Summary of the Invention

[0004] To address the aforementioned technical issues, this disclosure provides a service gateway configuration method, apparatus, device, and computer-readable storage medium to resolve the problems that accessing via a single unified port cannot meet the processing needs of distributed scenarios, or that specifying the port corresponding to an IP address requires pre-specifying the port, which is inflexible and inefficient.

[0005] In a first aspect, embodiments of this disclosure provide a service gateway configuration method, which includes: responding to a service gateway configuration instruction, obtaining a service instance list according to preset tags based on a tag selector; wherein the service instance list includes at least one service instance and an IP address, service role type, and port corresponding to each service instance; constructing a target port list based on the service role type and port corresponding to each service instance; wherein the target port list includes a mapping relationship between the service role type and at least one port; constructing a service address list for each service role type based on the IP address and the target port list and storing it in the service gateway; wherein the service address list for each service role type is used to process service requests for the corresponding service role type.

[0006] In some optional implementations, constructing the target port list based on the service role type and port corresponding to each service instance includes: counting all ports corresponding to each service role type according to the service role type and port corresponding to each service instance; constructing a mapping relationship between each service role type and all ports to obtain the target port list.

[0007] In some optional implementations, after constructing the target port list, the method further includes: obtaining the number of configuration ports for each service role type based on the pre-constructed port list; obtaining the number of construction ports for each service role type based on the target port list; and reconstructing the target port list if the number of configuration ports and the number of construction ports are inconsistent.

[0008] In some optional implementations, before responding to the service gateway configuration instruction, the method further includes: responding to the service gateway startup instruction and obtaining startup parameters; obtaining at least one service role type and port configuration information corresponding to the service role type based on the startup parameters; wherein the port configuration information includes the number of configured ports; and generating the pre-built port list based on at least one service role type and port setting information corresponding to the service role type.

[0009] In some optional implementations, after constructing the target port list, the method further includes: in response to the service gateway configuration update instruction, obtaining a listening message; if the listening message is an update to the service instance list, obtaining a service instance update list; updating the target port list based on the service instance update list; and updating the service address list for each service role type based on the updated target port list and storing it in the service gateway.

[0010] In some optional implementations, before obtaining the listening message in response to the service gateway configuration update instruction, the method further includes: adding or deleting service instances in the service instance list, generating the service instance update list and the listening message; and / or changing the port corresponding to any of the service role types in the service instance list, generating the service instance update list and the listening message.

[0011] In some optional implementations, after constructing and storing the service address list for each service role type based on the IP address and the target port list to the service gateway, the method further includes: the service gateway responding to the service request and determining the target processing service; determining the target service role type based on the target processing service and obtaining the target service address from the service address list of the target service role type; and sending the service request to the corresponding target service instance based on the target service address, so that the target service instance processes the service request, generates a service result, and returns it to the service gateway.

[0012] Secondly, embodiments of this disclosure provide a service gateway configuration apparatus, comprising: a response acquisition module, configured to acquire a service instance list according to preset tags based on a tag selector in response to a service gateway configuration instruction; wherein the service instance list includes at least one service instance and an IP address, service role type, and port corresponding to each service instance; a construction module, configured to construct a target port list based on the service role type and port corresponding to each service instance; wherein the target port list includes a mapping relationship between the service role type and at least one port; and a generation and storage module, configured to generate a service address list for each service role type based on the IP address and the target port list and store it in the service gateway; wherein the service address list for each service role type is used to process service requests for the corresponding service role type.

[0013] Thirdly, embodiments of this disclosure provide an electronic device, including: Memory; Processor; and Computer programs; The computer program is stored in the memory and configured to be executed by the processor to implement the service gateway configuration method as described in the first aspect.

[0014] Fourthly, embodiments of this disclosure provide a computer-readable storage medium having a computer program stored thereon, the computer program being executed by a processor to implement the service gateway configuration method described in the first aspect.

[0015] Fifthly, embodiments of this disclosure also provide a computer program product, which includes a computer program or instructions that, when executed by a processor, implement the service gateway configuration method described above.

[0016] The methods, apparatus, devices, and computer-readable storage media provided in this disclosure, in response to service gateway configuration instructions, obtain a service instance list based on a tag selector according to preset tags. The service instance list includes at least one service instance and the corresponding IP address, service role type, and port for each service instance. A target port list is constructed based on the service role type and port for each service instance. The target port list includes a mapping relationship between service role types and at least one port. A service address list for each service role type is constructed based on the IP address and target port list and stored in the service gateway. The service address list for each service role type is used to process service requests for that service role type. This application achieves multi-port settings based on service role types by obtaining all ports corresponding to different service role types and constructing a target port list, thereby improving the flexibility and effectiveness of multi-port selection. Furthermore, by obtaining the corresponding service address list according to the service role type and storing it in the service gateway to respond to service processing requests, the service gateway can select different ports to access multiple service instances of the service node to process service requests, further improving the effectiveness and diversity of service processing. Attached Figure Description

[0017] The accompanying drawings, which are incorporated in and form a part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure.

[0018] To more clearly illustrate the technical solutions in the embodiments of this disclosure or the prior art, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 A schematic diagram illustrating an application scenario provided by an embodiment of this disclosure; Figure 2 A flowchart of a service gateway configuration method provided in this embodiment of the disclosure; Figure 3 A flowchart of a service gateway configuration method provided in another embodiment of this disclosure; Figure 4 This is a schematic diagram of the structure of the service gateway configuration device provided in the embodiments of this disclosure; Figure 5 This is a schematic diagram of the device provided in an embodiment of the present disclosure. Detailed Implementation

[0020] To better understand the above-mentioned objectives, features, and advantages of this disclosure, the solutions disclosed herein will be further described below. It should be noted that, unless otherwise specified, the embodiments and features described herein can be combined with each other.

[0021] Numerous specific details are set forth in the following description in order to provide a full understanding of this disclosure, but this disclosure may also be implemented in other ways different from those described herein; obviously, the embodiments in the specification are only some, and not all, of the embodiments of this disclosure.

[0022] This disclosure provides a service gateway configuration method, which will be described below with reference to specific embodiments.

[0023] Figure 1 This is a schematic diagram illustrating an application scenario provided by an embodiment of the present disclosure, such as... Figure 1 As shown, in this application scenario, the service gateway 100 is connected to at least one service node 200 (worker), and at least one service instance is configured on each service node, such as... Figure 1 The service instances shown are 210 (pod) to 21N, where N is a positive integer greater than 1. Different service instances can be configured with different ports according to their service role types (such as Prefill, Decode, etc.). Figure 1 Ports 220 to 22M, where M is a positive integer greater than 1, provide multiple service capabilities.

[0024] Specifically, after startup, the service gateway 100 obtains a list of service instances through a tag selector, and obtains the IP address, service role type, and port of each service instance. First, it performs statistical analysis on the ports corresponding to each service role type to construct a target port list that maps service role types to at least one port. Finally, it constructs a service address list for each service role type based on the IP address and target port list and stores it in the service gateway 100. Thus, the service gateway 100 selects a service node 200 to call one or more service instances of the service role type to handle service requests based on the service address list for each service role type.

[0025] Figure 2 This is a flowchart of a service gateway configuration method provided in an embodiment of the present disclosure. This method can be applied to... Figure 1 The application scenarios shown below. The following will be combined with... Figure 1 The application scenarios shown are for Figure 2 The following describes the service gateway configuration method, which includes the following specific steps: S201. In response to the service gateway configuration command, obtain a list of service instances based on the label selector according to preset labels; wherein, the list of service instances includes at least one service instance and the IP address, service role type and port corresponding to each service instance.

[0026] Among them, the service gateway is the unified request entry point for large model processing (such as inference) clusters. It receives service (such as inference) requests from the front end and routes them to the corresponding service role type (Prefill, Decode, etc.) backend service nodes (physical servers) according to business rules. The tag selector refers to the matching rules for filtering service instances of the same type, realizing automatic service association and service discovery. Preset tags can be selected and set according to actual application scenarios.

[0027] For example, the default label is "sglang". The label selector iterates through all service instances and checks whether each service instance has the label "sglang". Any service instance with the label "sglang" is selected to form a list of service instances. "sglang" refers to a large model inference engine (such as LLM).

[0028] In this embodiment of the disclosure, responding to a service gateway configuration command can be triggered and responded to after the service gateway starts, or it can be triggered after a preset time after the service gateway starts, depending on the application scenario. Service gateway configuration refers to the service gateway analyzing the service instance list to construct a service address list and storing it in order to respond to service requests.

[0029] In this embodiment of the disclosure, a service instance list is obtained based on a tag selector according to preset tags. The service instance list includes at least one service instance and the IP address, service role type and port corresponding to each service instance.

[0030] Among them, the IP address can uniquely identify a service node, and service instances on the same service node have the same IP address; the service role type refers to classifying service instances into different types according to service roles, such as Prefill, Decode and other service role types; a service node can include one or more ports, and each port has a corresponding numerical label. If the same service node has different numerical labels for different ports, the numerical labels for ports can be the same between different service nodes.

[0031] Specifically, a physical server (worker) can be configured as a service node with multiple service instances (pods). These service instances are then divided according to their service role types (Prefill, Decode, etc.), and each service instance is assigned a corresponding port. This allows us to obtain the port corresponding to the service role type. Therefore, for each service instance (pod), we can obtain its IP address, service role type, and port.

[0032] In other words, different service nodes are distinguished by different IP addresses, service instances on service nodes are divided according to service role types, the service role type corresponding to the service instance is obtained, and each service instance has a corresponding port, thereby determining the port corresponding to each service role type.

[0033] S202. Construct a target port list based on the service role type and port corresponding to each service instance; wherein, the target port list includes the mapping relationship between the service role type and at least one port.

[0034] In this embodiment of the disclosure, one service instance corresponds to one service role type and one service instance corresponds to one port. By statistically analyzing the service role type and port corresponding to each service instance, all ports corresponding to each service role type can be obtained, and a mapping relationship between each service role type and all ports can be constructed to obtain a target port list for each service instance.

[0035] For example, for a service instance F1 (pod), the service role type is obtained as "Prefill" and the port is "12121"; for other service instances F2 and F3 (pods), the service role type is obtained as "Decode" and the ports are "12121" and "12122" respectively. This establishes a mapping relationship between the service role type "Prefill" and the port "12121", and a mapping relationship between the service role type "Decode" and the ports "12121" and "12121", resulting in a list of target ports. Here, service instance F1 and service instances F2 and F3 are service instances on different service nodes.

[0036] S203. Construct a service address list for each service role type based on the IP address and target port list and store it in the service gateway; wherein, the service address list for each service role type is used to process service requests for the corresponding service role type.

[0037] The service address list refers to the collection of real service addresses in the service backend. By calling the service through the gateway, the service instance corresponding to the service role type can be called to provide service processing based on any service address in the service address list. For example, the inference service instance can be called to perform inference processing.

[0038] In this embodiment of the disclosure, the service address list of each service instance includes at least one service address. The service address is composed of the IP address of the service instance plus at least one port in the target port list. Thus, by combining the IP address with each port in the target port list to form multiple service addresses, a service address list is obtained and stored in the service gateway.

[0039] For example, taking the mapping relationship between the service role type "Prefill" and the port "12121" and the mapping relationship between the service role type "Decode" and the ports "12121" as described above, and obtaining the target port list, the IP addresses of service instances F1 to F3 are "10.0.0.1" and "10.0.0.2" respectively, thereby generating service addresses of "10.0.0.1:12121", "10.0.0.2:12121" and "10.0.0.2:12122" respectively.

[0040] Specifically, by combining the IP address associated with the service role type with all the ports corresponding to that service role type in the target port list, a service address list for that service role type can be obtained, and thus a service address list for all service role types can be stored in the service gateway.

[0041] Understandably, after constructing and storing the service address list for each service role type in the service gateway, the service gateway can perform service requests and scheduling based on the service address list. Specifically, for a service processing request, it obtains the service instances of several service role types required for the service processing request, retrieves the service address from the service address list based on the required service role type service instances, sends the service processing request to the corresponding service node through the service address, and processes the service request through the service instances in the service node to generate the service result and return it to the service gateway.

[0042] In this embodiment, in response to a service gateway configuration command, a service instance list is obtained based on a tag selector according to preset tags. The service instance list includes at least one service instance and the corresponding IP address, service role type, and port for each service instance. A target port list is constructed based on the service role type and port for each service instance. The target port list includes a mapping relationship between service role types and at least one port. A service address list for each service role type is constructed based on the IP address and target port list and stored in the service gateway. The service address list for each service role type is used to process service requests for the corresponding service role type. This application achieves multi-port settings based on service role types by obtaining all ports corresponding to different service role types and constructing a target port list, thereby improving the flexibility and effectiveness of multi-port selection. Furthermore, by obtaining the corresponding service address list according to the service role type and storing it in the service gateway to respond to service processing requests, the service gateway can select different ports to access multiple service instances of the service node to process service requests, further improving the effectiveness and diversity of service processing.

[0043] In some embodiments of this disclosure, a target port list is constructed based on the service role type and port corresponding to each service instance, including: counting all ports corresponding to each service role type according to the service role type and port corresponding to each service instance; constructing a mapping relationship between each service role type and all ports to obtain the target port list.

[0044] In this embodiment of the disclosure, by analyzing each service instance in the service instance list, the IP address, service role type, and port corresponding to the service instance can be obtained.

[0045] It should be noted that one service instance corresponds to one service role type. As an example, one service instance corresponds to one service role type "Prefill". As another example, one service instance corresponds to one service role type "Decode". Service instances can have the same service role type, that is, one service role type can correspond to multiple service instances. One service instance corresponds to one port. Different service instances within the same service node correspond to different ports. Different service instances between different service nodes may correspond to the same port. Therefore, one service role type can correspond to multiple ports, and different service role types can have the same port.

[0046] Specifically, based on the service role type and port corresponding to each service instance, all ports corresponding to each service role type are counted, and then the mapping relationship between each service role type and all ports is constructed to obtain the target port list.

[0047] For example, the obtained service instance list has 8 service instances F1 to F8. 3 service instances have service role types P and corresponding ports A1, A2 and A3, and 5 service instances have service role types D and corresponding ports A1, A2, A4, A5 and A6. This constructs a mapping relationship between service role type P and ports A1, A2 and A3, as well as a mapping relationship between service role type D and ports A1, A2, A4, A5 and A6, resulting in a target port list. Among these 8 service instances F1 to F8, there are service instances with different service nodes.

[0048] In this embodiment of the disclosure, by constructing a mapping relationship between service role types and all corresponding ports, multiple ports can be associated according to service role types, thereby further improving the accuracy of service discovery and enhancing the effectiveness and diversity of service processing.

[0049] In some embodiments of this disclosure, after constructing the target port list, the service gateway configuration method further includes: obtaining the number of configuration ports for each service role type based on the pre-constructed port list; obtaining the number of construction ports for each service role type based on the target port list; and reconstructing the target port list if the number of configuration ports and the number of construction ports are inconsistent.

[0050] In this embodiment of the disclosure, after constructing the target port list, that is, associating each service role type with at least one port, in order to further improve the accuracy of the subsequent service address generation, the target port list can be further verified. If the verification is inconsistent, the target port list is regenerated to further avoid errors in subsequent processing and improve the service processing effect.

[0051] Specifically, the number of configured ports for each service role type is obtained based on the pre-built port list, and the number of built ports for each service role type is obtained based on the target port list. If the number of configured ports and the number of built ports are inconsistent, the target port list is rebuilt.

[0052] The pre-built port list includes the number of configured ports, which refers to the number of ports actually configured for each service role type based on the service gateway startup parameters; the number of built ports refers to the number of ports existing for each role type in the target port list obtained by statistical analysis of the target port list.

[0053] Specifically, if the number of configured ports and the number of constructed ports are inconsistent, it means that there may be cases of added or missing ports in the target port list, and the target port list needs to be reconstructed.

[0054] It should be noted that if the number of configured ports and the number of constructed ports are the same, no processing is performed to further improve processing efficiency.

[0055] Specifically, if the number of configured ports and the number of constructed ports are inconsistent, it may be due to an incorrect list of service instances. Therefore, the label selector can be triggered to re-obtain the list of service instances according to the preset labels to further ensure the correctness of the data. Furthermore, a prompt message can be generated or the service gateway can be restarted to respond to the service gateway configuration command to update the target port list, further improving the flexibility of the process.

[0056] In this embodiment of the disclosure, after constructing the target port list, the number of configured ports for each service role type in the pre-constructed port list can be flexibly selected to further verify the number of constructed ports for each service role type obtained from the target port list. If there is an inconsistency, the target port list can be reconstructed to further ensure the correctness of the subsequently generated service address list and ultimately improve the accuracy of service processing.

[0057] In some embodiments of this disclosure, before responding to the service gateway configuration instruction, the service gateway configuration method further includes: responding to the service gateway startup instruction and obtaining startup parameters; obtaining at least one service role type and port configuration information corresponding to the service role type based on the startup parameters; wherein the port configuration information includes the number of configured ports; and generating a pre-built port list based on at least one service role type and port setting information corresponding to the service role type.

[0058] In this embodiment of the disclosure, when the service gateway starts, a service gateway startup command is triggered and responded to, startup parameters are obtained, and the service role type and the port configuration information corresponding to the service role type are obtained from the startup parameters, such as the number of configured ports and the numerical labels corresponding to the configured ports, thereby constructing an association relationship between at least one service role type and the port setting information corresponding to the service role type, and generating a pre-built port list.

[0059] In this embodiment of the disclosure, startup parameters can be read directly through a preset command, which can be selected and set according to the actual application scenario.

[0060] Specifically, after obtaining the startup parameters, identifying the startup parameters can reveal each service role type and the corresponding port configuration information, such as the number of configured ports and the numerical labels corresponding to the configured ports.

[0061] In this embodiment of the disclosure, by reading startup parameters when the service gateway starts to obtain service role types and corresponding port configuration information to construct a port list, the service role types and corresponding port settings of all service nodes connected to the service gateway can be determined, and the correctness of the generated target port list can be further determined based on the pre-constructed port list, thereby further improving the effectiveness of the service gateway configuration.

[0062] In some embodiments of this disclosure, after constructing the target port list, the service gateway configuration method further includes: in response to a service gateway configuration update instruction, obtaining a listening message; if the listening message is an update to the service instance list, obtaining a service instance update list; updating the target port list based on the service instance update list; and updating the service address list for each service role type based on the updated target port list and storing it in the service gateway.

[0063] It is understandable that the service nodes connected to the service gateway, the service instances on the service nodes, the service role types corresponding to the service instances, and the ports corresponding to the service instances may all change. Therefore, when the aforementioned changes occur, the service gateway configuration update command is triggered and responded to, thereby obtaining the listening message; the listening message is used to indicate the specific update content.

[0064] In this embodiment of the disclosure, after obtaining the listening message, the listening message is parsed, and different processing methods are selected according to different update content. As an example, when the listening message is an update of the service instance list, the service instance update list is obtained, the target port list is updated based on the service instance update list, and the service address list of each service role type is updated based on the updated target port list and stored in the service gateway. That is, the service gateway configuration information according to this embodiment of the disclosure is reconfigured according to the new service instance update list in order to update and store the service address list of each service role type.

[0065] In this embodiment of the disclosure, there are many scenarios for updating the service instance list. These scenarios include adding or deleting service nodes, adding or deleting service instances for a specific service node, changing the service role type of a service instance, and changing the port corresponding to a service instance. This allows for updating mechanisms that cover more scenarios and further ensures the stability and effectiveness of the service gateway configuration.

[0066] In this embodiment of the disclosure, when the service instance update list is determined to be updated by obtaining the listening message, the service gateway configuration information according to this embodiment of the disclosure is reconfigured according to the new service instance update list, so as to update and store the service address list of each service role type, thereby further improving the effectiveness of the service gateway configuration.

[0067] In some embodiments of this disclosure, before obtaining the listening message in response to the service gateway configuration update instruction, the service gateway configuration method further includes: adding or deleting service instances in the service instance list, generating a service instance update list and listening messages; and / or, changing the port corresponding to any service role type in the service instance list, generating a service instance update list and listening messages.

[0068] Specifically, scenarios such as adding or deleting service instances, changing the service role type of a service instance, or changing the port corresponding to a service instance can all be understood as changing the port corresponding to any service role type. Timely generation of service instance update lists and listening messages further ensures the stability and effectiveness of the service gateway configuration.

[0069] In this embodiment of the disclosure, by promptly generating listening messages and a service instance update list in response to updates to service instances or ports corresponding to service role types, the real-time effectiveness of the subsequently generated target port list is ensured, further guaranteeing the stability and effectiveness of the service gateway configuration.

[0070] In some embodiments of this disclosure, after constructing and storing the service address list for each service role type based on the IP address and target port list to the service gateway, the service gateway configuration method further includes: the service gateway responding to a service request and determining the target processing service; determining the target service role type based on the target processing service and obtaining the target service address from the service address list of the target service role type; and sending the service request to the corresponding target service instance based on the target service address, so that the target service instance processes the service request, generates a service result, and returns it to the service gateway.

[0071] Specifically, the service gateway can perform service request scheduling and distribution. After receiving a service request, it can parse the service request to obtain the target processing service, such as a population service or an encoding service. Based on the target processing service, it can determine the target service role type, such as a population service role type or an encoding service role type. It can then obtain the target service address from the service address list of the target service role type. Finally, based on the target service address, it sends the service request to the corresponding target service instance so that the target service instance can process the service request, generate the service result, and return it to the service gateway.

[0072] For example, in the "Prefill" stage, the target service instance of the service node is selected from the service address list of service role type "Prefill" for processing; in the "Decode" stage, the target service instance of the service node is selected from the service address list of service role type "Decode" for processing.

[0073] It is understandable that there are many ways to obtain the target service address from the list of service addresses of the target service role type. In other words, when there are multiple service nodes that can handle the service request, a service instance on a service node can be randomly selected as the target service instance to process the service request, generate the service result, and return it to the service gateway. In order to improve the efficiency and effectiveness of the service, a service instance on a target service node can be determined as the target service instance based on the node information of the service node, such as real-time load, historical processing accuracy, memory, etc., to process the service request, generate the service result, and return it to the service gateway, thus ensuring the stability of the processing.

[0074] In this embodiment of the disclosure, the service gateway responds to service requests based on a stored list of service addresses for each service role type, and can select multiple service instances on different ports to process service requests, thereby further improving the effectiveness and diversity of service processing.

[0075] Based on the foregoing embodiments, in many application scenarios, multiple service instances are set up on the same service node, and each service instance needs to correspond to a different port to meet the usage requirements. The existing single unified port access method cannot meet the processing requirements of distributed scenarios, or the method of specifying the port corresponding to the IP needs to specify the port corresponding to the IP in advance, which is not flexible and has low efficiency. Alternatively, different service instances are deployed separately on different service nodes, which results in resource waste and the port must be unified. Ultimately, the service gateway cannot correctly perceive all available service addresses and cannot meet the service processing requirements.

[0076] This disclosure embodiment constructs a target port list by obtaining all ports corresponding to different service role types to realize multi-port settings divided according to service role types, thereby improving the flexibility and effectiveness of multi-port selection. It also obtains a list of corresponding service addresses according to service role types and stores it in the service gateway to respond to service processing requests. This allows the service gateway to select different ports to access multiple service instances of service nodes to process service requests, further improving the effectiveness and diversity of service processing.

[0077] Figure 3 A flowchart of a service gateway configuration method provided in another embodiment of this disclosure is shown below. Figure 3 As shown, the service gateway configuration method includes: Step 301: Respond to the service gateway startup command, obtain startup parameters, and obtain at least one service role type and port configuration information corresponding to the service role type based on the startup parameters; wherein, the port configuration information includes the number of configured ports; generate a pre-built port list based on at least one service role type and port setting information corresponding to the service role type.

[0078] Specifically, when the service gateway starts, a service gateway startup command is triggered and responded to, startup parameters are obtained, and the service role type and the port configuration information corresponding to the service role type are obtained from the startup parameters, such as the number of configured ports and the numerical labels corresponding to the configured ports, thereby constructing at least one association relationship between the service role type and the port setting information corresponding to the service role type, and generating a pre-built port list.

[0079] Step 302: In response to the service gateway configuration command, obtain a list of service instances based on the tag selector according to preset tags; wherein, the list of service instances includes at least one service instance and the IP address, service role type and port corresponding to each service instance.

[0080] Step 303: Based on the service role type and port corresponding to each service instance, count all ports corresponding to each service role type, construct the mapping relationship between each service role type and all ports, and obtain the target port list.

[0081] Specifically, the service instance list is obtained based on the tag selector according to the preset tags. The service instance list includes at least one service instance and the IP address, service role type and port corresponding to each service instance. Based on the service role type and port corresponding to each service instance, all ports corresponding to each service role type are counted. Then, the mapping relationship between each service role type and all ports is constructed to obtain the target port list.

[0082] Step 304: Obtain the number of configured ports for each service role type based on the pre-built port list, and obtain the number of built ports for each service role type based on the target port list. If the number of configured ports and the number of built ports are inconsistent, rebuild the target port list.

[0083] Specifically, the number of configured ports for each service role type is obtained based on the pre-built port list, and the number of built ports for each service role type is obtained based on the target port list. If the number of configured ports and the number of built ports are inconsistent, the target port list is rebuilt.

[0084] Step 305: Construct a service address list for each service role type based on the IP address and target port list and store it in the service gateway; wherein, the service address list for each service role type is used to process service requests for the corresponding service role type.

[0085] Specifically, after constructing a service address list for each service role type and storing it in the service gateway, the service gateway can process service requests, obtain the service instance of the service role type required for the service processing request, and retrieve the service address from the service address list based on the required service role type service instance. The service gateway then sends the service processing request to the corresponding service node through the service address, and processes the service request through the service instance in the service node to generate the service result and return it to the service gateway.

[0086] Step 306: Add or delete service instances in the service instance list, generate a service instance update list and listening messages; and / or, change the port corresponding to any service role type in the service instance list, generate a service instance update list and listening messages.

[0087] Specifically, scenarios such as adding or deleting service instances, changing the service role type of a service instance, or changing the port corresponding to a service instance can all be understood as changing the port corresponding to any service role type. Service instance update lists and listening messages should be generated in a timely manner.

[0088] Step 307: In response to the service gateway configuration update command, obtain the listening message. If the listening message is an update of the service instance list, obtain the service instance update list, update the target port list based on the service instance update list, update the service address list of each service role type based on the updated target port list, and store it in the service gateway.

[0089] Specifically, after obtaining the listening message, the listening message is parsed. If the listening message is an update of the service instance list, the updated service instance list is obtained, the target port list is updated based on the updated service instance list, and the service address list of each service role type is updated based on the updated target port list and stored in the service gateway.

[0090] Step 308: The service gateway responds to the service request, determines the target processing service, determines the target service role type based on the target processing service, obtains the target service address from the service address list of the target service role type, and sends the service request to the corresponding target service instance based on the target service address, so that the target service instance processes the service request, generates the service result, and returns it to the service gateway.

[0091] Specifically, after receiving a service request, the service request can be parsed to obtain the target processing service. Based on the target processing service, the target service role type can be determined. The target service address can be obtained from the service address list of the target service role type. Finally, the service request is sent to the corresponding target service instance based on the target service address, so that the target service instance can process the service request, generate the service result, and return it to the service gateway.

[0092] In this embodiment of the disclosure, the service gateway is supported to automatically discover multiple port registrations of the same service node, and the service gateway is supported to automatically discover different port sets of different service role types.

[0093] For example, service instances are divided according to service role types, and each service instance has a corresponding port, that is, the port specifically corresponding to the service role type. For example, the service role type "Prefill" corresponds to port "12121", and the service role type "Decode" corresponds to ports "12121" and "12122".

[0094] As an example scenario, a list of service instances is obtained based on a tag selector. For each service instance, its IP address, service role type, and port are retrieved, and a target port list is constructed, such as Prefill_ports=

[12121] , Decode_ports = [12121, 12122] Next, based on the IP address and target port list, a service address list is constructed for each service role type: "Prefill" is "10.0.0.1:12121"; "Decode" is "10.0.0.2:12121" and "10.0.0.2:12122". The service gateway performs different processing according to the request stage. In the "Prefill" stage, a service node is selected from "10.0.0.1:12121" to process the service request through the corresponding service instance, generate the service result, and return it to the service gateway. In the "Decode" stage, a service node is selected from "10.0.0.2:12121" and "10.0.0.2:12122" to process the service request through the corresponding service instance, generate the service result, and return it to the service gateway.

[0095] It should be noted that, in scenarios without multi-port settings, the embodiments disclosed herein are also applicable to single-port scenarios, further ensuring processing in different scenarios and meeting the usage needs of different scenarios.

[0096] In this embodiment of the disclosure, a target port list is constructed by obtaining all ports corresponding to different service role types to realize multi-port settings according to service role types, thereby improving the flexibility and effectiveness of multi-port selection. The corresponding service address list is obtained according to the service role type and stored in the service gateway to respond to service processing requests. This allows the service gateway to select different ports to access multiple service instances of the service node to process service requests, further improving the effectiveness and diversity of service processing.

[0097] Figure 4 This is a schematic diagram of a service gateway configuration device provided in an embodiment of this disclosure. The device provided in this embodiment can execute the processing flow provided in the method embodiment, such as... Figure 4 As shown, the service gateway configuration device 40 includes: a response acquisition module 41, a construction module 42, and a generation and storage module 43; The response acquisition module 41 is used to obtain a list of service instances based on a preset tag selector in response to a service gateway configuration command; wherein the list of service instances includes at least one service instance and the IP address, service role type and port corresponding to each service instance; The construction module 42 is used to construct the target port list based on the service role type and port corresponding to each service instance; wherein, the target port list includes the mapping relationship between the service role type and at least one port; The generation and storage module 43 is used to generate a service address list for each service role type based on the IP address and the target port list and store it in the service gateway; wherein, the service address list for each service role type is used to process service requests for the corresponding service role type.

[0098] In some embodiments of this disclosure, the construction module 42 is specifically used to: count all ports corresponding to each service role type based on the service role type and port corresponding to each service instance; construct a mapping relationship between each service role type and all ports to obtain the target port list.

[0099] In some embodiments of this disclosure, after constructing the target port list, the apparatus further includes: a first acquisition module, configured to acquire the number of configuration ports for each service role type based on the pre-constructed port list; a second acquisition module, configured to acquire the number of construction ports for each service role type based on the target port list; and a reconstruction module, configured to reconstruct the target port list if the number of configuration ports and the number of construction ports are inconsistent.

[0100] In some embodiments of this disclosure, prior to responding to the service gateway configuration instruction, the apparatus further includes: a response parameter acquisition module, configured to acquire startup parameters in response to the service gateway startup instruction; a third acquisition module, configured to acquire at least one service role type and port configuration information corresponding to the service role type based on the startup parameters; wherein the port configuration information includes the number of configured ports; and a pre-construction module, configured to generate the pre-constructed port list based on at least one service role type and port setting information corresponding to the service role type.

[0101] In some embodiments of this disclosure, after constructing the target port list, the apparatus further includes: a response acquisition message module, configured to acquire a listening message in response to the service gateway configuration update instruction; a fourth acquisition module, configured to acquire a service instance update list if the listening message is an update of the service instance list; a first update module, configured to update the target port list based on the service instance update list; and a second update module, configured to update the service address list of each service role type based on the updated target port list and store it in the service gateway.

[0102] In some embodiments of this disclosure, before obtaining the listening message in response to the service gateway configuration update instruction, the apparatus further includes: a generation module, configured to add or delete service instances in the service instance list, generate the service instance update list and the listening message; and / or, the generation module is further configured to change the port corresponding to any of the service role types in the service instance list, generate the service instance update list and the listening message.

[0103] In some embodiments of this disclosure, after constructing and storing the service address list for each service role type based on the IP address and the target port list to the service gateway, the apparatus further includes: a processing module, configured to: respond to the service request by the service gateway, determine a target processing service, determine a target service role type based on the target processing service, obtain a target service address from the service address list of the target service role type, and send the service request to the corresponding target service instance based on the target service address, so that the target service instance processes the service request, generates a service result, and returns it to the service gateway.

[0104] Figure 4 The service gateway configuration device shown in the embodiment can be used to execute the technical solution of the above service gateway configuration method embodiment. Its implementation principle and technical effect are similar, and will not be repeated here.

[0105] Figure 5 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this disclosure. The electronic device can be an upgradeable device as described in the above embodiments. The electronic device provided in this disclosure can execute the processing flow provided in the service gateway configuration method embodiment, such as... Figure 5 As shown, device 50 includes: memory 51, processor 52, computer program and communication interface 53; wherein the computer program is stored in memory 51 and configured to be executed by processor 52 as described above.

[0106] In addition, this disclosure also provides a computer-readable storage medium having a computer program stored thereon, the computer program being executed by a processor to implement the method described in the above embodiments.

[0107] Furthermore, this disclosure also provides a computer program product comprising a computer program or instructions that, when executed by a processor, implement the method described above.

[0108] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0109] The above description is merely a specific embodiment of this disclosure, enabling those skilled in the art to understand or implement it. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this disclosure. Therefore, this disclosure is not to be limited to the embodiments described herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A service gateway configuration method, characterized in that, The method includes: In response to a service gateway configuration command, a list of service instances is obtained based on a tag selector according to preset tags; wherein, the list of service instances includes at least one service instance and the IP address, service role type and port corresponding to each service instance; A target port list is constructed based on the service role type and port corresponding to each service instance; wherein, the target port list includes the mapping relationship between the service role type and at least one port; Based on the IP address and the target port list, a service address list for each service role type is constructed and stored in the service gateway; wherein, the service address list for each service role type is used to process service requests for the corresponding service role type.

2. The service gateway configuration method according to claim 1, characterized in that, The construction of the target port list based on the service role type and port corresponding to each service instance includes: Based on the service role type and port corresponding to each service instance, count all ports corresponding to each service role type; Construct a mapping relationship between each service role type and all ports to obtain the target port list.

3. The service gateway configuration method according to claim 1, characterized in that, After constructing the target port list, the method further includes: The number of configured ports for each service role type is obtained based on the pre-built port list; Based on the target port list, obtain the number of construction ports for each service role type; If the number of configured ports and the number of constructed ports are inconsistent, the target port list shall be reconstructed.

4. The service gateway configuration method according to claim 3, characterized in that, Prior to responding to the service gateway configuration instruction, the method further includes: Respond to the service gateway startup command and obtain startup parameters; Based on the startup parameters, at least one service role type and port configuration information corresponding to the service role type are obtained; wherein, the port configuration information includes the number of configured ports; The pre-built port list is generated based on at least one of the service role types and the port setting information corresponding to the service role type.

5. The service gateway configuration method according to claim 1, characterized in that, After constructing the target port list, the method further includes: In response to the service gateway configuration update command, obtain the listening message; If the listening message is an update to the service instance list, obtain the updated service instance list; Update the target port list based on the service instance update list; The service address list for each service role type is updated based on the updated target port list and stored in the service gateway.

6. The service gateway configuration method according to claim 5, characterized in that, Before obtaining the listening message in response to the service gateway configuration update instruction, the method further includes: Adding or deleting service instances from the service instance list, generating an updated service instance list and the listening message; and / or, Change the port corresponding to any of the service role types in the service instance list to generate the service instance update list and the listening message.

7. The service gateway configuration method according to claim 1, characterized in that, After constructing the service address list for each service role type based on the IP address and the target port list and storing it in the service gateway, the method further includes: The service gateway responds to the service request and determines the target processing service; The target service role type is determined based on the target processing service, and the target service address is obtained from the service address list of the target service role type. The service request is sent to the corresponding target service instance based on the target service address, so that the target service instance processes the service request, generates a service result, and returns it to the service gateway.

8. A service gateway configuration device, characterized in that, The service gateway configuration device includes: The response acquisition module is used to obtain a list of service instances according to preset tags based on a tag selector in response to a service gateway configuration command; wherein, the list of service instances includes at least one service instance and the IP address, service role type and port corresponding to each service instance; A construction module is used to construct a target port list based on the service role type and port corresponding to each service instance; wherein, the target port list includes a mapping relationship between the service role type and at least one port; A storage generation module is used to generate a service address list for each service role type based on the IP address and the target port list, and store it in the service gateway; wherein, the service address list for each service role type is used to process service requests for the corresponding service role type.

9. An electronic device, characterized in that, include: Memory; processor; as well as Computer programs; The computer program is stored in the memory and configured to be executed by the processor to implement the method as described in any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1 to 7.

Images