Service calling method, device and electronic device

Abstract

The present invention provides a service calling method, device, and electronic device, relating to the field of artificial intelligence technology. The service calling method includes: receiving an inference service calling request encapsulated in a first programming language and sent by a microservice gateway; modifying the recipient address in the inference service calling request to an address corresponding to an inference service module encapsulated and generated in a second programming language; sending the modified inference service calling request to the inference service module; receiving an inference result of the modified inference service calling request sent by the inference service module; and sending the inference result to the microservice gateway. In an embodiment of the present invention, by using a proxy service module to proxy the inference service of the inference service module encapsulated and generated in a second programming language under a microservice architecture of a first programming language system, it is possible to implement an inference service corresponding to an inference model encapsulated in the Python language under a microservice architecture of a JAVA system.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a service calling method, device and electronic device. Background Art

[0002] Microservices architecture is a commonly used service architecture. It's a new technology for deploying applications and services in the cloud. Currently, Java-based microservices architectures are relatively mature, with numerous open source components available. However, inference models, such as TensorFlow models, are typically encapsulated in Python. Therefore, inference services corresponding to these models cannot be implemented within Java-based microservices architectures. Summary of the Invention

[0003] Embodiments of the present invention provide a service calling method, device, and electronic device to solve the problem that an inference service corresponding to an inference model cannot be implemented under a microservice architecture of a JAVA system.

[0004] To solve the above-mentioned technical problems, the present invention is achieved as follows:

[0005] In a first aspect, an embodiment of the present invention provides a service invocation method, the method comprising:

[0006] Receive an inference service call request encapsulated in a first programming language and sent by a microservice gateway;

[0007] Modify the recipient address in the inference service call request to an address corresponding to the inference service module generated based on the second programming language encapsulation;

[0008] Sending the modified reasoning service call request to the reasoning service module;

[0009] receiving an inference result of the modified inference service call request sent by the inference service module;

[0010] Send the inference result to the microservice gateway.

[0011] Optionally, receiving the inference service call request encapsulated in the first programming language and sent by the microservice gateway includes:

[0012] Receive an inference service call request encapsulated in a first programming language and sent by the microservice gateway from the first port;

[0013] The sending the modified inference service call request to the inference service module includes:

[0014] Send the modified reasoning service call request to the second port of the reasoning service module.

[0015] Optionally, the method further includes:

[0016] Send a registration request to the microservice registration center, the registration request including the service address and service name;

[0017] Receive a registration response from the microservice registration center to the registration request.

[0018] Optionally, the method further includes:

[0019] Requesting the inference service module to perform service status detection at preset time intervals and obtaining service status detection results;

[0020] Send the service status detection result to the microservice registration center.

[0021] In a second aspect, an embodiment of the present invention provides a service invocation method, the method comprising:

[0022] Receiving an inference service call request sent by a proxy service module generated based on the first programming language encapsulation;

[0023] Based on the inference service call request, the inference model encapsulated in the second programming language is called to perform inference, and the inference result is sent to the proxy service module.

[0024] Optionally, the invoking a reasoning model encapsulated in a second programming language to perform reasoning based on the reasoning service call request includes:

[0025] Based on the inference service call request, a target inference process is determined from N inference processes started on the graphics card, and based on the target inference process, an inference model encapsulated in the second programming language is called to perform inference, where N is an integer greater than 1.

[0026] Optionally, the inference model is loaded on the graphics card with preset video memory, and N is determined based on the ratio of video memory occupancy to the total video memory, the video memory occupancy is the video memory occupied when the inference model is loaded on the graphics card using the video memory occupancy self-growth mode, and the total video memory is the total video memory of the graphics card.

[0027] In a third aspect, an embodiment of the present invention provides a service invocation device, the service invocation device comprising:

[0028] A first receiving unit is configured to receive an inference service call request encapsulated in a first programming language and sent by a microservice gateway;

[0029] a modification unit, configured to modify the recipient address in the inference service call request to an address corresponding to the inference service module generated based on the encapsulation of the second programming language;

[0030] A first sending unit, configured to send a modified inference service call request to the inference service module;

[0031] A second receiving unit is configured to receive an inference result of the modified inference service call request sent by the inference service module;

[0032] The second sending unit is used to send the inference result to the microservice gateway.

[0033] Optionally, the first receiving unit is specifically configured to:

[0034] Receive an inference service call request encapsulated in a first programming language and sent by the microservice gateway from the first port;

[0035] The first sending unit is specifically configured to:

[0036] Send the modified reasoning service call request to the second port of the reasoning service module.

[0037] Optionally, the service calling device further includes:

[0038] The third sending unit is used to send a registration request to the microservice registration center, where the registration request includes a service address and a service name;

[0039] The third receiving unit is configured to receive a registration response from the microservice registration center to the registration request.

[0040] Optionally, the service calling device further includes:

[0041] an acquiring unit, configured to request the inference service module to perform service status detection at a preset time interval and acquire a service status detection result;

[0042] The fourth sending unit is used to send the service status detection result to the microservice registration center.

[0043] In a fourth aspect, an embodiment of the present invention provides a service invocation device, the service invocation device comprising:

[0044] A receiving unit, configured to receive an inference service call request sent by a proxy service module generated based on encapsulation in the first programming language;

[0045] The reasoning unit is configured to call the reasoning model encapsulated in the second programming language to perform reasoning based on the reasoning service call request, and send the reasoning result to the proxy service module.

[0046] Optionally, the reasoning unit is specifically configured to:

[0047] Based on the inference service call request, a target inference process is determined from the N inference processes started on the graphics card, and based on the target inference process, an inference model encapsulated in the second programming language is called to perform inference, and the inference result is sent to the proxy service module, where N is an integer greater than 1.

[0048] Optionally, the inference model is loaded on the graphics card with preset video memory, and N is determined based on the ratio of video memory occupancy to the total video memory, the video memory occupancy is the video memory occupied when the inference model is loaded on the graphics card using the video memory occupancy self-growth mode, and the total video memory is the total video memory of the graphics card.

[0049] In the fifth aspect, an embodiment of the present invention provides an electronic device, comprising: a processor, a memory, and a program stored on the memory and runnable on the processor, wherein when the program is executed by the processor, the steps of the service calling method as described in the first aspect are implemented, or when the program is executed by the processor, the steps of the service calling method as described in the second aspect are implemented.

[0050] In an embodiment of the present invention, a microservice gateway receives an inference service call request encapsulated in a first programming language; the recipient address in the inference service call request is modified to the address corresponding to the inference service module encapsulated and generated in a second programming language; the modified inference service call request is sent to the inference service module; the inference result of the modified inference service call request is received from the inference service module; and the inference result is sent to the microservice gateway. In this way, by using a proxy service module to proxy the inference service of the inference service module encapsulated and generated in the second programming language within the microservice architecture of the first programming language system, the inference service corresponding to the inference model encapsulated in the Python language can be implemented within the microservice architecture of the JAVA system. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in describing the embodiments of the present invention. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0052] Figure 1 This is a flow chart of a service calling method provided by an embodiment of the present invention;

[0053] Figure 2 is a structural diagram of an electronic device provided by an embodiment of the present invention;

[0054] Figure 3is a flowchart of another service calling method provided by an embodiment of the present invention;

[0055] Figure 4 This is a structural diagram of a service calling device provided by an embodiment of the present invention;

[0056] Figure 5 It is a structural diagram of another service calling device provided by an embodiment of the present invention;

[0057] Figure 6 It is a structural diagram of another electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0058] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of them. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0059] Embodiments of the present invention provide a service calling method, device, and electronic device to solve the problem that an inference service corresponding to an inference model cannot be implemented under a microservice architecture of a JAVA system.

[0060] See also Figure 1 , Figure 1 This is a flow chart of a service calling method provided by an embodiment of the present invention. Figure 1 As shown, the method includes the following steps:

[0061] Step 101: Receive an inference service call request encapsulated in a first programming language and sent by a microservice gateway.

[0062] The service call method can be executed by a proxy service module to implement a proxy reasoning service. The microservice gateway can receive a reasoning service call request sent by a client and forward the received reasoning service call request to the proxy service module.

[0063] In addition, the client can initiate an inference service call request. Specifically, the client can find the service address of the inference service in the microservice registry. The service address of the inference service in the microservice registry is the network address and first port of the proxy service module. Based on the network address and first port of the proxy service module, the client sends the inference service call request to the proxy service module through the microservice gateway.

[0064] Step 102: Modify the recipient address in the inference service call request to an address corresponding to the inference service module generated based on the second programming language encapsulation.

[0065] Among them, the first programming language and the second programming language are different programming languages. For example, the first programming language may be Java, and the second programming language may be Python. The proxy service module may be a module for executing a Java proxy service. The inference service module may be a module for executing an inference service. The Java proxy service may occupy a first port, and in the configuration parameters of the Java proxy service, the second port occupied by the inference service proxied by the Java proxy service and the service status detection interface address of the inference service may be configured. After receiving the inference service call request from the first interface, the Java proxy service may forward the inference service call request to the second port. The request content of the inference service call request is determined based on the inference model. Taking the inference model as a model for recognizing text on a picture as an example, the inference service call request may be an HTTP post request, and the body content in the HTTP post request is a picture.

[0066] In addition, the inference service module can externally call the inference interface and the service status detection interface. The inference service module receives these two interface call request messages on a second port through a Python-encapsulated gunicorn service. The function of the inference interface is determined by the inference model. For example, if the inference model is a model for recognizing text in an image, when calling the inference interface, an image with text can be passed as input. After calling the inference interface, the output is the text on the image.

[0067] Step 103: Send the modified reasoning service call request to the reasoning service module.

[0068] Among them, the proxy service module can send a modified inference service call request to the inference service module. Specifically, after the java proxy service receives the inference service call request from the first interface, it modifies the recipient address in the inference service call request to the address corresponding to the inference service module, and forwards the inference service call request to the second port of the gunicorn service.

[0069] Step 104: Receive the inference result of the modified inference service call request sent by the inference service module.

[0070] Among them, the reasoning service module can receive an reasoning service call request sent by a proxy service module generated based on the first programming language encapsulation; the reasoning service module can call the reasoning model encapsulated based on the second programming language based on the reasoning service call request to perform reasoning, and send the reasoning result to the proxy service module.

[0071] In addition, the inference model can be a TensorFlow model encapsulated in Python. After receiving the inference service call request from the Java proxy service, the gunicorn service can randomly select an inference process from the idle Python inference processes and process the inference service call request based on the inference process. For example, if the inference service call request is used to request the recognition of text on an image, the inference service call request includes an image with text. The inference process inputs the image with text in the service call request into the TensorFlow model, invokes the TensorFlow model for inference processing, and returns the output of the TensorFlow model, i.e., the text on the image, to the Java proxy service.

[0072] Step 105: Send the inference result to the microservice gateway.

[0073] Among them, after receiving the inference result, the Java proxy service can return the inference result to the microservice gateway, and can return the inference result to the microservice gateway in JSON format, and the microservice gateway pushes the result back to the client.

[0074] As a specific embodiment, the electronic device may include a graphics card, and the graphics card may be provided with only one GPU, such as Figure 2 As shown, the graphics card runs a Java proxy service, a Gunicorn service, and multiple Python inference processes. Each Python inference process corresponds to a TensorFlow model, and the Python inference process can call the TensorFlow model for inference. The client sends an inference service call request to the Java proxy service through the microservice gateway. The Java proxy service sends the inference service call request received on the first port to the Gunicorn service's second port. After receiving the inference service call request from the Java proxy service, the Gunicorn service randomly selects an inference process from the idle Python inference processes and calls the inference model for inference based on the selected inference process.

[0075] It should be noted that in existing technologies, TensorFlow models used for inference are typically encapsulated in Python. Therefore, inference services corresponding to these models cannot be implemented within a Java-based microservices architecture. In this embodiment, a Java proxy service forwards inference service call requests, enabling the TensorFlow model trained using the TensorFlow framework to be encapsulated into an online inference service and registered with the Spring Cloud microservices framework. This allows the integration of inference services developed in Python into a Java-based microservices architecture.

[0076] In an embodiment of the present invention, a microservice gateway receives an inference service call request encapsulated in a first programming language; the recipient address in the inference service call request is modified to the address corresponding to the inference service module encapsulated and generated in a second programming language; the modified inference service call request is sent to the inference service module; the inference result of the modified inference service call request is received from the inference service module; and the inference result is sent to the microservice gateway. In this way, by using a proxy service module to proxy the inference service of the inference service module encapsulated and generated in the second programming language within the microservice architecture of the first programming language system, the inference service corresponding to the inference model encapsulated in the Python language can be implemented within the microservice architecture of the JAVA system.

[0077] Optionally, receiving the inference service call request encapsulated in the first programming language and sent by the microservice gateway includes:

[0078] Receive an inference service call request encapsulated in a first programming language and sent by the microservice gateway from the first port;

[0079] The sending the modified inference service call request to the inference service module includes:

[0080] Send the modified reasoning service call request to the second port of the reasoning service module.

[0081] The inference service call request may be a service call request encapsulated using the HTTP protocol. The proxy service module may support forwarding HTTP inference service call requests in both GET and POST formats. The HTTP-encapsulated inference service call request may be an HTTP inference service call request. For forwarding a GET-format HTTP inference service call request, the proxy service module may construct a new GET-format HTTP inference service call request and send the constructed GET-format HTTP inference service call request to the address corresponding to the inference service module. The proxy service module may traverse and store all parameters and headers in the received HTTP inference service call request. When constructing the new GET-format HTTP inference service call request, the recipient address of the constructed GET-format HTTP inference service call request may include the network address and second port of the Gunicorn service, and the parameters and headers of the constructed GET-format HTTP inference service call request may be the stored parameters and headers. For forwarding a POST-format HTTP inference service call request, when constructing the new POST-format HTTP inference service call request, the proxy service module may not only traverse and store all parameters and headers in the received HTTP inference service call request, but also construct a form and populate the constructed form with all the contents of the received HTTP inference service call request form.

[0082] In this embodiment, the proxy service module occupies the first port, changes the recipient address of the reasoning service call request sent by the client, and forwards it to the second port, so that the proxy service module can proxy the reasoning service on the second port of the reasoning service module.

[0083] Optionally, the method further includes:

[0084] Send a registration request to the microservice registration center, the registration request including the service address and service name;

[0085] Receive a registration response from the microservice registration center to the registration request.

[0086] The service address and service name may be the service address and service name of a proxy service module.

[0087] Among them, the network address of the microservice registration center can be configured in the configuration parameters of the Java proxy service. The proxy service module can read the network address of the microservice registration center from the configuration parameters and send a registration request to the network address of the microservice registration center. After the microservice registration center receives the registration request sent by the proxy service module, it can store the service address and service name of the proxy service module, so that the client can find the proxy service module through the microservice registration center. A message can be sent to the proxy service module through the service address of the proxy service module, and the service name of the proxy service module can be used to identify the proxy service module. In the case of successful registration, the registration response can be a response message used to indicate a successful registration; in the case of failed registration, the registration response can be a response message used to indicate a failed registration.

[0088] In this implementation, the proxy service module registers the service address and service name in the microservice registration center, so that the client can find the proxy service module through the microservice registration center, and then call the reasoning service of the reasoning service module through the proxy service module.

[0089] Optionally, the method further includes:

[0090] Requesting the inference service module to perform service status detection at preset time intervals and obtaining service status detection results;

[0091] Send the service status detection result to the microservice registration center.

[0092] Among them, the preset time interval can be 30s, or 5min, or 1h, and can be set according to actual needs, and this embodiment does not limit this. The proxy service module can call the service status detection interface of the reasoning service module to perform service status detection and obtain the service status detection result. The service status detection result can be used to indicate whether the reasoning service of the reasoning service module is normal. For example, the service status detection result can include two states: normal service and abnormal service. When the service status detection result is abnormal service, the reasoning service of the reasoning service module cannot be called. The proxy service module can report the service status detection result of the reasoning service module as its own service status detection result to the microservice registration center.

[0093] In addition, the representation of the service status detection result sent by the proxy service module to the microservice registry center is represented in accordance with the format requirements of the microservice registry center. Taking the Eureka microservice registry center as an example, the service status detection interface can comply with the RESTful specification and return a JSON string with fixed content.

[0094] In this implementation, the proxy service module periodically requests the inference service module to perform service status detection, obtains the service status detection result of the inference service module, and sends the service status detection result to the microservice registration center, so that the microservice registration center can monitor the service status of the inference service in real time.

[0095] See also Figure 3 , Figure 3 This is a flow chart of a service calling method provided by an embodiment of the present invention. Figure 3 As shown, the method includes the following steps:

[0096] Step 201: Receive an inference service call request sent by a proxy service module generated based on the first programming language encapsulation;

[0097] Step 202: Based on the inference service call request, the inference model encapsulated in the second programming language is called to perform inference, and the inference result is sent to the proxy service module.

[0098] The service calling method can be applied to an inference service module to implement inference services.

[0099] It should be noted that this embodiment is Figure 1 The implementation of the corresponding reasoning service module in the embodiment shown in the figure can be found in the specific implementation. Figure 1 In order to avoid duplication, the relevant descriptions of the embodiment shown will not be repeated in this embodiment, and the same beneficial effects can still be achieved.

[0100] Optionally, the invoking a reasoning model encapsulated in a second programming language to perform reasoning based on the reasoning service call request includes:

[0101] Based on the inference service call request, a target inference process is determined from N inference processes started on the graphics card, and based on the target inference process, an inference model encapsulated in the second programming language is called to perform inference, where N is an integer greater than 1.

[0102] The target inference process may be an idle process among the N inference processes. For example, the target inference process may be a randomly selected inference process from the idle processes. The inference service module may expose only the second port to the outside when the N inference processes are started.

[0103] It should be noted that the GPU resources of the graphics card are relatively important. Currently, when using the TensorFlow model encapsulated in the Python language for inference services, a single-process solution is used, which cannot make good use of GPU resources.

[0104] In this embodiment, the inference service module starts N inference processes on the graphics card, and multiple inference processes share the graphics card's video memory. Compared with starting only one inference process on the graphics card, the GPU utilization rate of the graphics card can be improved.

[0105] Optionally, the inference model is loaded on the graphics card with preset video memory, and N is determined based on the ratio of video memory occupancy to the total video memory, the video memory occupancy is the video memory occupied when the inference model is loaded on the graphics card using the video memory occupancy self-growth mode, and the total video memory is the total video memory of the graphics card.

[0106] Wherein, N may be 1 / (n1 / n2), n1 may be the video memory usage, and n2 may be the total video memory. If 1 / (n1 / n2) is not an integer, 1 / (n1 / n2) may be rounded to obtain N. For example, rounding may be performed. The preset video memory may be determined based on the video memory usage n1. The preset video memory may be the product of the video memory usage n1 and a preset coefficient. The preset coefficient may be less than or equal to 1. For example, the preset video memory may be the video memory usage n1.

[0107] In addition, before starting the inference service, you can read the total amount of video memory on the graphics card in the idle state and the video memory occupied by an inference model when it is loaded on the graphics card using the self-growth mode. For example, you can start a Python inference service on an idle graphics card. When the Python inference service starts, it first reads the total amount of video memory n2 on the graphics card when no model is loaded, then uses the self-growth mode, that is, set allowGrouth = True in the code, loads the TensorFlow model, and executes an inference call to ensure that the TensorFlow model is loaded normally. Then, it reads the video memory occupied n1 on the graphics card when the TensorFlow model is loaded.

[0108] Furthermore, the TensorFlow model can be set in the configuration file to preset the video memory loading. For example, the video memory occupancy ratio of the TensorFlow model loading can be set in the configuration file. In actual applications, the video memory occupancy ratio n1 / n2 of the TensorFlow model and the maximum number N of TensorFlow models that can be loaded simultaneously can be calculated. The video memory occupancy ratio n1 / n2 and the number N are set in the configuration file. When the inference service is started, the video memory occupancy ratio n1 / n2 and the number N of loadable TensorFlow models in the configuration file are read, the TensorFlow model is loaded with the video memory occupancy ratio n1 / n2, and N python inference processes are started.

[0109] It should be noted that when starting the inference service, you can start the Gunicorn service on an idle graphics card. The Gunicorn service occupies the second port and reads the number N of loadable TensorFlow models in the configuration file to start N Python inference processes. Each Python inference process corresponds to a TensorFlow model, and the TensorFlow models are loaded according to the graphics memory usage ratio n1 / n2 specified in the configuration file.

[0110] In this implementation, when the inference model is loaded on the graphics card, the video memory occupied by the inference model is limited, so that N inference processes can share the video memory.

[0111] See also Figure 4 , Figure 4 FIG. 1 is a structural diagram of a service calling device provided by an embodiment of the present invention. Figure 4 As shown, the service calling device 300 includes:

[0112] The first receiving unit 301 is configured to receive an inference service call request encapsulated in a first programming language and sent by a microservice gateway;

[0113] A modifying unit 302 is configured to modify the recipient address in the inference service call request to an address corresponding to the inference service module generated based on the encapsulation of the second programming language;

[0114] A first sending unit 303 is configured to send the modified reasoning service call request to the reasoning service module;

[0115] The second receiving unit 304 is configured to receive the inference result of the modified inference service call request sent by the inference service module;

[0116] The second sending unit 305 is used to send the inference result to the microservice gateway.

[0117] Optionally, the first receiving unit 301 is specifically configured to:

[0118] Receive an inference service call request encapsulated in a first programming language and sent by the microservice gateway from the first port;

[0119] The first sending unit 303 is specifically configured to:

[0120] Send the modified reasoning service call request to the second port of the reasoning service module.

[0121] Optionally, the service calling device 300 further includes:

[0122] The third sending unit is used to send a registration request to the microservice registration center, where the registration request includes a service address and a service name;

[0123] The third receiving unit is configured to receive a registration response from the microservice registration center to the registration request.

[0124] Optionally, the service calling device 300 further includes:

[0125] an acquiring unit, configured to request the inference service module to perform service status detection at a preset time interval and acquire a service status detection result;

[0126] The fourth sending unit is used to send the service status detection result to the microservice registration center.

[0127] See also Figure 5 , Figure 5 FIG. 1 is a structural diagram of another service calling device provided by an embodiment of the present invention. Figure 5 As shown, the service calling device 400 includes:

[0128] A receiving unit 401 is configured to receive an inference service call request sent by a proxy service module generated based on encapsulation in a first programming language;

[0129] The reasoning unit 402 is configured to call the reasoning model encapsulated in the second programming language to perform reasoning based on the reasoning service call request, and send the reasoning result to the proxy service module.

[0130] Optionally, the reasoning unit 402 is specifically configured to:

[0131] Based on the inference service call request, a target inference process is determined from the N inference processes started on the graphics card, and based on the target inference process, an inference model encapsulated in the second programming language is called to perform inference, and the inference result is sent to the proxy service module, where N is an integer greater than 1.

[0132] Optionally, the inference model is loaded on the graphics card with preset video memory, and N is determined based on the ratio of video memory occupancy to the total video memory, the video memory occupancy is the video memory occupied when the inference model is loaded on the graphics card using the video memory occupancy self-growth mode, and the total video memory is the total video memory of the graphics card.

[0133] It should be noted that the service calling device in the embodiment of the present invention can be used to execute the service calling method described in the above embodiment and can achieve the same technical effect. To avoid repetition, it will not be described here.

[0134] See also Figure 6 , Figure 6 is a schematic structural diagram of an electronic device provided by an embodiment of the present invention, such as Figure 6As shown, the electronic device 500 includes: a memory 502, a processor 501, and a program stored in the memory 502 and executable on the processor 501. When the program is executed by the processor 501, the various processes of the above-mentioned service calling method embodiment are implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0135] The present invention also provides a computer-readable storage medium having a computer program stored thereon. When executed by a processor, the computer program implements the various processes of the above-described service call method embodiment and achieves the same technical effects. To avoid repetition, the details are not described here. The computer-readable storage medium may be, for example, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0136] It should be noted that, in this document, the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, article, or apparatus comprising a series of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, article, or apparatus. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, method, article, or apparatus comprising the element.

[0137] Through the description of the above embodiments, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus the necessary general hardware platform, and of course can also be implemented by hardware, but in many cases the former is a better embodiment. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions for enabling a terminal (which can be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in each embodiment of the present invention.

[0138] The embodiments of the present invention are described above in conjunction with the accompanying drawings, but the present invention is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the present invention, ordinary technicians in this field can also make many forms without departing from the scope of protection of the present invention and the claims, all of which are protected by the present invention.

Claims

1. A service calling method, characterized in that: Applied to the proxy service module, the method includes: Receiving, from the first port, an inference service call request encapsulated in a first programming language and sent by the microservice gateway, wherein the inference service call request is: the client searches for the service address and the first port of the proxy service module from the microservice registration center, and sends the call request to the microservice gateway based on the service address and the first port of the proxy service module found; Modify the recipient address in the inference service call request to an address corresponding to the inference service module generated based on the second programming language encapsulation; Sending the modified inference service call request to the second port of the inference service module, wherein the configuration parameters of the proxy service module are configured with the second port and the service status detection interface of the inference service module; receiving an inference result of the modified inference service call request sent by the inference service module; Sending the inference result to the microservice gateway; The method further comprises: Send a registration request to the microservice registration center, where the registration request includes the service address and service name of the proxy service module. The service address and service name of the proxy service module are used by the client to search for the proxy service module in the microservice registration center; Receive a registration response from the microservice registration center to the registration request; The method further comprises: Requesting the service status detection interface of the reasoning service module to perform service status detection at a preset time interval and obtaining the service status detection result; Send the service status detection result to the microservice registration center.

2. A service calling method, characterized in that: The method comprises: Receive an inference service call request sent by a proxy service module generated based on the first programming language encapsulation from the second port, wherein the inference service call request sent by the proxy service module is a request after the proxy service module receives the inference service call request from the microservice gateway through the first port and modifies the address of the recipient in the inference service call request to the address corresponding to the inference model encapsulated based on the second programming language; the inference service call request received by the proxy service module from the microservice gateway is: the client searches for the service address and the first port of the proxy service module from the microservice registration center, and sends a call request to the microservice gateway based on the service address and the first port of the proxy service module found , wherein the microservice registration center is used to receive a registration request sent by the proxy service module, the registration request includes the service address and service name of the proxy service module, the service address and service name of the proxy service module are used by the client to find the proxy service module in the microservice registration center, the configuration parameters of the proxy service module are configured with the second port and the service status detection interface of the reasoning service module; the service status detection interface of the reasoning service module is used by the proxy service module to request the service status detection interface of the reasoning service module to perform service status detection at a preset time interval, obtain service status detection results, and send the service status detection results to the microservice registration center; Invoking a reasoning model encapsulated in a second programming language to perform reasoning based on the reasoning service call request, and sending the reasoning result to the proxy service module; The invoking the inference model encapsulated in the second programming language to perform inference based on the inference service invocation request includes: Determining a target inference process from N inference processes started on the graphics card based on the inference service call request, and calling an inference model encapsulated in a second programming language to perform inference based on the target inference process, where N is an integer greater than 1; In which, the inference model is loaded on the graphics card with a preset video memory, the N is determined based on the ratio of the video memory occupancy to the total video memory, the N is the number obtained by rounding the reciprocal of the ratio, the preset video memory is the product of the video memory occupancy and a preset coefficient, the preset coefficient is less than or equal to 1, the video memory occupancy is the video memory occupied when the inference model is loaded on the graphics card using the video memory occupancy self-growth mode, and the total video memory is the total video memory of the graphics card.

3. A service calling device, characterized in that: The service calling device includes: A first receiving unit is configured to receive, from a first port, an inference service call request encapsulated in a first programming language and sent by a microservice gateway, wherein the inference service call request is a call request sent by a client to the microservice gateway based on the service address and the first port of the proxy service module found by the client from the microservice registration center; a modification unit, configured to modify the recipient address in the inference service call request to an address corresponding to the inference service module generated based on the encapsulation of the second programming language; A first sending unit, configured to send the modified inference service call request to the second port of the inference service module, wherein the configuration parameters of the proxy service module are configured with the second port and the service status detection interface of the inference service module; A second receiving unit is configured to receive an inference result of the modified inference service call request sent by the inference service module; A second sending unit, configured to send the inference result to the microservice gateway; The service calling device also includes: The third sending unit is used to send a registration request to the microservice registration center, where the registration request includes a service address and a service name of the proxy service module, and the service address and the service name of the proxy service module are used by the client to search for the proxy service module in the microservice registration center; A third receiving unit is configured to receive a registration response from the microservice registration center to the registration request; The service calling device also includes: an acquiring unit, configured to request the service status detection interface of the inference service module to perform service status detection at a preset time interval and acquire a service status detection result; The fourth sending unit is used to send the service status detection result to the microservice registration center.

4. A service calling device, characterized in that: The service calling device includes: A receiving unit is used to receive, from the second port, an inference service call request sent by a proxy service module generated based on the first programming language encapsulation, wherein the inference service call request sent by the proxy service module is a request after the proxy service module receives the inference service call request from the microservice gateway through the first port and modifies the address of the recipient in the inference service call request to the address corresponding to the inference model encapsulated based on the second programming language; the inference service call request received by the proxy service module from the microservice gateway is: the client searches for the service address and the first port of the proxy service module from the microservice registration center, and sends a request to the microservice gateway based on the service address and the first port of the proxy service module found A call request, wherein the microservice registration center is used to receive a registration request sent by the proxy service module, the registration request including the service address and service name of the proxy service module, the service address and service name of the proxy service module are used by the client to search for the proxy service module in the microservice registration center, the configuration parameters of the proxy service module are configured with the second port and the service status detection interface of the reasoning service module; the service status detection interface of the reasoning service module is used by the proxy service module to request the service status detection interface of the reasoning service module to perform service status detection at a preset time interval, obtain a service status detection result, and send the service status detection result to the microservice registration center; An inference unit, configured to call an inference model encapsulated in a second programming language to perform inference based on the inference service call request, and send an inference result to the proxy service module; The reasoning unit is specifically used for: Determining a target inference process from N inference processes started on the graphics card based on the inference service call request, calling an inference model encapsulated in a second programming language to perform inference based on the target inference process, and sending the inference result to the proxy service module, where N is an integer greater than 1; In which, the inference model is loaded on the graphics card with a preset video memory, the N is determined based on the ratio of the video memory occupancy to the total video memory, the N is the number obtained by rounding the reciprocal of the ratio, the preset video memory is the product of the video memory occupancy and a preset coefficient, the preset coefficient is less than or equal to 1, the video memory occupancy is the video memory occupied when the inference model is loaded on the graphics card using the video memory occupancy self-growth mode, and the total video memory is the total video memory of the graphics card.

5. An electronic device, characterized in that: include: A processor, a memory, and a program stored in the memory and executable on the processor, wherein the program, when executed by the processor, implements the steps of the service calling method according to claim 1, or the program, when executed by the processor, implements the steps of the service calling method according to claim 2.

Images