Service processing method, device and electronic equipment

Abstract

The embodiment of the present invention provides a service processing method, device and electronic device, wherein the method includes: creating multiple model service units for building a model service; editing the management models corresponding to the multiple model service units, the management model is used to manage the communication status between the model service units during operation; arranging the multiple model service units and their corresponding management models to build the model service. The solution of the embodiment of the present invention can effectively avoid the drawbacks of the monolithic model service, support microservice governance, and realize flexible deployment of model services.

Description

Technical Field

[0001] The present application relates to the field of communication technology, and in particular to a service processing method, device and electronic device. Background Art

[0002] With the widespread application of machine learning, how to efficiently deploy trained machine learning models to production environments as model services is being supported by more and more tools. The formation of model services includes several links: data collection, data analysis, data transformation, data verification, data training, model creation, model verification, large-scale training, model publishing, service provision, model monitoring and logging. In this process, many machine learning tools provide different options for model developers, but also bring different challenges to model deployment.

[0003] Traditional model deployment and runtime environment support have certain drawbacks, including:

[0004] 1. Model service applications are monolithic and do not support microservice governance, which results in the inability to dynamically scale service resources during model runtime based on different workload types;

[0005] 2. The routing management of the model service is tightly coupled with the machine learning model algorithm, which means that when the model algorithm changes, even if the routing management does not change, all services still need to be redeployed;

[0006] 3. The conversion rules from the output of one model service unit to the input of the next model service unit are tightly coupled with each model unit and the engine, making it impossible to achieve flexible protocol conversion and data format conversion. Summary of the invention

[0007] The present invention provides a service processing method, device and electronic device, which can effectively avoid the disadvantages of monolithic model services, support microservice governance, and realize flexible deployment of model services.

[0008] To achieve the above object, the embodiments of the present invention adopt the following technical solutions:

[0009] In a first aspect, a service processing method is provided, comprising:

[0010] Create multiple model service units for building model services;

[0011] Editing management models corresponding to the multiple model service units, wherein the management model is used to manage the communication status between the model service units during operation;

[0012] The multiple model service units and their corresponding management models are orchestrated to construct the model service.

[0013] In a second aspect, a service processing device is provided, including:

[0014] A service unit creation module, used to create multiple model service units for building model services;

[0015] A management model editing module, used to edit the management models corresponding to the multiple model service units, wherein the management models are used to manage the communication status between the model service units during operation;

[0016] The model service orchestration module is used to orchestrate the multiple model service units and their corresponding management models to construct the model service.

[0017] In a third aspect, an electronic device is provided, including:

[0018] Memory, used to store programs;

[0019] The processor is coupled to the memory and is used to execute the program. When the program is running, the service processing method provided by the present invention is executed.

[0020] The present invention provides a service processing method, device and electronic device, which creates multiple model service units for building model services; edits management models corresponding to multiple model service units, and the management model is used to manage the communication status between model service units during operation; and orchestrates multiple model service units and their corresponding management models to build model services, thereby realizing the deployment of models into production environments to become model services. In this solution, since the model service to be built is split into multiple model service units and the communication between the model service units is realized through the management model, so as to provide services to the outside as a whole, it can effectively avoid the disadvantages of the monolithic model service, support microservice governance, and realize flexible deployment of model services.

[0021] The above description is only an overview of the technical solution of the present application. In order to more clearly understand the technical means of the present application, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the present application more obvious and easy to understand, the specific implementation methods of the present application are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Various other advantages and benefits will become apparent to those of ordinary skill in the art by reading the detailed description of the preferred embodiments below. The accompanying drawings are only for the purpose of illustrating the preferred embodiments and are not to be considered as limiting the present application. Also, the same reference symbols are used throughout the accompanying drawings to represent the same components. In the accompanying drawings:

[0023] Figure 1A diagram of the service processing logic architecture of an embodiment of the present invention;

[0024] Figure 2 The service processing method flow of the embodiment of the present invention is as follows Figure 1 ;

[0025] Figure 3 The service processing method flow of the embodiment of the present invention is as follows Figure 2 ;

[0026] Figure 4 The service processing method flow of the embodiment of the present invention is as follows Figure 3 ;

[0027] Figure 5 The service processing method flow of the embodiment of the present invention is as follows Figure 4 ;

[0028] Figure 6 The service processing device structure of the embodiment of the present invention is Figure 1 ;

[0029] Figure 7 The service processing device structure of the embodiment of the present invention is Figure 2 ;

[0030] Figure 8 The service processing device structure of the embodiment of the present invention is Figure 3 ;

[0031] Fig. 9 Schematic diagram of the structure of an electronic device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0032] The exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although the exemplary embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided to enable a more thorough understanding of the present disclosure and to fully convey the scope of the present disclosure to those skilled in the art.

[0033] The embodiments of the present invention improve the defects of the prior art when deploying machine learning models as model services: the model service application is monolithic, does not support microservice governance, the routing management of the model service is tightly coupled with the machine learning model algorithm, and the model service cannot be flexibly deployed. The core idea is that when building a model service, instead of performing machine learning on the entire model service to obtain a modeled service, the model service is split according to different business functions, and a corresponding model service unit is created for each business function; then, the corresponding management model is edited according to the communication status between each model service unit at runtime, and the management model is used to realize the interactive communication between each model service unit; finally, the multiple model service units created and their corresponding management models are orchestrated to build a model service to provide services to the outside world. Since the model service constructed by this solution is split into multiple model service units, it can support microservice governance and realize flexible deployment of model services.

[0034] Based on the above model deployment solution idea, Figure 1 The service processing logic architecture diagram of the embodiment of the present invention includes the following functional modules and processing links:

[0035] Model 101:

[0036] The models (such as Model A, Model B, Model C) used to build model services in this solution can be machine learning models, including but not limited to Python-based models, Tensorflow models, scikit-learn models, Spark models, H2O models, R models and other industry-supported models. The models used to build multiple model service units of the same model service in this solution can be models that support different implementation methods, for example, Model A is a scikit-learn model, Model B is a Tensorflow model, and Model C is a Spark model. These trained models already have certain service capabilities.

[0037] The formats implemented in the above model include but are not limited to YAML or JSON formats commonly used in the service mesh field.

[0038] Service Mesh Metadata 102:

[0039] The service grid metadata 102 in this solution is the metadata of the management model corresponding to the model service. The management model may include service request traffic routing rules, such as routing rules based on request content, routing rules based on weight, etc.; it may also include AB testing rules for the testing field, Multi-Armed Bandit rules for intelligent decision-making problems, etc. The implementation format includes but is not limited to the YAML or JSON format commonly used in the service grid field.

[0040] Service mesh controller 103:

[0041] As a core component in this solution, a mechanism for service-gridding the machine learning model 101 at runtime is provided, and the model service deployment is realized by coordinating and managing the model service processor 104 and the service grid metadata processor 106. The service grid controller 103 can be provided as a pluggable mode for coordinating the model service processor 104 and the service grid metadata processor 106 in different implementation modes.

[0042] Model service processor 104:

[0043] After being processed by the model service processor 104 , the model 101 will be injected into a unified proxy module and converted into a model service within the service grid, namely, the model service 105 injected into the proxy.

[0044] Model service 105 injected into the proxy:

[0045] Model 101 is processed by model service processor 104, that is, after injecting the proxy module into the model, it is converted into a model service in the service grid. The interaction between these injected proxy model services will be taken over by the proxy module and maintained as a service grid. The implementation format includes but is not limited to the YAML or JSON format commonly used in the service grid field.

[0046] Service mesh metadata processor 106:

[0047] After being processed by the service mesh metadata processor 106 , the service mesh metadata 102 generates usable service mesh metadata and stores it in the service mesh metadata storage 107 .

[0048] Service grid metadata storage 107:

[0049] The service grid metadata storage unit in this solution includes, but is not limited to, distributed data storage commonly used in the service grid field. The stored service grid metadata includes, but is not limited to, metadata used to generate routing rules 109 and conversion rules 111.

[0050] Routing Manager 108

[0051] The routing manager 108 provides a pluggable method for supporting routing rule management in different implementations. The routing manager 108 converts the routing rule metadata in the service grid metadata storage 107 into routing rules 109 supported in the service grid by monitoring data changes therein.

[0052] Routing rule 109

[0053] Routing rules 109 are used to perform request routing tasks within the service grid. Routing can accept an incoming service request at one level, match it with an upstream service cluster, obtain a connection pool in the upstream service cluster, and forward the request.

[0054] Transformation Manager 110

[0055] The conversion manager 110 provides a pluggable method for supporting conversion rule management in different implementations. The conversion manager 110 converts the conversion rule metadata in the service grid metadata storage 107 into conversion rules 111 supported in the service grid by monitoring data changes therein.

[0056] Conversion Rule 111

[0057] Conversion rule 111 is used to perform protocol and data format conversion for service interactions within the service grid. Conversion tasks include, but are not limited to, conversion of different communication protocols, such as conversion from HTTP to gRPC protocol; conversion of different data formats, such as conversion from JSON to Protobuffer; and filtering or enhancing data content, such as removing non-business data in the request header or adding request header data required for tracking.

[0058] Model Service Orchestrator112

[0059] The model service orchestrator 112 is used to orchestrate the model service 105 injected into the proxy, the routing service implemented by the routing manager 108 and the routing rules 109, and the transformation service implemented by the transformation manager 110 and the transformation rules 111, and provide a service grid model service capability to the outside world.

[0060] Figure 1 The service processing logic shown in is as follows:

[0061] Step 201, containerize the machine learning model and describe the model runtime parameter configuration in YAML or JSON format.

[0062] In step 202 , the service grid controller 103 implements the conversion process of the model by coordinating and managing the model service processor 104 .

[0063] Step 203, after the model 101 is processed by the model service processor 104, a proxy module is injected into the model, converted into a model service in the service grid, and a proxy-injected model service 105 is generated. The interaction between these proxy-injected model services will be taken over by the proxy and maintained as a service grid.

[0064] Step 204, edit the routing rules and conversion rule metadata of the model service (in the actual processing process, the metadata corresponding to other management models can also be edited according to the model service requirements).

[0065] Step 205 , the service grid controller 103 implements processing (conflict checking, merging, etc.) of the service grid metadata by coordinating and managing the service grid metadata processor 106 .

[0066] Step 206 , storing the processed service grid metadata in the service grid metadata storage 107 .

[0067] In step 207 , the routing manager 108 converts the routing rule metadata in the service grid metadata storage 107 into the routing rules 109 supported in the service grid by monitoring the data changes in the service grid metadata storage 107 .

[0068] Step 208 , the conversion manager 108 converts the conversion rule metadata in the service grid metadata storage 107 into the conversion rules 111 supported in the service grid by monitoring the data changes in the service grid metadata storage 107 .

[0069] In step 209, the model service orchestrator 112 orchestrates the model service 105 injected into the proxy, the routing service implemented by the routing manager 108 and the routing rules 109, and the transformation service implemented by the transformation manager 110 and the transformation rules 111, and provides a service grid model service capability to the outside world.

[0070] Compared with the existing technology, this solution has the following advantages:

[0071] 1. A mechanism for service-gridding the machine learning model 101 at runtime is provided through the service grid controller 103, and the mechanism is implemented by coordinating the management model service processor 104 and the service grid metadata processor 106; and a mechanism for separating the algorithm logic module from the service governance capability is implemented, supporting independent release, deployment and operation of the model;

[0072] 2. The model service orchestrator 112 provides a mechanism for orchestrating the model service 105 injected into the proxy, the routing service implemented by the routing manager 108 and the routing rules 109, and the transformation service implemented by the transformation manager 110 and the transformation rules 111, thereby providing a service grid model service capability to the outside world, thereby supporting traffic management, policy enhancement, and telemetry collection capabilities.

[0073] 3. After being processed by the model service processor 104, a unified proxy module is injected into each business model and converted into a model service within the service grid, supporting dynamic expansion and contraction of models at runtime according to different workload types.

[0074] 4. A pluggable method is provided through the routing manager 108 to support routing rule management under different implementation modes, and the changes of the supporting model algorithm and routing management can remain independent.

[0075] 5. Through the conversion manager 110, a flexible protocol conversion and data format conversion mechanism is provided to support the decoupling of the conversion rules from the output of one model service to the input of another model service from each model and engine.

[0076] The technical solution of the present application is further illustrated below through multiple embodiments.

[0077] Embodiment 1

[0078] Based on the above model deployment solution ideas, such as Figure 2 As shown, it is a model processing method flow shown in an embodiment of the present invention Figure 1 , this method can be achieved by Figure 1 The service grid controller 103 shown in FIG. 1 coordinates the model service processor 104, the service grid metadata processor 106, and the model service orchestrator 112 to complete the execution. Figure 2 As shown, the service processing method includes the following steps:

[0079] S210, creating multiple model service units for building model services.

[0080] Specifically, before building a model service based on a machine learning model, the model service with specified business processing capabilities to be built can be broken down according to the business processing capabilities it has, and presented in the form of multiple model service units, each of which is endowed with a corresponding part of the business processing capabilities. Then, these model service units are created using models under different implementation methods.

[0081] Among them, the model used to build the model service unit can be a machine learning model, including but not limited to Python models, Tensorflow models, scikit-learn models, Spark models, H2O models, R models and other industry-supported models.

[0082] In this solution, the models used to build multiple model service units of the same model service can be models that support different implementation modes, for example, model service unit A uses the scikit-learn model, model service unit B uses the Tensorflow model, and model service unit C uses the Spark model. These trained models already have certain service capabilities.

[0083] Furthermore, when creating multiple model service units for building model services, the service grid mechanism can be used to implement the steps as follows:

[0084] Step 1: Obtain the business model used to create each model service unit;

[0085] Among them, the business model can be obtained by machine learning the training data using the model under any of the above implementation methods. These trained business models already have the business processing capabilities specified by the corresponding target model service unit.

[0086] Step 2: Inject the network proxy in the service mesh into the business model to form a model service unit under the service mesh.

[0087] Among them, Service Mesh is an infrastructure layer dedicated to solving the communication between services. Service Mesh is usually implemented through a set of lightweight network proxies (Sidecar proxies) deployed together with the application code without the need to perceive the application itself.

[0088] In this solution, the above business model is used as an application, and the network proxy in the service grid is injected into the business model to form a model service unit under the service grid. The interaction between these model service units injected with the network proxy will be taken over by the corresponding proxy module and maintained as a service grid.

[0089] S220, editing the management models corresponding to the multiple model service units, where the management model is used to manage the communication status between the model service units during runtime.

[0090] Specifically, after forming multiple model service units, the model service units are still isolated from each other, and it is necessary to manage the communication status between the model service units at runtime so that they can be presented as a whole model service during the interaction process, thereby providing complete service capabilities to the outside world. By pre-configuring the rules for how to manage the communication status between model service units at runtime, the model service units can perform the interaction process based on these rules. Here, all the rules required to implement interaction are collectively referred to as management models, which at least include routing rules and conversion rules.

[0091] Furthermore, when editing the management models corresponding to multiple model service units, the service grid mechanism can be used to implement the process as follows:

[0092] Step 1: Get the service grid metadata corresponding to multiple model service units.

[0093] The service grid metadata may include at least metadata for writing routing rules and conversion rules required for communication between model service units.

[0094] After obtaining the service grid metadata, the obtained service grid metadata may be processed such as clearing data conflicts and merging data to improve the quality of the service grid metadata.

[0095] Step 2: Process the service grid metadata to form a management model corresponding to multiple model service units.

[0096] The purpose of this step is to convert the service grid metadata obtained in different formats and forms into a management model that can be supported and used by the model service unit. For example, the routing rule metadata is converted into the routing rules supported by the service grid, and the transformation rule metadata is converted into the transformation rules supported by the service grid. In the model service unit formed after the network agent is injected into the above business model, the network agent needs to take over the interaction tasks between the model service units according to these management models.

[0097] S230, orchestrate multiple model service units and their corresponding management models to build a model service.

[0098] For example, when using the service grid mechanism to implement model service deployment, the model service unit injected into the network agent, the routing service implemented by forming routing rules based on routing metadata, and the conversion service implemented by forming conversion rules based on conversion metadata can be orchestrated together to provide service grid model service capabilities to the outside world.

[0099] In addition, a very important feature of this solution is that it separates the creation of the model service unit from the writing of the corresponding management model, which makes it easier to manage the model service unit and the management model separately, allowing flexible deployment when building the model service.

[0100] The service processing method provided by the present invention creates multiple model service units for building model services; edits the management models corresponding to the multiple model service units, and the management model is used to manage the communication status between the model service units during operation; and orchestrates the multiple model service units and their corresponding management models to build a model service, thereby realizing the deployment of the model into a production environment to become a model service. In this solution, since the model service to be built is split into multiple model service units and the communication between the model service units is realized through the management model, so as to provide services to the outside world as a whole, it can effectively avoid the disadvantages of the monolithic model service, support microservice governance, and realize flexible deployment of model services.

[0101] Embodiment 2

[0102] like Figure 3 FIG. 1 is a flow chart of a service processing method according to an embodiment of the present invention. Figure 2The method steps shown in this embodiment can be used as a supplementary solution to the method in embodiment 1. Figure 3 As shown in the figure, after building the model service, you can also perform the following steps:

[0103] S310, obtaining a change business model for creating a model service unit.

[0104] After the model service is built, if the business processing algorithm of a certain model service unit needs to be changed, the business model corresponding to the model service unit can be directly re-machined to obtain the changed business model without changing other model service units and management models.

[0105] S320, injecting a network agent in the service grid into the changed business model to form a changed model service unit.

[0106] S330, orchestrate the change model service unit and its corresponding management model to rebuild the model service.

[0107] After a model service unit in the service model is changed, the changed model service unit and its corresponding management model, such as routing rules and conversion rules, need to be rearranged to rebuild the model service.

[0108] Further, or alternatively, if Figure 4 As shown in the figure, after building the model service, you can also perform the following steps:

[0109] S410, obtaining change data of the service grid metadata corresponding to the model service unit.

[0110] After the model service is built, if you need to change the communication logic between a model service unit and other model service units, you can rewrite the service grid metadata corresponding to the corresponding model service unit to obtain the changed data without changing other model service units and management models.

[0111] S420: Process the change data of the service grid metadata to form a change management model.

[0112] S430, orchestrate the change management model and the corresponding model service unit to rebuild the model service.

[0113] After changing the management model corresponding to a model service unit in the service model, it is necessary to rearrange the change management model, such as routing rules and conversion rules, and the corresponding model service unit to rebuild the model service.

[0114] Further, or alternatively, if Figure 5As shown in the figure, after building the model service, you can also perform the following steps:

[0115] S510, obtaining a change business model for creating a model service unit;

[0116] S520, injecting a network agent in the service grid into the changed business model to form a changed model service unit;

[0117] S530, obtaining change data of the service grid metadata corresponding to the model service unit;

[0118] S540, processing the change data of the service grid metadata to form a change management model;

[0119] S550, orchestrate the change model service unit and its corresponding management model, and orchestrate the change management model and the corresponding model service unit to rebuild the model service.

[0120] Figure 5 The method shown combines Figure 3 , Figure 4 The method content is that after the model service is built, the business processing algorithm and management model of the model service unit need to be changed at the same time. Among them, the changed model service unit and management model may not have a corresponding relationship. Therefore, it is necessary to re-arrange the changed subject and the corresponding model service unit or management model respectively to rebuild the model service.

[0121] This embodiment Figure 2 On the basis of the illustrated embodiment, multiple ways of reconstructing the model service are further illustrated, for example, by obtaining a change business model for creating a model service unit; injecting a network agent in the service grid into the change business model to form a change model service unit; orchestrating the change model service unit and its corresponding management model to reconstruct the model service, thereby realizing the reconstruction of the model service for the change of the model service unit; for another example, by obtaining the change data of the service grid metadata corresponding to the model service unit; processing the change data of the service grid metadata to form a change management model; orchestrating the change management model and the corresponding model service unit to reconstruct the model service, thereby realizing the reconstruction of the model service for the change of the management model. Furthermore, the model service unit and the management model can also be changed at the same time to realize the reconstruction of the model service.

[0122] This embodiment can achieve the purpose of reconstructing the model service by changing the local business capabilities or management capabilities of the model service, making the model service deployment more flexible, realizing a mechanism to separate the algorithm logic module and service governance capabilities in the model service, and supporting independent model release, deployment and operation and maintenance.

[0123] By orchestrating the model service unit injected into the network agent, the routing service implemented by writing routing rules based on routing metadata, and the transformation service implemented by writing transformation rules based on transformation metadata, a service grid model service capability is provided to the outside world, thereby supporting traffic management, policy enhancement, and telemetry collection capabilities.

[0124] By injecting a unified network agent into the business model, it is converted into a model service unit within the entire service grid, which can support dynamic expansion and contraction of resources such as CPU and memory according to the resource requirements of model service units of different workload types at runtime.

[0125] In addition, this solution can support routing rule management under different implementation methods, and support changes in model algorithms and routing management to remain independent; at the same time, it provides a flexible protocol conversion and data format conversion mechanism, and supports decoupling the conversion rules from the output of one model service unit to the input of another model service unit from each business model and engine.

[0126] Embodiment 3

[0127] like Figure 6 FIG. 1 is a diagram showing the structure of a service processing device according to an embodiment of the present invention. Figure 1 , the service processing device may be arranged at Figure 1 In the service mesh controller shown, it is used to execute Figure 2 The method steps shown in , which include:

[0128] A service unit creation module 610, used to create a plurality of model service units for building a model service;

[0129] A management model editing module 620 is used to edit management models corresponding to multiple model service units, and the management model is used to manage the communication status between the model service units during operation;

[0130] The model service orchestration module 630 is used to orchestrate multiple model service units and their corresponding management models to construct model services.

[0131] Furthermore, if Figure 7 As shown, the service unit creation module 610 may include:

[0132] A business model acquisition unit 710, configured to acquire a business model for creating each of the model service units;

[0133] The service unit creation unit 720 is used to inject the network agent in the service grid into the business model to form the model service unit under the service grid.

[0134] Furthermore, if Figure 8 As shown, the management model editing module 620 may include:

[0135] A metadata acquisition unit 810 is used to acquire service grid metadata corresponding to multiple model service units;

[0136] The management model editing unit 820 is used to process the service grid metadata to form a management model corresponding to multiple model service units.

[0137] Furthermore, the business model acquisition unit 710 may also be used to acquire a changed business model for creating a model service unit;

[0138] The service unit creation unit 720 may also be used to inject a network agent in the service grid into the change business model to form a change model service unit;

[0139] The model service orchestration module 630 may also be used to orchestrate the change model service unit and its corresponding management model to rebuild the model service.

[0140] Furthermore, the metadata acquisition unit 810 may also be used to acquire change data of the service grid metadata corresponding to the model service unit;

[0141] The management model editing unit 820 may also be used to process the change data of the service grid metadata to form a change management model;

[0142] The model service orchestration module 630 may also be used to orchestrate the change management model and the corresponding model service unit to rebuild the model service.

[0143] The service processing device provided by the present invention creates multiple model service units for building model services; edits the management models corresponding to the multiple model service units, and the management model is used to manage the communication status between the model service units during operation; and orchestrates the multiple model service units and their corresponding management models to build a model service, thereby realizing the deployment of the model into the production environment as a model service. In this solution, since the model service to be built is split into multiple model service units and the communication between the model service units is realized through the management model, so as to provide services to the outside as a whole, it can effectively avoid the disadvantages of the monolithic model service, support microservice governance, and realize flexible deployment of model services.

[0144] Furthermore, this embodiment shows a variety of reconstruction methods for reconstructing model services, for example, by obtaining a change business model for creating a model service unit; injecting a network agent in a service grid into the change business model to form a change model service unit; orchestrating the change model service unit and its corresponding management model to reconstruct the model service, thereby realizing the reconstruction of the model service for the change of the model service unit; for another example, by obtaining the change data of the service grid metadata corresponding to the model service unit; processing the change data of the service grid metadata to form a change management model; orchestrating the change management model and the corresponding model service unit to reconstruct the model service, thereby realizing the reconstruction of the model service for the change of the management model. Furthermore, the model service unit and the management model can also be changed at the same time to realize the reconstruction of the model service.

[0145] This embodiment can achieve the purpose of reconstructing the model service by changing the local business capabilities or management capabilities of the model service, making the model service deployment more flexible, realizing a mechanism to separate the algorithm logic module and service governance capabilities in the model service, and supporting independent model release, deployment and operation and maintenance.

[0146] By orchestrating the model service unit injected into the network agent, the routing service implemented by writing routing rules based on routing metadata, and the transformation service implemented by writing transformation rules based on transformation metadata, a service grid model service capability is provided to the outside world, thereby supporting traffic management, policy enhancement, and telemetry collection capabilities.

[0147] By injecting a unified network agent into the business model, it is converted into a model service unit within the entire service grid, which can support dynamic expansion and contraction of resources such as CPU and memory based on the resource requirements of model service units of different workload types at runtime.

[0148] In addition, this solution can support routing rule management under different implementation methods, and support changes in model algorithms and routing management to remain independent; at the same time, it provides a flexible protocol conversion and data format conversion mechanism, and supports decoupling the conversion rules from the output of one model service unit to the input of another model service unit from each business model and engine.

[0149] Embodiment 4

[0150] The above embodiments describe the overall architecture of the service processing device. The functions of the device can be implemented by means of an electronic device, such as Fig. 9 As shown, it is a schematic diagram of the structure of an electronic device according to an embodiment of the present invention, which specifically includes: a memory 910 and a processor 920.

[0151] The memory 910 is used to store programs.

[0152] In addition to the above-mentioned programs, the memory 910 may also be configured to store various other data to support operations on the electronic device. Examples of such data include instructions for any application or method operating on the electronic device, contact data, phone book data, messages, pictures, videos, etc.

[0153] The memory 910 may be implemented by any type of volatile or nonvolatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.

[0154] The processor 920 is coupled to the memory 910 and is used to execute the program in the memory 910. When the program is executed, Figures 2 to 5 Any service processing method in .

[0155] The above-mentioned specific processing operations have been described in detail in the previous embodiments and will not be repeated here.

[0156] Further, if Fig. 9 As shown, the electronic device may also include: a communication component 930, a power component 940, an audio component 950, a display 960 and other components. Fig. 9 Only some components are shown schematically, which does not mean that the electronic device only includes Fig. 9 Components shown.

[0157] The communication component 930 is configured to facilitate wired or wireless communication between the electronic device and other devices. The electronic device can access a wireless network based on a communication standard, such as WiFi, 2G or 3G, or a combination thereof. In an exemplary embodiment, the communication component 930 receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 930 also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.

[0158] The power supply component 940 provides power to various components of the electronic device. The power supply component 940 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to the electronic device.

[0159] The audio component 950 is configured to output and / or input audio signals. For example, the audio component 950 includes a microphone (MIC), and when the electronic device is in an operating mode, such as a call mode, a recording mode, and a voice recognition mode, the microphone is configured to receive an external audio signal. The received audio signal can be further stored in the memory 910 or sent via the communication component 930. In some embodiments, the audio component 950 also includes a speaker for outputting audio signals.

[0160] The display 960 includes a screen, which may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from a user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor may not only sense the boundaries of a touch or slide action, but also detect the duration and pressure associated with the touch or slide operation.

[0161] Those skilled in the art can understand that all or part of the steps of implementing the above-mentioned method embodiments can be completed by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, the steps of the above-mentioned method embodiments are executed; and the aforementioned storage medium includes: ROM, RAM, disk or optical disk and other media that can store program codes.

[0162] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present application.

Claims

1. A service processing method, comprising: A service grid mechanism is used to create a plurality of model service units for building a model service; the model is a machine learning model; Editing management models corresponding to the plurality of model service units, the management model being used to manage the communication status between the model service units during operation; the management model at least comprising: routing rules and conversion rules; The multiple model service units and their corresponding management models are orchestrated to construct the model service.

2. The method according to claim 1, wherein: The creating of multiple model service units for building model services includes: Acquire a business model for creating each of the model service units; A network agent in a service grid is injected into the business model to form the model service unit under the service grid.

3. The method according to claim 2, wherein: The editing of the management models corresponding to the multiple model service units comprises: Obtaining service grid metadata corresponding to the multiple model service units; The service grid metadata is processed to form a management model corresponding to the multiple model service units.

4. The method according to claim 3, wherein: The method further comprises: Acquire a change business model for creating the model service unit; Injecting a network agent in a service grid into the change business model to form a change model service unit; The change model service unit and its corresponding management model are orchestrated to rebuild the model service.

5. The method according to claim 3, wherein: The method comprises: Obtaining change data of the service grid metadata corresponding to the model service unit; Processing the change data of the service grid metadata to form a change management model; The change management model and the corresponding model service unit are orchestrated to rebuild the model service.

6. The method according to claim 3, wherein: The method comprises: Acquire a change business model for creating the model service unit; Injecting a network agent in a service grid into the change business model to form a change model service unit; Obtaining change data of the service grid metadata corresponding to the model service unit; Processing the change data of the service grid metadata to form a change management model; The change model service unit and its corresponding management model are orchestrated, and the change management model and the corresponding model service unit are orchestrated to reconstruct the model service.

7. A service processing device, comprising: A service unit creation module, used to create multiple model service units for building model services by using a service grid mechanism; the model is a machine learning model; A management model editing module, used to edit the management models corresponding to the multiple model service units, wherein the management models are used to manage the communication status between the model service units during operation; The management model at least includes: routing rules and conversion rules; The model service orchestration module is used to orchestrate the multiple model service units and their corresponding management models to construct the model service.

8. The device according to claim 7, wherein: The service unit creation module includes: A business model acquisition unit, used to acquire a business model used to create each of the model service units; The service unit creation unit is used to inject the network agent in the service grid into the business model to form the model service unit under the service grid.

9. The device according to claim 8, wherein: The management model editing module includes: A metadata acquisition unit, used to acquire service grid metadata corresponding to the multiple model service units; The management model editing unit is used to process the service grid metadata to form a management model corresponding to the multiple model service units.

10. The device according to claim 9, wherein: The business model acquisition unit is further used to acquire a changed business model used to create the model service unit; The service unit creation unit is further used to inject a network agent in the service grid into the change business model to form a change model service unit; The model service orchestration module is also used to orchestrate the change model service unit and its corresponding management model to rebuild the model service.

11. The device according to claim 9, wherein: The metadata acquisition unit is further used to obtain change data of the service grid metadata corresponding to the model service unit; The management model editing unit is further used to process the change data of the service grid metadata to form a change management model; The model service orchestration module is also used to orchestrate the change management model and the corresponding model service unit to rebuild the model service.

12. The device according to claim 9, wherein: The business model acquisition unit is further used to acquire a changed business model used to create the model service unit; The service unit creation unit is further used to inject a network agent in the service grid into the change business model to form a change model service unit; The metadata acquisition unit is further used to obtain change data of the service grid metadata corresponding to the model service unit; The management model editing unit is further used to process the change data of the service grid metadata to form a change management model; The model service orchestration module is also used to orchestrate the change model service unit and its corresponding management model, and to orchestrate the change management model and the corresponding model service unit to reconstruct the model service.

13. An electronic device comprising: Memory, used to store programs; A processor, coupled to the memory, is used to execute the program, and when the program is running, the service processing method described in any one of claims 1 to 6 is executed.

Images