Model service method, device, and storage medium

Abstract

Embodiments of the present disclosure provide a model service method, a device, and a storage medium. A plurality of service instances deployed for a model service are set up as a data transmission network. When receiving a model calling request, a service instance can first inquire whether model data required for model loading is present in a local storage space, and if not, the service instance can, on the basis of the data transmission network, attempt to remotely download the required model data from other service instances, and store the downloaded model data into the local storage space. In this way, model data sharing can be realized on the basis of the data transmission network established among the plurality of service instances; the network transmission rate between the service instances in the data transmission network is far higher than an origin-pull downloading rate, and therefore, even if model data required for loading is not present in a service instance locally, model data can be remotely downloaded from other service instances. The model loading efficiency can be effectively improved, thereby improving the efficiency of responding to a model calling request.

Description

[0001]This disclosure claims priority to Chinese patent application number 202410199289.8, filed with the China Patent Office on February 22, 2024, entitled "A Model Service Method, Device, and Storage Medium," the entire contents of which are incorporated herein by reference. Technical Field: This disclosure relates to the field of cloud computing, and more particularly to a model service method, device, and storage medium. Background: Model servitization refers to deploying a model and providing services to other services or clients through APIs and other means. Model servitization technology allows users to deploy models as online inference services or A-WEB applications with one click. With the development of model servitization technology, the concept of dynamic multi-model loading has been proposed. This concept allows for dynamic switching of loaded models based on usage needs, allowing the appropriate model to be used to meet the needs. Currently, model data for various models is typically stored in a model repository. Dynamic model loading requires first downloading the corresponding model data through a back-to-source method before loading can be completed, resulting in poor response efficiency for model services. SUMMARY Various aspects of the present disclosure provide a model service method, device, and storage medium for improving the response efficiency of model services. An embodiment of the present disclosure provides a model service method, wherein multiple service instances are deployed for a model service. The method is applicable to any of the multiple service instances and includes: determining a target model to be loaded in response to a received model call request; if the model data required for loading the target model does not exist in local storage, obtaining the model data from the target service instance storing the model data and storing it in the local storage; wherein the multiple service instances are pre-configured as a data transmission network; and loading the target model based on the model data stored in the local storage. Furthermore, obtaining the model data from the target service instance storing the model data includes: querying storage status information maintained for each service instance in the data transmission network to select the target service instance storing the model data; and downloading the model data from the target service instance; wherein the storage status information is used to record description information of the model data stored in the corresponding service instance. Furthermore, the method further includes: after storing the model data in the local storage space, broadcasting a message carrying identification information of the model data to other service instances in the data transmission network, so that each other service instance updates its own storage status information maintained for the service instance based on the identification information of the model data.Furthermore, the method further includes: if there are multiple service instances in the data transmission network storing the model data, selecting the service instance with the lowest load as the target service instance; wherein each service instance in the data transmission network broadcasts its own load status. Furthermore, the method further includes: if the model data does not exist in any other service instances in the data transmission network, performing a back-to-source download operation to download the model data from the data source; and storing the downloaded model data in the local storage space. Furthermore, the data source is mounted on a user-mode file system constructed in the service instance, and the back-to-source download operation includes: querying metadata maintained for the user-mode file system for first file description information corresponding to the model data in the data source; and downloading the model data from the data source according to the first file description information. Furthermore, the local storage space is mounted on a user-mode file system constructed in the service instance. The method further includes: querying metadata maintained for the user-mode file system for second file description information corresponding to the model data in the local storage space; if no data matching the second file description information is found in the local storage space, determining that the model data does not exist in the local storage space. Furthermore, the method further includes: after the service instance is created, requesting a list of service instances corresponding to the model service from a management instance deployed for the model service; and joining the service instances in the service instance list to the data transmission network established by each service instance. Furthermore, the method further includes: upon receiving a remote download instruction issued by any other service instance in the data transmission network, determining the target model data requested by the remote download instruction; reading the target model data from the local storage space; and sending the target model data to the service instance that issued the remote download instruction via the data transmission network. An embodiment of the present disclosure further provides a computing device, comprising a memory, a processor, and a communication component. The computing device carries a service instance deployed for a model service, the service instance being used to dynamically load a model, and the service instance pre-connecting with other service instance components deployed for the model service as a data transmission network; the memory being used to store one or more computer instructions; and the processor being coupled to the memory and the communication component to execute the one or more computer instructions to perform the aforementioned model service method.Embodiments of the present disclosure also provide a model service device, respectively disposed in multiple service instances deployed for a model service. The model service device includes a service module, a processing model, and a network assembly module. The network assembly module is configured to control the service instance in which it is located to establish a data transmission network with other service instances deployed for the model service. The service module is configured to, in response to a received model call request, determine a target model to load. The processing module is configured to, if the model data required for loading the target model does not exist in the local storage space of the service instance in which it is located, obtain the model data from the target service instance that stores the model data and store it in the local storage space. The target model is then loaded based on the model data stored in the local storage space. Embodiments of the present disclosure also provide a computer-readable storage medium storing a computer program. When the computer program is executed by one or more processors, the computer program causes the one or more processors to perform the aforementioned model service method. Embodiments of the present disclosure also provide a computer program product, including the computer program. When the computer program is executed by one or more processors, the computer program causes the one or more processors to perform the aforementioned model service method. In the disclosed embodiments, multiple service instances deployed for model services are organized into a data transmission network. The processing logic for model call requests within each service instance is also modified. Based on this, from the perspective of a single service instance, upon receiving a model call request, it can first query its local storage space for the model data required for model loading. If not, it can attempt to remotely download the required model data from other service instances via the data transmission network and store the downloaded model data in its local storage space. In this way, in a multi-model dynamic loading scenario, model data sharing can be achieved via the data transmission network established between multiple service instances. Because the network transmission rate between service instances in the data transmission network is much higher than the back-to-source download rate, even if a service instance does not have the required model data locally, it can still efficiently download the model data remotely from other service instances via the data transmission network. This effectively improves model loading efficiency and, in turn, the efficiency of responding to model call requests. BRIEF DESCRIPTION OF THE DRAWINGS The drawings described herein are used to provide a further understanding of the present disclosure and constitute a part of the present disclosure. The illustrative embodiments of the present disclosure and their descriptions are used to explain the present disclosure and do not constitute an improper limitation on the present disclosure.In the accompanying drawings: Figure 1 is a flow chart of a model service method provided by an exemplary embodiment of the present disclosure; Figure 2 is a logic diagram of a model service method provided by an exemplary embodiment of the present disclosure; Figure 3 is a schematic diagram of an exemplary internal logical structure of a model service device provided by an exemplary embodiment of the present disclosure; and Figure 4 is a schematic diagram of the structure of a computing device provided by another exemplary embodiment of the present disclosure. DETAILED DESCRIPTION To further clarify the objectives, technical solutions, and advantages of the present disclosure, the technical solutions of the present disclosure will be clearly and completely described below in conjunction with the specific embodiments of the present disclosure and the corresponding drawings. Obviously, the described embodiments are only some of the embodiments of the present disclosure, and are not exhaustive. All other embodiments derived by persons of ordinary skill in the art based on the embodiments of the present disclosure without inventive effort are within the scope of protection of the present disclosure. Before describing the technical solutions provided by the various embodiments of the present disclosure, several technical concepts that may be involved in the present disclosure are briefly explained below. Model servitization refers to the provision of external services through API interfaces and other means after the model is deployed, allowing other services or clients to call it. Model service: A model service is a software service based on a machine learning model. It provides prediction or inference capabilities for machine learning models by encapsulating them into callable APIs or services. Model loading: This refers to loading a machine learning model from a storage medium (such as a disk, database, or cloud storage) into a service instance for use in inference or other machine learning tasks. The model loading process requires the model data corresponding to the machine learning model, also known as the model file. This model data includes various data required for model loading, such as model parameters. Service instance: A service instance is a resource instance deployed to implement a model service. This embodiment does not limit the type of service instance. For example, a service instance can be implemented as a resource group (POD), container, virtual machine, or bare metal server. As described in the background technology, dynamic model loading mechanisms have been proposed in model servitization technology. From the perspective of a single service instance, different model invocation requests it receives may require different models, requiring the service instance to dynamically switch the loaded model on demand. Currently, the number of models involved in the dynamic model loading mechanism is increasing, while the local storage space of the service instance itself is limited. Therefore, after receiving each model call request, the service instance almost always needs to perform a back-to-source download to complete the model loading. The back-to-source download is relatively time-consuming, resulting in the service instance's slow response to the model call request.To this end, this embodiment proposes a model service method to improve the efficiency of responding to model call requests in a multi-model dynamic loading scenario. The following, in conjunction with the accompanying drawings, details the technical solutions provided by various embodiments of the present disclosure. Figure 1 is a flow chart of a model service method provided by an exemplary embodiment of the present disclosure. This method can be executed by a model service device, which can be implemented as software, hardware, or a combination of software and hardware. The model service device can run in each service instance deployed for the model service. Figure 2 is a logical diagram of a model service method provided by an exemplary embodiment of the present disclosure. Referring to Figure 2, in this embodiment, a distributed service system can be established for the model service, which includes the multiple service instances described above. In practical applications, architectures such as Kubernetes (K8S) can be used to orchestrate and manage multiple service instances in the distributed service system, but this embodiment does not limit this. Model call requests submitted to the model service reach service instances in the distributed service system through a distributed scheduling mechanism. In other words, the service instances will respond to the model call requests. As described in the background, model call requests may originate from other services or clients, and this embodiment does not limit the initiator of model call requests. Furthermore, as mentioned above, in a multi-model dynamic switching scenario, any service instance deployed for the model service can be used to dynamically load models. Dynamic model loading can be understood as dynamically switching the loaded model based on the received model call request to adapt to the model call request. For ease of description, the model service method provided in this embodiment will be described in detail below using any service instance deployed by the model service as an example. However, it should be understood that each service instance deployed for the model service can run the model service device provided in this embodiment to execute the model service method provided in this embodiment. Referring to Figure 2, this embodiment, while maintaining the distributed service system, proposes pre-organizing multiple service instances deployed for the model service into a data transmission network. The data transmission network in this embodiment can be a peer-to-peer network or other network capable of supporting data transmission between service instances. This embodiment does not limit the type of data transmission network. Peer-to-peer (P2P) networking is a network architecture that allows network participants (peers) to directly share and use their hardware resources, such as processor power, storage space, network connectivity, and printers. In this network architecture, participants act as both providers and consumers of resources.It should be understood that the data transmission network established in this embodiment does not destroy the existing network relationships between service instances in a distributed service system. Instead, it can be understood as an extended network (overlay network) built on top of the existing network relationships. Furthermore, it should be understood that multiple service instances deployed for a model service are typically located within the same virtual private network (VPC). Therefore, based on the data transmission network established in this embodiment, multiple service instances can utilize existing VPC network resources for data transmission, thereby achieving relatively high network transmission rates between service instances. Based on this, with reference to Figure 1 , the model service method provided in this embodiment may include: Step 100: Determining a target model to be loaded in response to a received model call request; Step 102: If the model data required for loading the target model does not exist in the local storage space, obtaining the model data from the target service instance storing the model data and storing it in the local storage space. Multiple service instances are pre-established as a data transmission network; Step 103: Loading the target model based on the model data stored in the local storage space. As mentioned above, the model service method provided in this embodiment can be executed by a model service device. Figure 3 is a schematic diagram of an exemplary internal logical structure of a model service device provided in an exemplary embodiment of the present disclosure. Referring to Figure 3, in this embodiment, the model service device may include at least a service module, a processing module, and a network assembly module. It should be understood that this is an exemplary division of modules within the model service device and this embodiment is not limited thereto. The model service method provided in this embodiment will be described in detail below in conjunction with the various modules included in the model service device. Referring to Figure 3, the network assembly module can be configured to control the service instance in which it is located to establish a data transmission network with other service instances deployed for the model service. Referring to Figure 3, after a service instance is created, the network assembly module deployed within it can request the aforementioned control instance for a list of service instances corresponding to the model service to obtain information about other service instances deployed for the model service. Based on this information, the current service instance can be controlled to join the data transmission network already established by each service instance in the service instance list. In this way, based on the network assembly module in each service instance, it can be ensured that all service instances created for the model service can be successfully added to the corresponding data transmission network. Moreover, based on the network assembly module in each service instance, the data transmission networks between different model services can be isolated from each other to ensure the data security in the service instance.Optionally, in this embodiment, the network assembly module may utilize decentralized tools such as serf to implement the assembly of the data transmission network. It should be understood that this embodiment is not limited to this. Referring to Figures 1 and 3 , the service module may execute the aforementioned step 100. Furthermore, referring to Figure 3 , in actual applications, the distributed service system may also include a management and control instance for service discovery. This management and control instance may be used to discover information about each service instance deployed for the model service and maintain a service instance list for the model service. This management and control instance may control the provision of services by each service instance in the service instance list. In some exemplary designs, a network proxy may be deployed within the management and control instance, and all model call requests sent to the model service may be distributed by this network proxy. Referring to Figure 3 , the model call requests will be distributed to the service instances. In step 100, the service module may determine the target model to load in response to the received model call request. As mentioned above, different model call requests may require different models to be loaded. If the target model required to be loaded in the model call request received in step 100 differs from the model required to be loaded in the previous model call request, a model switch occurs in the service instance. In this embodiment, local storage space can be requested for the service instance. It should be understood that the service instance is hosted on a physical server, and based on virtualization technology, the hardware resources (such as memory, hard disk, etc.) on the physical server can be provided to the service instance for use. Based on this, in this embodiment, the local storage space requested for the service instance can be used to provide the service instance with local caching capabilities. In this way, the service instance can use its local storage space to temporarily store model data corresponding to previously loaded models. Continuing with Figures 1 and 3, steps 101 and 102 can be implemented by the processing module. In step 101, it is first determined whether the model data required for loading the target model exists in the local storage space of the service instance. However, during research, the inventors discovered that the local storage space allocated for a service instance is typically limited. Therefore, the local storage space can only temporarily store model data for a limited number of models, far fewer than the number of models required in a multi-model dynamic loading scenario. This results in a low probability of finding the required model data in the local storage space of a service instance. Therefore, referring to FIG2 , in step 101, it is further proposed that, if the model data required for loading the target model does not exist in the local storage space, an attempt may be made to obtain the model data from other service instances deployed for the model service.Based on the pre-established data transmission network, the multiple service instances deployed for the model service are able to communicate with each other. In step 101, this communication capability is fully utilized to enable model data sharing between the multiple service instances. Therefore, if the required model data is stored on any other service instance deployed for the model service, the required model data can be retrieved from the target service instance storing the model data in step 101 based on the data transmission network established between the service instances. Referring to Figure 3 , the model service device provided in this embodiment may also include a remote download module. Based on this, a remote download module can be used to execute operations to retrieve the required model data from the target service instance. Referring to Figure 3 , after determining that the required model data does not exist in the local storage space, the processing module can trigger the remote download module to query whether the required model data exists on other service instances. If so, a remote download instruction can be initiated to the targeted service instance to retrieve the required model data from the target service instance and store it in the local storage space. If the remote download module finds that the required model data does not exist on other service instances, it can provide feedback to the processing module, allowing the processing module to trigger other remedial logic. Accordingly, a service instance may also receive remote download instructions from other service instances in the data transmission network. If a service instance receives a remote download instruction from any other service instance in the data transmission network, it determines the target model data requested by the remote download instruction, reads the target model data from local storage, and transmits the read target model data to the service instance that issued the remote download instruction via the data transmission network. Referring to Figure 3, communication channels exist between the remote download modules in each service instance in the data transmission network. Based on this, the remote download modules can send remote download instructions to each other to achieve mutual transfer of model data, i.e., model data sharing. During their research, the inventors discovered that although the model data that can be temporarily stored in the local storage space of a single service instance is limited, from the global perspective of multiple service instances deployed for the model service, the temporarily stored model data is relatively comprehensive. This is because the model data temporarily stored in different service instances may differ. Therefore, the model data corresponding to models previously provided by a model service is typically not overwritten. In particular, the model data corresponding to models recently provided by a model service is generally stored on a particular service instance. Therefore, in step 101, even if the required model data does not exist in the local storage space of a service instance, it can usually be obtained from other service instances.Furthermore, during their research, the inventors discovered that multiple service instances are typically located within the same virtual private network (VPN), ensuring excellent network transmission rates between service instances. Therefore, a service instance can efficiently obtain model data from other service instances, and this process is extremely time-consuming. After obtaining the required model data from other service instances, the service instance can store the obtained model data in local storage. Continuing with Figure 1 , in step 102, the target model can be loaded based on the model data stored in the local storage. That is, after the model data required for loading the target model is stored in the service instance's local storage, the service instance completes loading the target model. After loading the target model, the target model can be used to respond to the aforementioned model call request. In actual applications, a model call request typically carries an inference task. The loaded target model can complete the inference task, and the service instance can use the processing results generated by the target model as the response to the model call request. It is understandable that if a service instance subsequently receives other model call requests and the requested model calls no longer require the target model to be loaded, but a new model, the service instance must load the new model according to the model service method provided in this embodiment to achieve efficient model switching. Furthermore, in this embodiment, if the model data does not exist in any other service instances in the data transmission network, the model data can be downloaded from the data source and stored in the local storage space. As mentioned above, referring to FIG3 , if the remote download module finds that the required model data does not exist in any other service instances, it can provide feedback to the processing module. Upon receiving this feedback, the processing module can trigger the remedial logic of downloading the model data from the data source. Referring to FIG3 , the model service device provided in this embodiment can also include a back-to-source download module. Upon receiving this feedback from the remote download module, the processing module can trigger the back-to-source download module to download the required model data from the data source and store it in the local storage space. The data source stores various model data and can be understood as a relatively comprehensive model library. This way, the data source can provide a guaranteed return for model call requests. It should be understood that the network transmission rate between the data source and the service instance is typically low, so the time required for the service instance to perform back-to-source downloads will be relatively long.However, as mentioned above, based on the model service method provided in this embodiment, upon receiving a model call request, the local storage space is first queried. If the required model data does not exist in the local storage space, the required model data is attempted to be obtained from other service instances. In actual applications, the required model data is likely to be obtained through the direct model data sharing mechanism among multiple service instances in the data transmission network, thus effectively reducing the probability of back-to-source calls. In summary, this embodiment proposes organizing the multiple service instances deployed for the model service into a data transmission network and installing the model service device provided in this embodiment within the service instances. This allows for the modification of the processing logic for model call requests within the service instances. Based on this, from the perspective of a single service instance, upon receiving a model call request, the local storage space is first queried to determine whether the model data required for model loading exists. If not, the required model data can be remotely downloaded from other service instances via the data transmission network and stored in the local storage space. In this way, in a multi-model dynamic loading scenario, model data sharing can be achieved based on the data transmission network established between multiple service instances. Since the network transmission rate between service instances in the data transmission network is much higher than the back-to-source download rate, even if a service instance does not have the model data required for loading locally, it can still efficiently download the model data remotely from other service instances via the data transmission network. This effectively improves model loading efficiency and, in turn, the efficiency of responding to model call requests. In the above or following embodiments, it is also proposed that storage status information can be created for each service instance in the data transmission network. This storage status information can be used to record descriptions of the model data stored in the corresponding service instance. In this way, the storage status information can reflect the model data stored in the service instance. Based on this, this embodiment proposes that within a service instance, storage status information can be maintained for each service instance in the data transmission network. Furthermore, if the model data required for loading a target model does not exist in the aforementioned local storage space, the storage status information maintained by each service instance in the data transmission network can be queried to select the target service instance that stores the model data. Furthermore, this embodiment also introduces a broadcast mechanism in the data transmission network: for a service instance, after storing the model data in the local storage space, it can broadcast to other service instances in the data transmission network, so that other service instances in the data transmission network can update the storage status information corresponding to the service instance.That is, after any service instance in the data transmission network stores model data in its local storage space, it can, based on the broadcast mechanism introduced in the data transmission network, broadcast the model data storage event to other service instances. This can trigger other service instances to promptly update the storage status information they maintain for that service instance. In this way, by introducing a broadcast mechanism in the data transmission network and utilizing it to promptly synchronize the model data storage status of each service instance in the data transmission network, the storage status information maintained by each service instance in the data transmission network can be maintained consistent and updated in a timely manner, thereby ensuring accurate model data sharing between service instances. Optionally, in this embodiment, the broadcast mechanism can be implemented between service instances in the data transmission network based on a gossip protocol, for example. This embodiment does not limit the broadcast protocol used in the broadcast mechanism. Based on the internal logical structure of the model service device in FIG3 , the processing module can maintain the aforementioned storage status information, and the network assembly module can perform the broadcast operation. Based on this, after completing the model data storage operation, the processing module can trigger the network component module to perform a broadcast operation. Upon receiving broadcasts from other service instances, the network component module can trigger the processing module to promptly update the storage status information maintained for the corresponding service instance. In this way, the processing module and the network component module work together to ensure that the storage status information maintained by each service instance in the data transmission network remains consistent and updated in a timely manner. This also provides the processing module with an accurate basis for ensuring that the selected target service instance actually stores the required model data. Furthermore, during research, the inventors discovered that there may be multiple service instances in the data transmission network storing the same model data. In other words, more than one service instance in the data transmission network may be able to provide the model data required for loading the target model. To address this, this embodiment further proposes a remote download instruction balancing mechanism: if more than one service instance stores the same model data, the service instance with the lowest load will respond to the remote download instruction for that model data. In an exemplary balancing solution, each service instance in the data transmission network can broadcast its load status to each other based on the broadcast mechanism introduced in the aforementioned network assembly module. This allows a single service instance to maintain load status information for each of the other service instances in the data transmission network. Accordingly, the load status information can record information such as the number of remote download instructions loaded by the corresponding service instance, thereby reflecting the load status of the corresponding service instance.Based on this, if a service instance needs to remotely download model data, and if there are multiple service instances storing the model data in the data transmission network, the current service instance can identify the service instance that stores the required model data and has the lowest load as the target service instance and initiate a remote download instruction to the target service instance. This allows remote download instructions for the same model data to be distributed to related service instances for response, avoiding concentrating them on a single service instance and thus burdening it. In summary, this embodiment introduces a broadcast mechanism into the data transmission network and innovatively proposes broadcasting the storage status of service instances based on this mechanism. This ensures that each service instance in the data transmission network can accurately perceive the model data present in the data transmission network and the service instance or instances where the model data is stored, thereby enabling service instances to seamlessly identify the service instance that stores the required model data. Furthermore, this innovatively proposes broadcasting the load status of service instances based on this mechanism, ensuring that each service instance in the data transmission network can accurately perceive the load status of other service instances. This allows load balancing of remote download instructions and avoids burdening some service instances. In the above or following embodiments, it is also proposed that a user-space file system can be introduced to manage the model data stored in the service instance. To this end, referring to Figure 3 , a user-space file system (FUSE) component can be configured in the processing module of the service instance, and based on this component, a user-space file system can be established in the service instance. It should be understood that the user-space file system pre-creates file paths for each model data that may be used by the service instance, and these file paths can be maintained in the metadata information corresponding to the user-space file system. The implementation principles of the user-space file system will not be elaborated on here. Based on this, referring to Figure 3 , this embodiment further proposes that the aforementioned data source can be mounted in the user-space file system of the service instance. Furthermore, the metadata information corresponding to the user-space file system can record the file description information corresponding to the model data in the data source, including but not limited to the file identifier, file specification, file type, and storage location. In this way, when the processing module needs to download model data from the data source, it can query the metadata information maintained for the user-space file system for the first file description information corresponding to the model data in the data source and download the model data from the data source based on the first file description information. 3 , the processing module may provide the first file description information to the back-to-source download module, and the back-to-source download module may download data matching the first file description information from the data source, ie, the model data required for loading the target model.This ensures that the required model data is correctly downloaded from the data source. Referring to Figure 3, this embodiment also proposes that the local storage space in the service instance can be mounted to the user-mode file system in the service instance. Furthermore, the metadata information corresponding to the user-mode file system can record the file description information corresponding to the model data in the local storage space, including but not limited to the file identifier, file specification, file type, and storage location. In this way, the processing module can provide an access interface to the service module based on the user-mode file system created for the service instance. Upon receiving a model call request, the service module can query the user-mode file system for the file path of the required model data and, based on the retrieved file path, initiate a file access request to the processing module's access interface. Upon receiving the file access request, the processing module can query the metadata maintained for the user-mode file system for the second file description information corresponding to the required model data in the local storage space. Furthermore, the processing module can query whether data matching the second file description information exists in the local storage space. If so, it can be determined that the required model data already exists in the local storage space. If not, it can be determined that the required model data does not exist in the local storage space. In summary, this embodiment introduces a user-mode file system and mounts a local storage space on this user-mode file system. For the target model required for loading a model usage request, the user-mode file system provides the file path corresponding to the model data required for loading the target model. However, the model data may not necessarily already exist in the local storage space. Therefore, this embodiment proposes, based on the mounting relationship between the user-mode file system and the local storage space, to conveniently and quickly query whether the required model data exists in the local storage space. If not, the required model data can be promptly downloaded from the target service instance storing the model data. If other service instances in the data transmission network do not store the required model data, a back-to-source download operation can be triggered. This embodiment also proposes mounting the data source on the user-mode file system, which ensures that the required model data can be accurately downloaded during the back-to-source download process. As can be seen, by introducing the user-mode file system, data access operations to the current storage space and data source can be converted into unified user-mode file system access operations. This not only ensures accurate use of the required model data, but also effectively reduces the logical complexity within the service instance and improves the service instance's efficiency in responding to model call requests.It should be noted that some of the processes described in the above embodiments and accompanying drawings include multiple operations that appear in a specific order. However, it should be understood that these operations may be executed in a different order than the order in which they appear herein or in parallel. Operation numbers such as 101 and 102 are merely used to distinguish between different operations and do not represent any specific execution order. Furthermore, these processes may include more or fewer operations, and these operations may be executed sequentially or in parallel. It should be noted that terms such as "first" and "second" are used herein to distinguish between different file descriptions and do not represent a specific order or limit the "first" and "second" to different types. Figure 4 is a schematic diagram of the structure of a computing device according to another exemplary embodiment of the present disclosure. As shown in Figure 4, the computing device includes: a memory 40, a processor 41 and a communication component 42. The computing device carries a service instance deployed for the model service, the service instance is used to dynamically load the model, and the service instance is pre-organized into a data transmission network with other service instance components deployed for the model service; the memory 40 is used to store one or more computer instructions; the processor 41 is coupled with the memory 40 and the communication component 42, and is used to execute the one or more computer instructions to: determine the target model to be loaded in response to a received model call request; if the model data required for loading the target model does not exist in the local storage space, obtain the model data from the target service instance storing the model data and store it in the local storage space, wherein the multiple service instances are pre-organized into a data transmission network; and load the target model based on the model data stored in the local storage space. In an optional embodiment, when obtaining the model data from the target service instance storing the model data, the processor 41 may be specifically configured to: query storage status information maintained for each service instance in the data transmission network to select the target service instance storing the model data; and download the model data from the target service instance; wherein the storage status information records description information of the model data stored in the corresponding service instance. In an optional embodiment, the processor 41 may be further configured to: after storing the model data in the local storage space, broadcast a message carrying identification information of the model data to other service instances in the data transmission network, so that each other service instance updates its own storage status information maintained for the service instance based on the identification information of the model data.In an optional embodiment, the processor 41 may be further configured to: if there are multiple service instances storing the model data in the data transmission network, select the service instance with the lowest load as the target service instance; wherein each service instance in the data transmission network broadcasts its own load status. In an optional embodiment, the processor 41 may be further configured to: if the model data does not exist in any other service instances in the data transmission network, perform a back-to-source download operation to download the model data from the data source; and store the downloaded model data in the local storage space. In an optional embodiment, the data source is mounted on a user-mode file system constructed in the service instance. When performing the back-to-source download operation, the processor 41 may be specifically configured to: query metadata maintained for the user-mode file system for first file description information corresponding to the model data in the data source; and download the model data from the data source according to the first file description information. In an optional embodiment, the local storage space is mounted on a user-mode file system constructed in the service instance. The processor 41 may further be configured to: query metadata maintained for the user-mode file system for second file description information corresponding to the model data in the local storage space; if no data matching the second file description information is found in the local storage space, determine that the model data does not exist in the local storage space. In an optional embodiment, the processor 41 may further be configured to: after the service instance is created, request a list of service instances corresponding to the model service from the management instance deployed for the model service; and join the service instances in the service instance list to the data transmission network established by each service instance. In an optional embodiment, the processor 41 may further be configured to: upon receiving a remote download instruction issued by any other service instance in the data transmission network, determine the target model data requested by the remote download instruction; read the target model data from the local storage space; and send the target model data to the service instance that issued the remote download instruction via the data transmission network. Furthermore, as shown in FIG. 4 , the computing device also includes other components, such as a power supply component 43. FIG4 schematically illustrates only some components and does not necessarily mean that the computing device includes only the components shown in FIG4 . It is worth noting that the technical details of the computing device embodiments described above can be found in the related descriptions of the aforementioned method embodiments. To save space, these details will not be repeated here, but this should not diminish the scope of protection of this disclosure.Accordingly, embodiments of the present disclosure also provide a computer-readable storage medium storing a computer program. When executed, the computer program can implement each step of the above-described method embodiment. Accordingly, embodiments of the present disclosure also provide a computer program product. When executed, the computer program contained therein can implement each step of the above-described method embodiment. The memory in FIG. 4 is used to store the computer program and can be configured to store various other data to support operations on the computing platform. Examples of such data include instructions for any application or method operating on the computing platform, contact data, phone book data, messages, images, videos, etc. The memory can be implemented by any type of volatile or non-volatile storage device, or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk, or optical disk. The communication component in Figure 4 is configured to facilitate wired or wireless communication between the device in which the communication component resides and other devices. The device in which the communication component resides can access a wireless network based on a communication standard, such as Wi-Fi, 2G, 3G, 4G / LTE, 5G, or other mobile communication networks, or a combination thereof. In one exemplary embodiment, the communication component receives broadcast signals or broadcast-related information from an external broadcast management system via a broadcast channel. In one exemplary embodiment, the communication component also includes a near-field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID), infrared data association (IrDA), ultra-wideband (UWB), Bluetooth (BT), or other technologies. The power supply component in Figure 4 provides power to various components of the device in which the power supply component resides. The power supply component may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to the device in which the power supply component resides. Those skilled in the art will appreciate that embodiments of the present disclosure may be provided as methods, systems, or computer program products. Therefore, the present disclosure may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present disclosure may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.The present disclosure is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present disclosure. It should be understood that each process and / or block in the flowcharts and / or block diagrams, as well as combinations of processes and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, such that execution of the instructions by the processor of the computer or other programmable data processing device produces means for implementing the functions specified in one or more processes in the flowcharts and / or one or more blocks in the block diagrams. These computer program instructions can also be stored in a computer-readable memory capable of directing the computer or other programmable data processing device to operate in a specific manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means that implement the functions specified in one or more processes in the flowcharts and / or one or more blocks in the block diagrams. These computer program instructions can also be loaded onto a computer or other programmable data processing device, causing the computer or other programmable device to execute a series of operational steps to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device with steps for implementing the functions specified in one or more flow charts and / or one or more blocks in the block diagrams. It should also be noted that the terms "comprise," "include," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, commodity, or apparatus comprising a series of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, commodity, or apparatus. In the absence of further limitations, an element defined by the phrase "comprising a..." does not preclude the presence of additional identical elements in the process, method, commodity, or apparatus comprising the element. It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, storage, and display) involved in this disclosure are all information and data authorized by the user or fully authorized by all parties. The collection, use, and processing of relevant data must comply with the relevant laws, regulations, and standards of relevant countries and regions, and corresponding operation portals are provided for users to choose to authorize or reject. The above description is merely an embodiment of the present disclosure and is not intended to limit the present disclosure. Those skilled in the art will appreciate that various modifications and variations of the present disclosure are possible.Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of this disclosure should be included in the scope of protection of this disclosure.

Claims

Claims 1. A model service method, wherein a plurality of service instances are deployed for a model service, the method being applicable to any one of the plurality of service instances, the method comprising: In response to the received model call request, determining a target model to be loaded; If the model data required for loading the target model does not exist in the local storage space, the model data is obtained from the target service instance storing the model data and stored in the local storage space, wherein the multiple service instances are pre-organized into a data transmission network; and the target model is loaded based on the model data stored in the local storage space.

2. The method according to claim 1, obtaining the model data from a target service instance storing the model data, comprising: querying storage status information maintained for each service instance in the data transmission network to select the target service instance storing the model data; Downloading the model data from the target service instance; The storage status information is used to record description information of the model data stored in the corresponding service instance.

3. The method according to claim 2, further comprising: After storing the model data in the local storage space, a broadcast carrying identification information of the model data is sent to other service instances in the data transmission network, so that each other service instance updates the storage status information maintained by itself for the service instance based on the identification information of the model data.

4. The method according to any one of claims 1 to 3, further comprising: If there are multiple service instances storing the model data in the data transmission network, selecting the service instance with the lowest load as the target service instance; Each service instance in the data transmission network broadcasts its own load status respectively.

5. The method according to any one of claims 1 to 4, further comprising: If the model data does not exist in any other service instance in the data transmission network, performing a back-to-source download operation to download the model data from the data source; The downloaded model data is stored in the local storage space.

6. The method according to claim 5, wherein the data source is mounted on a user-mode file system constructed in the service instance, and the back-to-source download operation comprises: Querying first file description information corresponding to the model data in the data source from metadata information maintained for the user-mode file system; The model data is downloaded from the data source according to the first file description information.

7. The method according to any one of claims 1 to 6, wherein the local storage space is mounted on a user-mode file system constructed in the service instance, and the method further comprises: Querying the second file description information corresponding to the model data in the local storage space from the metadata information maintained for the user-mode file system; If no data matching the second file description information is found in the local storage space, it is determined that the model data does not exist in the local storage space.

8. The method according to any one of claims 1 to 7, further comprising: After the service instance is created, requesting a list of service instances corresponding to the model service from the management and control instance deployed for the model service; Join the data transmission network that has been established by each service instance in the service instance list.

9. The method according to any one of claims 1 to 8, further comprising: If a remote download instruction is received from any other service instance in the data transmission network, the target model data requested by the remote download instruction is determined; the target model data is read from the local storage space; and the target model data is sent to the service instance that issued the remote download instruction via the data transmission network.

10. A computing device comprising a memory, a processor, and a communication component, wherein the computing device carries a service instance deployed for a model service, the service instance being used to dynamically load a model, and the service instance pre-connecting to a data transmission network with other service instance components deployed for the model service; the memory being used to store one or more computer instructions; and the processor being coupled to the memory and the communication component and configured to execute the one or more computer instructions to perform the model service method according to any one of claims 1 to 9.

11. A model service device, respectively arranged in multiple service instances deployed for a model service, the model service device comprising a service module, a processing model and a network assembly module; the network assembly module is used to control the service instance in which it is located to establish a data transmission network with other service instances deployed for the model service; the service module is used to determine a target model to be loaded in response to a received model call request; the processing module is used to obtain the model data from the target service instance storing the model data and store it in the local storage space if the model data required for loading the target model does not exist in the local storage space of the service instance in which it is located; and load the target model based on the model data stored in the local storage space.

12. A computer-readable storage medium storing a computer program, which, when executed by one or more processors, causes the one or more processors to execute the model service method according to any one of claims 1 to 9.

13. A computer program product, comprising a computer program, which, when executed by one or more processors, causes the one or more processors to execute the model service method according to any one of claims 1 to 9.

Images