Dual-strategy model calling method and device, electronic equipment and storage medium

Abstract

The embodiment of the invention relates to a double-strategy model calling method and device, electronic equipment and a storage medium. The method comprises the steps that a calling request for a target model is received; determining a loading strategy corresponding to the target model according to the calling request; if the loading strategy is an on-demand loading strategy, determining first target equipment capable of loading the target model from a preset equipment cluster, and loading the target model into the first target equipment; if the loading strategy is a pre-loading strategy, determining a preset number of devices pre-loaded with the target model from a device cluster; determining a second target device from the preset number of devices; and in response to to-be-processed data sent by the sending end receiving the calling request, processing the to-be-processed data by using the target model on the first target device or the second target device. According to the embodiment of the invention, the resource utilization rate of the service containing a plurality of models is improved, the response speed of containing fewer models is also improved, and the application scene of the scheme is expanded at the same time.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a dual-strategy model calling method, device, electronic device and computer-readable storage medium. Background Art

[0002] In actual model prediction business scenarios, we usually encounter two types of business. One type of business requires providing multiple models for users to choose from. For example, the text and image business is a business where users provide text or other content and use various models to generate images. The text and image business provides hundreds of models for users to choose from, but if a server is allocated to each model, it will cause a huge waste of computing resources and high costs.

[0003] Another type of business only needs to provide one model, and the response time of the model cannot be too long. For example, in a business that adds special effects, style changes, etc. to images uploaded by users, there is only one model for image processing. If the user loads the model to the server when using this business, the response time to the user will be too long, and the efficiency will be low.

[0004] Therefore, how to meet the model usage requirements of different businesses on limited hardware devices, improve the utilization efficiency of the equipment, and avoid long response time, is a problem that needs to be solved at present. Summary of the invention

[0005] In view of this, in order to solve some or all of the above technical problems, the embodiments of the present application provide a dual-strategy model calling method, device, electronic device and computer-readable storage medium.

[0006] In a first aspect, an embodiment of the present application provides a dual-strategy model calling method, the method comprising: receiving a calling request for a target model; determining, based on the calling request, a loading strategy corresponding to the target model; if the loading strategy is an on-demand loading strategy, determining, from a preset device cluster, a first target device on which the target model can be loaded, and loading the target model into the first target device; if the loading strategy is a preloading strategy, determining, from the device cluster, a preset number of devices on which the target model is preloaded; determining a second target device from the preset number of devices; and in response to the data to be processed sent by the sending end that receives the calling request, processing the data to be processed using the target model on the first target device or the second target device.

[0007] In one possible implementation, determining a first target device on which a target model can be loaded from a preset device cluster includes: determining from the preset device cluster whether there is a device on which the target model is currently loaded; if not, determining from the device cluster a device of an idle type and in a running state as the first target device; if so, determining the first target device from a device of a loaded type and on which the target model is loaded.

[0008] In one possible implementation, loading the target model into the first target device includes: if the type of the first target device is an idle type, setting the type of the target model loading task of the target model to a normal loading task type, and setting the state of the target model loading task to a waiting state, and starting to load the target model to the first target device; adjusting the state of the target model loading task to a model loading state, and adjusting the state of the first target device from a running state to a model loading state; in response to successful loading of the target model, adjusting the type of the first target device to a loaded type, adjusting the state of the first target device to a running state, and adjusting the state of the target model loading task to a task loading completion state.

[0009] In a possible implementation, after starting to load the target model to the first target device, the method further includes: in response to a target model loading failure, adjusting the loading task type to a task loading failure state, and sending a loading failure prompt message to the sender of the call request.

[0010] In one possible implementation, determining a first target device on which a target model can be loaded from a preset device cluster includes: determining whether the number of model loading tasks contained in a model loading task queue corresponding to the target model reaches a preset number of tasks; if the preset number of tasks is reached, determining whether the number of devices on which the target model can be loaded reaches a preset number of devices; if the preset number of devices is not reached, determining from the device cluster a device whose device type is an idle type and whose device status is a running status as the first target device; if the preset number of devices is reached, determining the first target device from the devices on which the target model has been loaded.

[0011] In one possible embodiment, loading the target model into the first target device includes: if the number of devices loading the target model does not reach a preset number of devices, setting the type of the target model loading task of the target model to a temporary loading task type, and setting the state of the target model loading task to a waiting state, and starting to load the target model to the first target device; adjusting the state of the target model loading task to a model loading state, and adjusting the state of the first target device from a running state to a model loading state; in response to successful loading of the target model, adjusting the type of the first target device to a temporary device type, adjusting the state of the first target device to a running state, and adjusting the state of the target model loading task to a task loading completed state; in response to a target model loading failure, adjusting the state of the first target device to a running state, and adjusting the state of the target model loading task to a task loading failed state.

[0012] In one possible embodiment, in response to successful target model loading, the type of the first target device is adjusted to a temporary device type, the state of the first target device is adjusted to a running state, and the state of the target model loading task is adjusted to a task loading completion state. The method also includes: if the number of model loading tasks contained in the model loading task queue does not reach a preset task number, adjusting the type of the first target device to an idle type.

[0013] In one possible implementation, after loading the target model into the first target device, the method further includes: generating first identification information for associating the first target device and the target model, and setting the validity period of the identification information to a preset duration; and sending the identification information to the sender of the call request.

[0014] In a possible implementation, after setting the validity period of the identification information to a preset duration, the method further includes: if all first identification information associated with the first target device has expired, adjusting the type of the first target device from a loaded type to an idle type.

[0015] In one possible implementation, before receiving a call request for the target model, the method further includes: determining, from a device cluster, a preset number of devices on which the target model can be loaded according to a preloading strategy, and loading the target model into a preset number of second target devices; generating second identification information for associating a preset number of devices and the target model, and setting the validity period of the identification information to be permanent.

[0016] In the second aspect, an embodiment of the present application provides a dual-strategy model calling device, which includes: a receiving module, used to receive a calling request for a target model; a first determination module, used to determine the loading strategy corresponding to the target model according to the calling request; a second determination module, used to determine the first target device on which the target model can be loaded from a preset device cluster if the loading strategy is an on-demand loading strategy, and load the target model into the first target device; a third determination module, used to determine a preset number of devices on which the target model is pre-loaded from the device cluster if the loading strategy is a pre-loading strategy; a fourth determination module, used to determine the second target device from a preset number of devices; a processing module, used to process the data to be processed using the target model on the first target device or the second target device in response to the data to be processed sent by the sending end that receives the calling request.

[0017] In a third aspect, an embodiment of the present application provides an electronic device, comprising: a memory for storing a computer program; a processor for executing the computer program stored in the memory, and when the computer program is executed, the method of any embodiment of the dual-strategy model calling method of the first aspect of the present application is implemented.

[0018] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the method of any embodiment of the dual-strategy model calling method of the first aspect described above is implemented.

[0019] In a fifth aspect, an embodiment of the present application provides a computer program, which includes a computer readable code. When the computer readable code runs on a device, the processor in the device implements a method as in any embodiment of the dual-strategy model calling method of the first aspect above.

[0020] The dual-strategy model calling method, device, electronic device and computer-readable storage medium provided in the embodiment of the present application receive a call request for a target model, determine the loading strategy corresponding to the target model according to the call request, if it is an on-demand loading strategy, determine the first target device that can load the target model from the preset device cluster, and load the target model into the first target device, if it is a pre-loading strategy, determine a preset number of devices pre-loaded with the target model from the device cluster, and determine the second target device therefrom, and finally use the target model on the first target device or the second target device to process the data to be processed. The embodiment of the present application realizes setting two model loading strategies in limited device resources. For businesses with a large number of models, an on-demand loading strategy is used, that is, the currently called model is loaded in real time; for businesses with a small number of models, a pre-loading strategy is used, that is, the model to be called is pre-loaded into the device, thereby realizing support for two different business types, which not only improves the resource utilization of businesses containing multiple models, but also improves the response speed of businesses containing fewer models, and at the same time expands the applicable scenarios of this solution. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0022] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0023] One or more embodiments are exemplarily described by pictures in the corresponding drawings, and these exemplified descriptions do not constitute limitations on the embodiments. Elements with the same reference numerals in the drawings represent similar elements, and unless otherwise stated, the figures in the drawings do not constitute proportional limitations.

[0024] Figure 1 A schematic diagram of a dual-strategy model calling method provided in an embodiment of the present application;

[0025] Figure 2 A flowchart of another dual-strategy model calling method provided in an embodiment of the present application;

[0026] Figure 3 A schematic diagram of the flow of device type, device status, task type and task status when loading a model provided in an embodiment of the present application;

[0027] Figure 4A flowchart of another dual-strategy model calling method provided in an embodiment of the present application;

[0028] Figure 5 A schematic diagram of the state flow process when expanding the capacity of a device loaded with a target model provided in an embodiment of the present application;

[0029] Figure 6 A flowchart of another dual-strategy model calling method provided in an embodiment of the present application;

[0030] Figure 7 A flowchart of determining a target device and loading a model onto the target device under different loading strategies provided in the embodiments of the present application;

[0031] Figure 8 A schematic diagram of the structure of a dual-strategy model calling device provided in an embodiment of the present application;

[0032] Fig. 9 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0033] Various exemplary embodiments of the present application will now be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are part of the embodiments of the present application, rather than all of the embodiments. It should be noted that unless otherwise specifically stated, the relative arrangement of the components and steps, the numerical expressions and the numerical values ​​described in these embodiments do not limit the scope of the present application.

[0034] Those skilled in the art will understand that the terms "first" and "second" in the embodiments of the present application are only used to distinguish between different steps, devices, modules and other objects, and do not represent any specific technical meanings, nor do they indicate the logical order between them.

[0035] It should also be understood that in this embodiment, “plurality” may refer to two or more than two, and “at least one” may refer to one, two or more than two.

[0036] It should also be understood that any component, data or structure mentioned in the embodiments of the present application can generally be understood as one or more, unless explicitly limited or otherwise indicated in the context.

[0037] In addition, the term "and / or" in this application is only a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this application generally indicates that the associated objects before and after are in an "or" relationship.

[0038] It should also be understood that the description of the various embodiments in this application focuses on the differences between the various embodiments, and the same or similar aspects thereof can be referenced to each other, and for the sake of brevity, they will not be described one by one.

[0039] The following description of at least one exemplary embodiment is merely illustrative in nature and is in no way intended to limit the present application, its application, or uses.

[0040] Technologies, circuits, and devices known to ordinary technicians in the relevant art may not be discussed in detail, but where appropriate, the above-mentioned technologies, circuits, and devices should be considered as part of the specification.

[0041] It should be noted that like reference numerals and letters refer to similar items in the following figures, and therefore, once an item is defined in one figure, it need not be further discussed in subsequent figures.

[0042] It should be noted that, in the absence of conflict, the embodiments in this application and the features in the embodiments can be combined with each other. To facilitate the understanding of the embodiments of the present application, the present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments. Obviously, the described embodiments are part of the embodiments of the present application, rather than all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.

[0043] In order to solve the technical problems that the existing technology cannot use limited hardware resources to support different types of businesses, and has low efficiency and slow response speed when calling models of different businesses, the present application provides a dual-strategy model calling method, which can use different model loading strategies according to different business types, thereby improving the resource utilization of businesses involving multiple models, and improving the response speed of businesses involving fewer models.

[0044] Figure 1 A flow chart of a dual-strategy model calling method provided for an embodiment of the present application. This method can be applied on a server, and can also be applied to electronic devices such as desktop computers, laptops, and smart phones. When applied on a server, multiple servers for loading models can be configured, and these servers can load models according to different loading strategies based on different business types. When applied on other types of electronic devices, multiple electronic devices for loading models can be configured, and these electronic devices can communicate through a network, and load models according to different loading strategies based on different business types.

[0045] In addition, the execution subject of the present method may be hardware or software. When the execution subject is hardware, the execution subject may be one or more of the electronic devices. For example, a single electronic device may execute the present method, or a plurality of electronic devices may cooperate with each other to execute the present method. When the execution subject is software, the present method may be implemented as a plurality of software or software modules, or as a single software or software module. No specific limitation is made herein.

[0046] like Figure 1 As shown, the method specifically includes:

[0047] Step 101: Receive a call request for a target model.

[0048] In some embodiments, the target model may be a model used in various business scenarios, such as a model included in a text and image business, or a model used in an image processing business. When a user uses a terminal device to call a model, a call request may be sent to an electronic device executing the method, and the type of the target model may be determined according to the call request.

[0049] Step 102: Determine the loading strategy corresponding to the target model according to the call request.

[0050] In some embodiments, the call requests sent in different business scenarios are different. According to the call request, the business type currently used by the user can be determined, and the corresponding loading strategy can be determined according to the business type.

[0051] Step 103: If the loading strategy is an on-demand loading strategy, a first target device on which the target model can be loaded is determined from a preset device cluster, and the target model is loaded into the first target device.

[0052] In some embodiments, the on-demand loading strategy is applicable to business types including multiple models, such as text and graphics business, which requires the use of hundreds of models. In this case, the first target device that can load the target model can be determined in real time from the device cluster according to the call request.

[0053] The above-mentioned device cluster may be a server cluster composed of multiple servers, or may be a device cluster composed of other types of electronic devices connected via a network.

[0054] Some devices in the device cluster can be pre-configured to apply the on-demand loading strategy. When a target model is called, the currently idle device can be determined as the first target device, and the model parameters are obtained from the memory that stores the model parameters, and the model parameters are sent to the first target device, and the first target device completes the loading of the target model. The above-mentioned idle devices are devices that are not currently loaded with a model and can operate normally. Optionally, the effective time for each device to load the model can be set. During the effective time, the device is in a non-idle state, and outside the effective time, the device is in an idle state.

[0055] Step 104 : If the loading strategy is a preloading strategy, a preset number of devices preloaded with the target model are determined from the device cluster.

[0056] In some embodiments, the preloading strategy is applicable to business types that include fewer models, such as image processing business, which only needs one model. In this case, a preset number of devices can be preconfigured and the target model can be preloaded on these devices. According to the business type to which the call request belongs, it can be determined which devices have loaded the target model.

[0057] Step 105: determine a second target device from a preset number of devices.

[0058] In some embodiments, when a call request is received from a user terminal, a device can be selected as a second target device from the devices pre-loaded with the target model according to the call request. The method of selecting the second target device can be implemented in various ways, for example, selecting a device that is currently in an idle state as the second target device, or determining the second target device corresponding to the current moment in sequence from a preset number of devices according to the call time sequence of the target model.

[0059] Step 106, in response to receiving the data to be processed sent by the sending end of the call request, use the target model on the first target device or the second target device to process the data to be processed.

[0060] In some embodiments, after determining the first target device or the second target device loaded with the target model, the first target device or the second target device can continue to receive the data to be processed input by the user end, and use the target model to perform the processing task on the data to be processed. For example, when the target model is a text and image service, the first target device can receive the text sent by the user end, and the target model processes the input text to generate a corresponding image. When the target model is an image processing service, the second target device can receive the image sent by the user end, and the target model processes the input image to obtain an image after adding special effects or image style conversion.

[0061] The dual-strategy model calling method provided in the embodiment of the present application receives a call request for a target model, determines the loading strategy corresponding to the target model according to the call request, and if it is an on-demand loading strategy, determines the first target device that can load the target model from the preset device cluster, and loads the target model into the first target device; if it is a pre-loading strategy, determines a preset number of devices pre-loaded with the target model from the device cluster, and determines the second target device therefrom, and finally uses the target model on the first target device or the second target device to process the data to be processed. The embodiment of the present application realizes setting two model loading strategies in limited device resources. For businesses with a large number of models, an on-demand loading strategy is used, that is, the currently called model is loaded in real time; for businesses with a small number of models, a pre-loading strategy is used, that is, the model to be called is pre-loaded into the device, thereby realizing support for two different business types, which not only improves the resource utilization of businesses containing multiple models, but also improves the response speed of businesses containing fewer models, and at the same time expands the applicable scenarios of this solution.

[0062] In some optional implementations, such as Figure 2 As shown, step 103 includes:

[0063] Step 10301, determine whether there is a device with the target model loaded in the preset device cluster.

[0064] Specifically, when a device has loaded a target model, a mark indicating that the target model has been loaded may be generated, and the electronic device executing the method may determine whether the target model has been loaded in each device based on the mark.

[0065] Step 10302: If it does not exist, determine a device of idle type and in running state from the device cluster as the first target device.

[0066] Specifically, if there is no device with the target model loaded in the device cluster, the type and status of each device can be determined. The type and status of the device can be set according to the loading model. For example, the device type includes the loaded type (valid), the idle type (free), and the temporary type (tmp). The loaded type is the device with the model currently loaded, the idle type is the device with no model currently loaded, and the temporary type is a temporary device that is dynamically expanded according to the task volume.

[0067] The status of the device may include the device creating state (creating), the model loading state (switching), the device running state (running), and the device stopped state (stopped). The device creating state refers to the state in which the device is being configured as the device used by this method, the model loading state refers to the state in which a certain model is currently being loaded, the device running state refers to the state in which the current device is operating normally, and the stopped state refers to the state in which the device has currently stopped executing the functions required by this method.

[0068] In this step, a device of the free type and the running state may be determined from the device cluster as the first target device.

[0069] Step 10303: If it exists, determine the first target device from the devices of the loaded type and loaded with the target model.

[0070] Specifically, if there is a device in the device cluster that has been loaded with the target model, the device can be directly used as the first target device. Optionally, based on the device type and device status shown in the above example, the device that has been loaded with the target model can be determined from the devices of valid type and running status. After determining the first target device in this step, since the target model has been loaded, the target model can be directly used to perform data processing tasks in subsequent steps.

[0071] This embodiment selects the first target device for loading the target model from the device cluster in real time when using the on-demand loading strategy. This enables the target model to be loaded to a certain device in real time when the number of models is large but the number of devices is limited. There is no need to load the model in advance, which saves hardware resources and improves the efficiency of model processing data.

[0072] In some optional implementations, such as Figure 2 As shown, step 103 also includes:

[0073] Step 10304, if the type of the first target device is an idle type, set the type of the target model loading task of the target model to a common loading task type, and set the state of the target model loading task to a waiting state, and start loading the target model to the first target device.

[0074] Specifically, when there is a device in the device cluster that has loaded the target model, a model loading task can be established. The types of model loading tasks can include normal loading task types (normal) and temporary loading task types (tmp). Normal loading task types are common model loading tasks, and temporary loading tasks are temporary loading tasks that are dynamically expanded according to the task volume.

[0075] The status of the model loading task can include waiting state (waiting), model loading state (switching), task loading failure state (failed) and task loading completion state (finished). The waiting state means that the current model loading task is waiting for the device to load the model, the model loading state means that the model is currently being loaded to the device, the task loading failure state means that the current model loading task has failed, and the task loading completion state means that the current model loading task has been successfully loaded.

[0076] In this step, after the first target device is determined, the type of the model loading task can be set to normal, and the state can be set to waiting.

[0077] Step 10305: adjust the state of the target model loading task to the model loading state, and adjust the state of the first target device from the running state to the model loading state.

[0078] After the target model starts to be loaded, the state of the target model loading task can be adjusted to switching, and the state of the first target device is also adjusted to switching.

[0079] Step 10306, in response to the target model being loaded successfully, the type of the first target device is adjusted to a loaded type, the state of the first target device is adjusted to a running state, and the state of the target model loading task is adjusted to a task loading completion state.

[0080] When the loading is successful, the type of the first target device can be adjusted from free type to valid type to prevent other model loading tasks from loading models to the device. The state of the first target device is adjusted from switching to running. The state of the target model loading task can be adjusted from switching to finished. The state of the first target device can remain running, and the type of the target model loading task remains normal.

[0081] Figure 3 The schematic diagram shows the flow of device type and device status, as well as model loading task type and model loading task status when loading a model, wherein steps 301-304 correspond to the process described in this embodiment.

[0082] This embodiment sets the type and status of the model loading task to achieve real-time recording of the current execution status of the model loading task when loading the model to the device, which helps to effectively control the loading process, and allows users to know the execution status of the model loading task, thereby improving the efficiency of model loading.

[0083] In some optional implementations, such as Figure 2 As shown, after step 10304, the method further includes:

[0084] Step 10307: In response to the target model loading failure, the state of the target model loading task is adjusted to a task loading failure state, and a loading failure prompt message is sent to the sender of the call request.

[0085] Specifically, when loading fails, the state of the target model loading task can be adjusted from switching to failed. The type of the target model loading task is still normal. The type of the first target device is still free, and the state of the first target device is still running. Figure 3 As shown, when the loading fails, the type and state of the first target device, and the type and state of the target model loading task are shown as 305.

[0086] When the target model fails to load, a loading failure prompt message is sent to the sender of the call request, so that the user can be informed of the loading failure in time, and the user can respond with reference to the loading failure prompt message.

[0087] This embodiment can realize timely and accurate recording of the status of the model loading task by changing the status of the target model loading task when the target model loading fails, which helps to further effectively respond to the loading process and improve the efficiency of model loading.

[0088] In some optional implementations, such as Figure 4 As shown, step 103 includes:

[0089] Step 10308, determining whether the number of model loading tasks contained in the model loading task queue corresponding to the target model reaches a preset task number.

[0090] Specifically, when the target model is loaded, in order to support multiple users to call the model in parallel, a corresponding model loading task queue can be set, and multiple devices can be set to load the same target model.

[0091] Step 10309: If the preset number of tasks is reached, determine whether the number of devices that load the target model reaches the preset number of devices.

[0092] If the number of model loading tasks in the task queue reaches the preset number of tasks, it means that the current loading tasks for the target model are backlogged. At this time, it is further determined whether the number of devices currently loading the target model has reached the preset number of devices. When determining whether the number of devices loading the target model has reached the preset number of devices, it can be determined based on whether the type of the device is valid and whether the loaded model is the target model.

[0093] It should be noted that if the number of model loading tasks included in the model loading task queue does not reach the preset number of tasks, the target model can be loaded to the first target device after the currently determined first target device completes executing other model loading tasks.

[0094] The above preset number of devices limits the maximum number of devices that load the target model.

[0095] Step 10310: If the preset number of devices is not reached, a device whose device type is an idle type and whose device status is a running status is determined as a first target device from the device cluster.

[0096] This step is basically the same as the execution method of step 10302 above, and will not be repeated here. That is, the device with the device type of free and the state of running is determined as the first target device. When the number of devices loaded with the target model does not exceed the preset number of devices, a new device can be added to load the target model again.

[0097] Step 10311: If the preset number of devices is reached, determine the first target device from the devices of the loaded type and loaded with the target model.

[0098] This step is basically the same as the execution method of step 10303 above, and will not be repeated here. When the number of devices loaded with the target model exceeds the preset number of devices, it is determined that the number of devices loaded with the target model has reached the upper limit, and at this time, any device can be selected from the devices loaded with the target model as the first target device.

[0099] This embodiment sets a model loading task queue and expands the device for loading the target model when a task backlog occurs in the queue, thereby achieving dynamic expansion based on user usage of the model, making full use of limited hardware resources, and improving the operating efficiency of the model.

[0100] In some optional implementations, such as Figure 4 As shown, step 103 includes:

[0101] Step 10312, if the number of devices loading the target model does not reach the preset number of devices, the type of the target model loading task of the target model is set to a temporary loading task type, and the state of the target model loading task is set to a waiting state, and the target model is started to be loaded to the first target device.

[0102] Specifically, when the number of devices loading the target model does not reach the preset number of devices, the type of the target model loading task can be set to tmp and the state can be set to waiting. Figure 5 As shown, it shows a schematic diagram of the state flow process when expanding the capacity of the device loaded with the target model. Figure 5 When step 501 in the example judges that the number of devices for loading the target model does not reach the preset number of devices, the type and state of the first target device and the type and state of the target model loading task are shown in step 502. When it is judged that the number of devices for loading the target model reaches the preset number of devices, the number of devices for loading the target model cannot be expanded.

[0103] Step 10313, adjusting the state of the target model loading task to the model loading state, and adjusting the state of the first target device from the running state to the model loading state.

[0104] like Figure 5 As shown in step 503, after the target model starts to be loaded, the state of the first target device is adjusted to switching, and the state of the target model loading task is adjusted to switching.

[0105] Step 10314, in response to the target model loading being successful, adjusting the type of the first target device to a temporary device type, adjusting the state of the first target device to a running state, and adjusting the state of the target model loading task to a task loading completion state.

[0106] like Figure 5 As shown in step 504 and step 505 in , after the target model is loaded successfully, the device type is adjusted to tmp, the device state is adjusted to running, the task type is adjusted to tmp, and the task state is adjusted to finished.

[0107] Step 10315, in response to the target model loading failure, adjusting the state of the first target device to the running state, and adjusting the state of the target model loading task to the task loading failure state.

[0108] like Figure 5 As shown in step 506 in , after the target model fails to load, the device type remains free, the device state is adjusted to running, the task type remains tmp, and the task state is adjusted to finished.

[0109] This embodiment realizes real-time recording of the current execution status of the first target device and the target model loading task by adjusting the device type and status, as well as the task type and status when the number of devices for loading the target model is expanded, which helps to effectively control the expansion process, and can enable users to know the execution status of the model loading task, thereby improving the efficiency of model loading.

[0110] In some optional implementations, for example, Figure 4 As shown, after step 10314, the method further includes:

[0111] Step 10316: If the number of model loading tasks contained in the model loading task queue does not reach the preset number of tasks, the type of the first target device is adjusted to an idle type.

[0112] That is, Figure 5 As shown in steps 507 and 508 in the example, when it is determined that the model loading task queue no longer has a backlog of tasks, the type of the first target device is adjusted from tmp to free, and the device state remains running, so that other model loading tasks can select the device to load the model when executing. When it is determined that the model loading task queue still has a backlog of tasks, wait for the current task to be completed.

[0113] This embodiment restores the type of the first target device to an idle type when there is no more task backlog in the model loading task queue, so that the expanded device can be recycled in time, which helps to save hardware resources and improve the efficiency of model processing.

[0114] In some optional implementations, such as Figure 6 As shown, after step 103, the method further includes:

[0115] Step 107: Generate first identification information for associating the first target device and the target model, and set the validity period of the identification information to a preset duration.

[0116] The identification information is a token, which is used to indicate the association between the first target device and the target model. Usually, the identification information can be a string, which is converted into the identification information by using a preset algorithm by converting the call request id, service provider id, model id and other information. Under the on-demand loading strategy, the identification information has a certain validity period, during which the target model loaded by the first target device can be used by the user.

[0117] Step 108: Send the identification information to the sender of the call request.

[0118] Specifically, after receiving the identification signal, the sender of the call request can return the identification information to the electronic device (such as the back-end server) that executes the method when calling the model for data processing. The electronic device can determine the first target device based on the identification information, and control the first target device to call the loaded target model to process the data input by the user.

[0119] This embodiment sets the validity period of the identification information under the on-demand loading strategy, thereby achieving that within the validity period, the identification information can be used to determine the first target device loaded with the target model for data processing, thereby helping to manage the model loading process of the device and improving more efficient allocation of device resources in businesses using multiple models.

[0120] In some optional implementations, such as Figure 6 As shown, after step 107, the method further includes:

[0121] Step 109: If the first identification information associated with the first target device has all expired, the type of the first target device is adjusted from a loaded type to an idle type.

[0122] like Figure 3 As shown, in step 306 and step 307, when it is determined that all tokens associated with the first target device have expired, the device type is adjusted to free, the device state is maintained as running, and the task type and task state remain unchanged.

[0123] This embodiment timely recycles the resources of the first target device by promptly adjusting the type of the first target device to an idle type when the first identification information associated with the first target device expires, thereby avoiding a certain model occupying the device for a long time when in use, and improving the utilization efficiency of hardware resources.

[0124] In some optional implementations, before step 101, the method further includes:

[0125] First, according to the preloading strategy, a preset number of devices on which the target model can be loaded are determined from the device cluster, and the target model is loaded into the preset number of second target devices.

[0126] Specifically, the above preset number can be set after the corresponding business scenario is determined. That is, in a single model business, it is set in advance by the business provider. For example, the business provider pre-evaluates the number of devices that need to set the target model based on the usage rate of the target model (such as a model with functions such as adding special effects to images and changing image styles). Then set a separate model loading task to load the target model to a preset number of devices.

[0127] Then, second identification information for associating a preset number of devices with the target model is generated, and the validity period of the identification information is set to be permanent.

[0128] Among them, the second identification information has the same meaning as the above-mentioned first identification information. The difference is that the validity period of the second identification information is set to be permanent, that is, under the preloading strategy, the target models loaded by a preset number of devices are permanently valid, and users will not encounter real-time model loading operations when using the model for data processing, thereby reducing the response time of the model.

[0129] This embodiment sets the validity period of the second identification information to be permanently valid when pre-loading the target model to a preset number of devices, thereby achieving the goal of pre-loading the model to multiple fixed devices under a single model business, and users can use the models on these devices to perform data processing at any time.

[0130] refer to Figure 7 , combined with the above embodiments, it shows a flow chart of determining a target device and loading a model to the target device under different loading strategies. Figure 7 As shown, strategy 1, i.e., on-demand loading strategy, can be executed when a user issues a model call request in real time. Strategy 2, i.e., preload measurement, is executed when the target model is preloaded to n devices.

[0131] In strategy one, the electronic device receives a single loading task request and first determines whether there is a device in the device cluster that has loaded the target model. If so, a token is generated and associated with any device that has loaded the target model. The device is used as a device to process the pending data sent by the user, and the user is notified at the same time to send the pending data to the first target device. If there is no device that has loaded the target model, determine whether there is an available device under the corresponding service. If so, the user is synchronously notified that the model loading task has been received and the model loading begins. Then, when the model is loaded successfully, a token is generated and associated with the device. The device is used as a device to process the pending data sent by the user, and the user is notified at the same time to send the pending data to the first target device. If the model fails to load, the user is notified of the loading failure.

[0132] In strategy 2, first determine whether there is a device in the device cluster that has loaded the target model. If so, reject the current loading request. If not, the electronic device receives n loading task requests, and then determines whether there are n available devices under the corresponding business. If so, synchronously notify the user that the model loading task has been received and start loading the model. Then, when all devices load the model successfully, a token is generated to associate all n devices, and these devices are used as devices to process the pending data sent by the user. If any model fails to load, the entire loading task fails.

[0133] Figure 8A schematic diagram of the structure of a dual-strategy model calling device provided in an embodiment of the present application. Specifically comprising: a receiving module 801, for receiving a call request for a target model; a first determining module 802, for determining the loading strategy corresponding to the target model according to the call request; a second determining module 803, for determining the first target device that can load the target model from a preset device cluster if the loading strategy is an on-demand loading strategy, and loading the target model into the first target device; a third determining module 804, for determining a preset number of devices pre-loaded with the target model from the device cluster if the loading strategy is a pre-loading strategy; a fourth determining module 805, for determining the second target device from a preset number of devices; a processing module 806, for processing the data to be processed using the target model on the first target device or the second target device in response to the data to be processed sent by the sending end that receives the call request.

[0134] In some optional implementations, the second determination module includes: a first determination unit, used to determine whether there is a device with the target model loaded in a preset device cluster; a second determination unit, used to determine, if not, a device of an idle type and in a running state from the device cluster as the first target device; and a third determination unit, used to determine, if exists, the first target device from devices of a loaded type and loaded with the target model.

[0135] In some optional implementations, the second determination module includes: a first setting unit, which is used to set the type of the target model loading task of the target model to a normal loading task type, and set the state of the target model loading task to a waiting state, if the type of the first target device is an idle type, and start loading the target model to the first target device; a first adjustment unit, which is used to adjust the state of the target model loading task to a model loading state, and adjust the state of the first target device from a running state to a model loading state; a second adjustment unit, which is used to adjust the type of the first target device to a loaded type, adjust the state of the first target device to a running state, and adjust the state of the target model loading task to a task loading completion state in response to successful loading of the target model.

[0136] In some optional implementations, the device further includes: a first adjustment module, configured to adjust the loading task type to a task loading failure state in response to a target model loading failure, and send a loading failure prompt message to a sender of the call request.

[0137] In some optional implementations, the second determination module includes: a fourth determination unit, used to determine whether the number of model loading tasks contained in the model loading task queue corresponding to the target model reaches a preset task number; a fifth determination unit, used to determine whether the number of devices for loading the target model reaches a preset device number if the preset task number is reached; a sixth determination unit, used to determine, from the device cluster, a device whose device type is an idle type and whose device status is a running status as the first target device if the preset device number is not reached; and a seventh determination unit, used to determine, from the devices that have loaded the target model, the first target device if the preset device number is reached.

[0138] In some optional implementations, the second determination module includes: a second setting unit, which is used to set the type of the target model loading task of the target model to a temporary loading task type, and set the state of the target model loading task to a waiting state, and start loading the target model to the first target device if the number of devices loading the target model does not reach a preset number of devices; a third adjustment unit, which is used to adjust the state of the target model loading task to a model loading state, and adjust the state of the first target device from a running state to a model loading state; a fourth adjustment unit, which is used to adjust the type of the first target device to a temporary device type, adjust the state of the first target device to a running state, and adjust the state of the target model loading task to a task loading completion state in response to successful target model loading; a fifth adjustment unit, which is used to adjust the state of the first target device to a running state, and adjust the state of the target model loading task to a task loading failure state in response to a target model loading failure.

[0139] In some optional implementations, the second determination module further includes: a sixth adjustment unit, configured to adjust the type of the first target device to an idle type if the number of model loading tasks included in the model loading task queue does not reach a preset number of tasks.

[0140] In some optional implementations, the device also includes: a first generating module, used to generate first identification information for associating the first target device and the target model, and set the validity period of the identification information to a preset duration; a sending module, used to send the identification information to the sending end of the call request.

[0141] In some optional implementations, the apparatus further includes: a second adjustment module, configured to adjust the type of the first target device from a loaded type to an idle type if all first identification information associated with the first target device has expired.

[0142] In some optional implementations, the device also includes: a fourth determination module, used to determine, from the device cluster, a preset number of devices on which the target model can be loaded according to a preloading strategy, and load the target model into a preset number of second target devices; a second generation module, used to generate second identification information for associating a preset number of devices and the target model, and setting the validity period of the identification information to permanent.

[0143] The dual strategy model calling device provided in this embodiment can be as follows Figure 8 The dual-strategy model calling device shown in can execute all the steps of the above dual-strategy model calling methods, thereby achieving the technical effects of the above dual-strategy model calling methods. Please refer to the above related description for details. For the sake of brevity, it will not be repeated here.

[0144] Fig. 9 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application, Fig. 9 The electronic device 900 shown includes: at least one processor 901, a memory 902, at least one network interface 904 and other user interfaces 903. The various components in the electronic device 900 are coupled together via a bus system 905. It is understood that the bus system 905 is used to achieve connection and communication between these components. In addition to the data bus, the bus system 905 also includes a power bus, a control bus and a status signal bus. However, for the sake of clarity, the bus system 905 is not described in detail. Fig. 9 Various buses are labeled as bus system 905.

[0145] The user interface 903 may include a display, a keyboard, or a pointing device (eg, a mouse, a trackball, a touch pad, or a touch screen).

[0146] It can be understood that the memory 902 in the embodiment of the present application can be a volatile memory or a non-volatile memory, or can include both volatile and non-volatile memories. Among them, the non-volatile memory can be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory can be a random access memory (RAM), which is used as an external cache. By way of example but not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM) and direct RAM bus random access memory (DRRAM). The memory 902 described herein is intended to include, but is not limited to, these and any other suitable types of memory.

[0147] In some embodiments, the memory 902 stores the following elements, executable units or data structures, or a subset thereof, or an extended set thereof: an operating system 9021 and an application program 9022 .

[0148] The operating system 9021 includes various system programs, such as a framework layer, a core library layer, a driver layer, etc., for implementing various basic services and processing hardware-based tasks. The application 9022 includes various application programs, such as a media player (Media Player), a browser (Browser), etc., for implementing various application services. The program for implementing the method of the embodiment of the present application can be included in the application 9022.

[0149] In this embodiment, by calling the program or instruction stored in the memory 902, specifically, the program or instruction stored in the application 9022, the processor 901 is used to execute the method steps provided by each method embodiment, for example, including:

[0150] Receive a call request for a target model; determine a loading strategy corresponding to the target model based on the call request; if the loading strategy is an on-demand loading strategy, determine a first target device on which the target model can be loaded from a preset device cluster, and load the target model into the first target device; if the loading strategy is a preloading strategy, determine a preset number of devices on which the target model is preloaded from the device cluster; determine a second target device from the preset number of devices; in response to the data to be processed sent by the sending end that receives the call request, process the data to be processed using the target model on the first target device or the second target device.

[0151] The method disclosed in the above embodiment of the present application can be applied to the processor 901, or implemented by the processor 901. The processor 901 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the hardware integrated logic circuit or software instructions in the processor 901. The above processor 901 can be a general processor, a digital signal processor (Digital Signal Processor, DSP), an application specific integrated circuit (Application Specific Integrated Circuit, ASIC), a field programmable gate array (Field Programmable Gate Array, FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The methods, steps and logic block diagrams disclosed in the embodiments of the present application can be implemented or executed. The general processor can be a microprocessor or the processor can also be any conventional processor, etc. The steps of the method disclosed in the embodiment of the present application can be directly embodied as a hardware decoding processor to execute, or the hardware and software units in the decoding processor can be executed. The software unit can be located in a mature storage medium in the field such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory 902, and the processor 901 reads the information in the memory 902 and completes the steps of the above method in combination with its hardware.

[0152] It is understood that the embodiments described herein may be implemented in hardware, software, firmware, middleware, microcode, or a combination thereof. For hardware implementation, the processing unit may be implemented in one or more application specific integrated circuits (ASIC), digital signal processors (DSP), digital signal processing devices (DSPDevice, DSPD), programmable logic devices (PLD), field programmable gate arrays (FPGA), general purpose processors, controllers, microcontrollers, microprocessors, other electronic units for performing the above functions of the present application, or a combination thereof.

[0153] For software implementation, the technology described above in this article can be implemented by a unit that performs the functions described above in this article. The software code can be stored in a memory and executed by a processor. The memory can be implemented in the processor or outside the processor.

[0154] The electronic device provided in this embodiment may be Fig. 9 The electronic device shown in can execute all the steps of the dual-strategy model calling methods described above, thereby achieving the technical effects of the dual-strategy model calling methods described above. Please refer to the above related description for details. For the sake of brevity, it will not be repeated here.

[0155] The embodiment of the present application also provides a storage medium (computer-readable storage medium). The storage medium here stores one or more programs. The storage medium may include a volatile memory, such as a random access memory; the memory may also include a non-volatile memory, such as a read-only memory, a flash memory, a hard disk or a solid-state drive; the memory may also include a combination of the above-mentioned types of memory.

[0156] When one or more programs in the storage medium can be executed by one or more processors, the dual-strategy model calling method executed on the electronic device side can be implemented.

[0157] The processor is used to execute the program stored in the memory to implement the following steps of the dual-strategy model calling method executed on the electronic device side:

[0158] Receive a call request for a target model; determine a loading strategy corresponding to the target model based on the call request; if the loading strategy is an on-demand loading strategy, determine a first target device on which the target model can be loaded from a preset device cluster, and load the target model into the first target device; if the loading strategy is a preloading strategy, determine a preset number of devices on which the target model is preloaded from the device cluster; determine a second target device from the preset number of devices; in response to the data to be processed sent by the sending end that receives the call request, process the data to be processed using the target model on the first target device or the second target device.

[0159] The professionals should further realize that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented by electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in terms of function in the above description. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different circuits to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0160] The steps of the circuits or algorithms described in conjunction with the embodiments disclosed herein may be implemented using hardware, software modules executed by a processor, or a combination of the two. The software modules may be placed in a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.

[0161] It should be understood that the terms used in the text are only for the purpose of describing specific example embodiments, and are not intended to be limiting. Unless the context clearly indicates otherwise, the singular forms "one", "an" and "said" as used in the text may also be meant to include plural forms. The terms "include", "comprise", "contain", and "have" are inclusive, and therefore specify the existence of stated features, steps, operations, elements and / or parts, but do not exclude the existence or addition of one or more other features, steps, operations, elements, parts, and / or combinations thereof. The steps, processes, and operations described in the text are not interpreted as necessarily requiring them to be performed in the specific order described or illustrated, unless the execution order is clearly indicated. It should also be understood that additional or alternative steps may be used.

[0162] The above description is only a specific implementation of the present application, so that those skilled in the art can understand or implement the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will conform to the widest range consistent with the principles and novel features applied for herein.

Claims

1. A dual-strategy model calling method, characterized in that: The method comprises: Receive a call request to the target model; Determining a loading strategy corresponding to the target model according to the calling request; If the loading strategy is an on-demand loading strategy, determining a first target device that can load the target model from a preset device cluster, and loading the target model into the first target device; If the loading strategy is a preloading strategy, determining a preset number of devices preloaded with the target model from the device cluster; Determine a second target device from the preset number of devices; In response to receiving the data to be processed sent by the sending end of the call request, the data to be processed is processed using the target model on the first target device or the second target device.

2. The method according to claim 1, characterized in that The step of determining, from a preset device cluster, a first target device on which the target model can be loaded includes: From a preset device cluster, determine whether there is a device that has loaded the target model; If not, determine a device of idle type and in running state from the device cluster as the first target device; If so, the first target device is determined from devices of the loaded type and loaded with the target model.

3. The method according to claim 2, characterized in that The step of loading the target model into the first target device comprises: If the type of the first target device is an idle type, setting the type of the target model loading task of the target model to a common loading task type, and setting the state of the target model loading task to a waiting state, and starting to load the target model to the first target device; Adjusting the state of the target model loading task to a model loading state, and adjusting the state of the first target device from the running state to a model loading state; In response to the target model being loaded successfully, the type of the first target device is adjusted to a loaded type, the state of the first target device is adjusted to a running state, and the state of the target model loading task is adjusted to a task loading completion state.

4. The method according to claim 3, characterized in that After starting to load the target model to the first target device, the method further includes: In response to the target model loading failure, the loading task type is adjusted to a task loading failure state, and loading failure prompt information is sent to the sending end of the call request.

5. The method according to claim 1, characterized in that The step of determining, from a preset device cluster, a first target device on which the target model can be loaded includes: Determine whether the number of model loading tasks contained in the model loading task queue corresponding to the target model reaches a preset number of tasks; If the preset number of tasks is reached, determining whether the number of devices loaded with the target model reaches the preset number of devices; If the preset number of devices is not reached, determining, from the device cluster, a device whose device type is an idle type and whose device status is a running status as the first target device; If the preset number of devices is reached, the first target device is determined from the devices that have loaded the target model.

6. The method according to claim 5, characterized in that The step of loading the target model into the first target device comprises: If the number of devices loading the target model does not reach the preset number of devices, the type of the target model loading task of the target model is set to a temporary loading task type, and the state of the target model loading task is set to a waiting state, and the target model is started to be loaded to the first target device; Adjusting the state of the target model loading task to a model loading state, and adjusting the state of the first target device from the running state to a model loading state; In response to the target model being loaded successfully, adjusting the type of the first target device to a temporary device type, adjusting the state of the first target device to a running state, and adjusting the state of the target model loading task to a task loading completion state; In response to the target model loading failure, the state of the first target device is adjusted to a running state, and the state of the target model loading task is adjusted to a task loading failure state.

7. The method according to claim 6, characterized in that After, in response to the target model loading being successful, adjusting the type of the first target device to a temporary device type, adjusting the state of the first target device to a running state, and adjusting the state of the target model loading task to a task loading completion state, the method further includes: If the number of model loading tasks included in the model loading task queue does not reach a preset number of tasks, the type of the first target device is adjusted to an idle type.

8. The method according to claim 1, characterized in that After loading the target model into the first target device, the method further includes: Generate first identification information for associating the first target device with the target model, and set the validity period of the identification information to a preset duration; The identification information is sent to the sending end of the call request.

9. The method according to claim 8, characterized in that After setting the validity period of the identification information to a preset duration, the method further includes: If the first identification information associated with the first target device has all expired, the type of the first target device is adjusted from a loaded type to an idle type.

10. The method according to claim 1, characterized in that Before receiving the call request for the target model, the method further includes: According to the preloading strategy, determining a preset number of devices on which the target model can be loaded from the device cluster, and loading the target model into the preset number of second target devices; Generate second identification information for associating the preset number of devices with the target model, and set the validity period of the identification information to be permanently valid.

11. A dual-strategy model calling device, characterized in that: The device comprises: A receiving module, used for receiving a call request to a target model; A first determination module, used to determine a loading strategy corresponding to the target model according to the call request; A second determination module is used to determine, if the loading strategy is an on-demand loading strategy, a first target device that can load the target model from a preset device cluster, and load the target model into the first target device; A third determining module is used to determine a preset number of devices preloaded with the target model from the device cluster if the loading strategy is a preloading strategy; A fourth determining module, configured to determine a second target device from the preset number of devices; The processing module is used to process the data to be processed by using the target model on the first target device or the second target device in response to receiving the data to be processed sent by the sending end of the call request.

12. An electronic device, characterized in that: include: Memory for storing computer programs; A processor is used to execute the computer program stored in the memory, and when the computer program is executed, the dual-strategy model calling method described in any one of claims 1 to 10 is implemented.

Images