Model operation method and device and electronic equipment

Abstract

The invention discloses a model operation method and device and electronic equipment, and the method comprises the steps: detecting the starting of an application program, and determining a model object which can be called by the application program; determining a first model data segment from the model data of the model object stored in the first storage area, wherein the first model data segment is data determined according to the operation sequence of the model data of the model object; loading the first model data segment to a second storage area, wherein the data processing speed of the second storage area is higher than that of the first storage area; in response to a first request of needing to call the model object, running the first model data segment loaded to the second storage area, and loading the second model data segment of the model object stored in the first storage area to the third storage area, the data processing speed of the third storage area being higher than the data processing speed of the first storage area, the second model data segment is data of which the operation sequence is after the operation sequence of the first model data segment.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a model operation method, device and electronic device. Background Art

[0002] With the continuous development of artificial intelligence technology, the scale of model objects such as machine learning models running in electronic devices is getting larger and larger. For example, the data volume of large language models may be around 5G.

[0003] Since model objects occupy a large amount of system memory, in order to reduce the impact of model objects occupying a large amount of memory on other applications, if the model object has not been used for a certain period of time, the data of the model object loaded into the memory will be cleared. However, each time the model object is used, the model object needs to be reloaded, and the amount of data in the model object is large. Reloading the model object takes a long time, which leads to a large delay in executing task processing using the model object. Summary of the invention

[0004] On the one hand, the present application provides a model operation method, comprising:

[0005] Detecting the launch of an application, and determining a model object that can be called by the application;

[0006] Determine a first model data segment from the model data of the model object stored in the first storage area, wherein the first model data segment is data determined according to a running order of the model data of the model object;

[0007] Loading the first model data segment into a second storage area, wherein a data processing speed of the second storage area is higher than a data processing speed of the first storage area;

[0008] In response to a first request to call the model object, the first model data segment loaded into the second storage area is run, and the second model data segment of the model object stored in the first storage area is loaded into the third storage area, the first request is triggered by user input information, the data processing speed of the third storage area is higher than the data processing speed of the first storage area, and the second model data segment is data whose running order is after the running order of the first model data segment.

[0009] In a possible implementation, the model running method further includes:

[0010] After the execution of the first model data segment is completed, the data in the second model data segment that has been loaded into the third storage area is executed in sequence.

[0011] In another possible implementation, the model running method further includes:

[0012] After completing the processing of the first request based on the model object, if no request to call the model object is detected within a set time period, at least part of the data in the second model data segment loaded in the third storage area is released, and the first model data segment loaded in the second storage area is maintained unchanged.

[0013] In yet another possible implementation, the releasing at least part of the data in the second model data segment loaded in the third storage area includes:

[0014] Based on the current available resource status of the electronic device, it is determined that at least part of the data in the second model data segment that has been loaded into the third storage area needs to be released, and the at least part of the data in the third storage area is released.

[0015] In yet another possible implementation, determining the first model data segment from the model data of the model object stored in the first storage area includes:

[0016] If the application can call multiple model objects, based on the current available resource status of the electronic device, determine at least one model object that needs to be preloaded among the multiple model objects;

[0017] For each model object that needs to be preloaded, determining a first model data segment from the model data of the model object stored in the first storage area;

[0018] The data volumes of the first model data segments of different model objects are different, and the space size of the second storage area corresponding to each model object is positively correlated with the data volume of the first model data segment of the model object.

[0019] In yet another possible implementation, the determining, based on the current available resource status of the electronic device, at least one model object that needs to be preloaded among the multiple model objects includes:

[0020] If the currently available resources of the electronic device are sufficient, all of the multiple model objects are determined as model objects that need to be preloaded;

[0021] If the currently available resources of the electronic device are insufficient, at least one model object that needs to be preloaded is determined according to at least one of the priorities of the multiple model objects and historical user usage data.

[0022] In yet another possible implementation, in response to a first request for calling the model object, executing the first model data segment loaded into the second storage area, and loading the second model data segment of the model object stored in the first storage area into the third storage area, includes:

[0023] In response to a first request for calling a first model object, running a first model data segment corresponding to the first model object and loaded into the second storage area, and loading a second model data segment of the first model object stored in the first storage area into a third storage area;

[0024] The model operation method further comprises:

[0025] Obtaining a second request for calling a second model object, and executing a first model data segment loaded into a second storage area corresponding to the second model object;

[0026] If the storage space of the target storage unit is insufficient, releasing at least part of the data in the second model data segment corresponding to the first model object cached in the third storage area, and loading the second model data segment of the second model object stored in the first storage area into the fourth storage area;

[0027] Wherein, the third storage area and the fourth storage area both belong to the target storage unit, and the data processing speed of the target storage unit is higher than the data processing speed of the first storage area;

[0028] The first model object and the second model object are both model objects that need to be preloaded among the multiple model objects.

[0029] In yet another possible implementation, loading the second model data segment of the model object stored in the first storage area into a third storage area includes:

[0030] Determine sub-model data in the model data of the model object that is required to process the first request;

[0031] The second model data segment stored in the first storage area is loaded into the third storage area, the second model data segment belonging to the sub-model data.

[0032] In another aspect, the present application also provides a model operation device, comprising:

[0033] An object detection unit detects the startup of an application and determines a model object that can be called by the application;

[0034] a data determining unit, configured to determine a first model data segment from the model data of the model object stored in the first storage area, wherein the first model data segment is data determined according to a running order of the model data of the model object;

[0035] a preloading unit, configured to load the first model data segment into a second storage area, wherein a data processing speed of the second storage area is higher than a data processing speed of the first storage area;

[0036] A model running unit is used to run the first model data segment loaded into the second storage area in response to a first request to call the model object, and load the second model data segment of the model object stored in the first storage area into the third storage area, the first request is triggered by user input information, the data processing speed of the third storage area is higher than the data processing speed of the first storage area, and the second model data segment is data whose running order is after the running order of the first model data segment.

[0037] In another aspect, the present application further provides an electronic device, comprising: at least one processor and at least one processing model capable of running on the processor, wherein the processing model can be called by an application program to perform the following operations:

[0038] Detecting the launch of an application, and determining a model object that can be called by the application;

[0039] Determine a first model data segment from the model data of the model object stored in the first storage area, wherein the first model data segment is data determined according to a running order of the model data of the model object;

[0040] Loading the first model data segment into a second storage area, wherein a data processing speed of the second storage area is higher than a data processing speed of the first storage area;

[0041] In response to a first request to call the model object, the first model data segment loaded into the second storage area is run, and the second model data segment of the model object stored in the first storage area is loaded into the third storage area, the first request is triggered by user input information, the data processing speed of the third storage area is higher than the data processing speed of the first storage area, and the second model data segment is data whose running order is after the running order of the first model data segment. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] The above and other features, advantages and aspects of the embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. Throughout the accompanying drawings, the same or similar reference numerals represent the same or similar elements. It should be understood that the drawings are schematic and the originals and elements are not necessarily drawn to scale.

[0043] Figure 1 A schematic diagram of a flow chart of the model operation method provided in this application;

[0044] Figure 2An example diagram showing a mapping relationship between a disk, virtual memory, and physical memory in this application is shown;

[0045] Figure 3 A schematic diagram of another flow chart of the model operation method provided in this application;

[0046] Figure 4 A schematic diagram of another flow chart of the model operation method provided in this application;

[0047] Figure 5 A schematic diagram of a structure of a model operation device provided in an embodiment of the present application;

[0048] Figure 6 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0049] The embodiments of the present application are described below in conjunction with the drawings in the embodiments of the present application. The terms used in the implementation mode of the present application are only used to explain the specific embodiments of the present application, and are not intended to limit the present application. It is known to those skilled in the art that with the development of technology and the emergence of new scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

[0050] The terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and need not be used to describe a specific order or sequential order. It should be understood that the terms used in this way can be interchangeable under appropriate circumstances, which is only to describe the distinction mode adopted by the objects of the same attributes when describing in the embodiments of the present application. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, so that the process, method, system, product or equipment comprising a series of units need not be limited to those units, but may include other units that are not clearly listed or inherent to these processes, methods, products or equipment.

[0051] like Figure 1 , shows a flow chart of the model operation method provided by the embodiment of the present application, and the method of the present embodiment can be applied to electronic devices, which can be terminal devices such as mobile phones, tablet computers or laptop computers. Of course, it can also be a server, or a device node in a system such as a cloud platform, without limitation.

[0052] The method of this embodiment may include:

[0053] S101, detecting the startup of an application, and determining a model object that can be called by the application.

[0054] The model object that can be called by the application can be a model object pre-configured in the application, or a model object associated with the application in other ways, and there is no restriction on this.

[0055] The model object refers to an artificial intelligence model built using artificial intelligence technology. For example, the model object may be a deep learning model or a machine learning model. Optionally, in order to improve the accuracy of data processing by the application using the model object, the model object may be a large language model (LIM).

[0056] It is understandable that in actual applications, depending on the application, the functions implemented by the model object that the application can call will also be different, and accordingly, the model type of the model object that the application can call will also be different, such as a model object for implementing intelligent question and answer processing, or a generative model for implementing text and graphics (or videos), etc. Of course, the model object can also be a model that can implement a variety of different functions, such as a large language model, etc., and there is no restriction on this.

[0057] S102: Determine a first model data segment from the model data of the model object stored in the first storage area.

[0058] The first storage area is a storage area for fixedly storing the model data of the model object. The first storage area is a storage area in a non-volatile memory, and the application cannot directly run the model data of the model object in the first storage area. For example, the first storage area can be a storage area in a disk.

[0059] The first model data segment is data determined according to the running order of the model data of the model object. In order to start the running of the model object more quickly when calling the model object, the first model data segment is the part of the model data of the model object with the most advanced running order.

[0060] For example, a model object can be composed of one or more components that need to be run in sequence, and each component module is used to implement different data processing functions. Taking the model object as a neural network model as an example, each layer of the neural network model is a component module of the neural network model, and each layer is used to implement different data processing functions. On this basis, the present application can determine that the data of one or more components with the highest running order is the first model data segment.

[0061] It should be noted that after each application is started, the remaining resources such as memory of the electronic device are different, and the performance such as inference speed of different models is also different. Therefore, in order to reasonably realize the preloading of model data, the first model data segment is not a fixed part of the model data of the model object. Each time the application startup is detected, the amount of data in the first model data segment determined may be different.

[0062] For example, each time an application is detected to be started, the amount of model data that needs to be preloaded can be comprehensively determined based on the current available resources of the electronic device and the inference performance of the model object, and the first model data segment can be determined based on the amount of model data and the running order of each component model in the model data of the model object.

[0063] For example, taking a model object with 100 layers, if the memory resources of the electronic device are sufficient and the reasoning speed of the model object is fast when the application is started, then the data of the first 30 layers in the model object can be determined as the first model data segment; if the memory resources of the electronic device are insufficient or the reasoning performance of the model object itself determines that its reasoning speed is slow when the application is started, then only the data of the first 10 layers of the model object can be determined as the first model data segment.

[0064] S103: Load the first model data segment into the second storage area.

[0065] The data processing speed of the second storage area is higher than the data processing speed of the first storage area.

[0066] In the present application, the second storage area belongs to a storage area in a storage unit in an electronic device used to store data required for processor operation. Therefore, after the first model data segment is loaded into the second storage area, the processor can run the first model data segment loaded in the second storage area.

[0067] The storage unit to which the second storage area belongs will be different if the processor used to run the model object is different. Usually, the processor that runs the model object is the same as the processor that runs the application program.

[0068] For example, the application can run on at least one or a combination of processors such as a central processing unit (CPU), a neural network processing unit (NPU), and a graphics processing unit (GPU). Correspondingly, if the application runs on the CPU, then the model object that the application needs to call also needs to run on the CPU, and the second storage area can be a storage area in the memory corresponding to the CPU.

[0069] Based on this, if the application is run on the GPU, the second storage area can be a storage area in the video memory corresponding to the GPU; if the application is run on the NPU, the second storage area can be a storage area in the memory unit corresponding to the NPU. Of course, if multiple processors are used to process the application, the second storage area can be a storage area in the memory space formed by multiple memory units.

[0070] In the present application, the second storage area and the first storage area may belong to storage areas in different storage units. For example, the first storage area may belong to a storage area in a disk, and the second storage area may be a storage area in the physical memory corresponding to each processor mentioned above.

[0071] Of course, with the development of technology, the second storage area and the first storage area may also belong to the same storage unit, but the first storage area is non-volatile, and the data stored in the first storage area cannot be directly executed by the processor; while the data stored in the second storage area can be directly executed by the processor.

[0072] It can be understood that after the first model data segment is determined, any data loading method can be used to load the first model data segment into the second storage area, and there is no limitation on this.

[0073] For example, after determining the first model data segment, a request can be made to the operating system to allocate a second storage area matching the first model data segment, and the operating system will create a mapping relationship between the sub-storage area where the first model data segment is located, the virtual memory, and the physical second storage area in the first storage area. On this basis, the first model data segment can be loaded into the second storage area based on the mapping relationship.

[0074] The virtual memory is only a virtual continuous address space, and its function is only to map to the physical memory. For ease of understanding, the first storage area is taken as a disk as an example, and combined with Figure 2 illustrate. Figure 2It shows the storage area on the disk of the model data that the application process (the process corresponding to the application program) needs to access, and the mapping relationship between the virtual memory and the physical memory.

[0075] Depend on Figure 2 It can be seen that the disk stores the model data of the model objects that can be called by the application process A and the application process B. For each application process, it is necessary to determine the virtual memory space to which the disk area needs to be mapped based on the disk area corresponding to the model data of the model object that can be called by the application process, and then determine the physical memory space to which the virtual memory space is mapped.

[0076] S104, in response to a first request for calling a model object, executing the first model data segment loaded into the second storage area, and loading the second model data segment of the model object stored in the first storage area into the third storage area.

[0077] The first request is triggered by user input information.

[0078] For example, in response to the information to be processed input by the user or the first instruction carrying the information to be processed, if the application confirms that the model object needs to be called, after performing information retrieval or other processing based on the information to be processed or the first instruction, a first request to call the model object is generated.

[0079] In another example, a first request input by a user to use a model object is obtained, and it is confirmed that a first request to call the model object is obtained. The first request may include data information that needs to be processed by the model object.

[0080] The data processing speed of the third storage area is higher than the data processing speed of the first storage area.

[0081] In the present application, the third storage area also belongs to a storage area in a storage unit in an electronic device for storing data required for the operation of a processor. Therefore, after the second model data segment of the model object is loaded into the third storage area, the processor can operate the second model data segment stored in the third storage area.

[0082] The processors used to run the model objects are different, and the storage units to which the third storage area belongs will also be different. The details can be similar to the second storage area, and no further description is given.

[0083] In the present application, the second storage area and the third storage area may belong to different storage areas in the same storage unit. For example, if the application runs on a CPU, then the second storage area and the third storage area both belong to storage areas in the physical memory corresponding to the CPU.

[0084] Of course, the second storage area and the third storage area may also belong to storage areas in different storage units. For example, if a plurality of processors are combined to run the application program, the second storage area and the third memory may also be located in the memory corresponding to different processors.

[0085] The second model data segment is data whose running sequence is after the running sequence of the first model data segment.

[0086] The process of loading the second model data segment into the third storage area may be similar to the process of loading the first model data segment into the second storage area, and details thereof will not be repeated herein.

[0087] From the above content, it can be seen that when the application is started, the first model data segment with a higher running order in the model data of the model object will be taken out from the first storage area and loaded into the second storage area. Based on this, after obtaining the first request to call the model object, the first model data segment loaded into the second storage area can be directly run, so that the operation of the model object can be started without waiting for the model data to be loaded separately. Moreover, in the process of the application running the first model data segment, the remaining model data segments will be loaded synchronously, realizing the loading and running of the model object, thereby ensuring the reliable operation of the model object, and naturally reducing the delay in the operation of the model object due to waiting for the model data to be loaded, thereby improving the efficiency of processing requests using the model object, and reducing the waiting time required for the user to wait for the model to load and obtain the data processing results of the model object.

[0088] It is understandable that the amount of data in the second model data segment is also relatively large. Therefore, after the application completes the operation of the first model data segment, it is possible that the second model data segment is not fully loaded into the third storage area. On this basis, in order to reduce the need to wait for model data to be loaded during the model object process, the present application can also sequentially run the data in the second model data segment that has been loaded into the third storage area after completing the operation of the first model data segment.

[0089] For example, the second model data segment may also include data of multiple component modules of the model object. Loading the second model data segment into the third storage area is actually loading each sub-data segment in sequence according to the running order of the second model data segment, and each sub-data segment may include data of at least one component module of the model object. Therefore, after the running of the first model data segment is completed, the data of each component model belonging to the second model data segment and loaded into the third storage area may be run in sequence according to the running order of the model data in the model data.

[0090] It is understandable that by running the model data of the model object, the first request can be processed by the model object. For example, the first request can be to call the model object to process an input question input by the user, and then the answer corresponding to the input question can be determined by the model object. For another example, the first request is to call the model object to generate an object such as an image or video corresponding to the prompt information input by the user, and then the corresponding object is generated by running the model object.

[0091] Based on this, in any of the above embodiments of the present application, after completing the processing of the first request based on the model object, if no request to call the model object is detected within the set time length, in order to reduce resource usage, the present application can also release at least part of the data in the second model data segment in the third storage area.

[0092] Combine the following Figure 3 For explanation. Figure 3 Another flow chart of a model operation method provided by an embodiment of the present application is shown. The method of this embodiment may include:

[0093] S301, detecting the startup of an application and determining a model object that can be called by the application.

[0094] S302: Determine a first model data segment from the model data of the model object stored in the first storage area.

[0095] The first model data segment is data determined according to the running order of the model data of the model object.

[0096] S303: Load the first model data segment into the second storage area.

[0097] The data processing speed of the second storage area is higher than that of the first storage area. For example, the first storage area is an area in a disk, and the second storage area is an area in a physical memory of a processor running an application program.

[0098] S304: In response to a first request for calling a model object, execute the first model data segment loaded into the second storage area, and load the second model data segment of the model object stored in the first storage area into the third storage area.

[0099] The first request is triggered by user input information.

[0100] The data processing speed of the third storage area is higher than that of the first storage area. For example, the third storage area may be an area in the physical memory of the processor running the application program, but the third storage area and the second storage area are different areas in the physical memory. Of course, the second storage area and the third storage area may have other possibilities, as described above.

[0101] The second model data segment is data whose running sequence is subsequent to the running sequence of the first model data segment.

[0102] S305, after completing the operation of the first model data segment, sequentially operate the data in the second model data segment that has been loaded into the third storage area.

[0103] The above steps can be referred to the related introduction of the previous embodiment, which will not be repeated here.

[0104] S306, after completing the processing of the first request based on the model object, if no request to call the model object is detected within the set time, release at least part of the data in the second model data segment loaded in the third storage area, and maintain the first model data segment loaded in the second storage area unchanged.

[0105] It is understandable that if no request to call the model object is received for a long time, it means that the user does not need to use the model object temporarily. On this basis, releasing at least part of the second model data loaded in the third storage area can reduce the third storage area occupied by the model object, which may cause other applications to be unable to be loaded and run.

[0106] For example, the third storage area is an area in the physical memory corresponding to the CPU. Releasing at least part of the data loaded in the third storage area can reduce the amount of physical memory occupied by the model object corresponding to the CPU, thereby reducing the situation where the application cannot run due to insufficient physical memory corresponding to the CPU.

[0107] Moreover, in order to be able to start the operation of the model object more efficiently when the model object is called again subsequently, the present application will maintain the first model data segment cached in the second storage area unchanged while releasing at least part of the data in the third storage area. In this way, when the model object is called again, the first model data segment loaded in the second storage area can still be directly run, thereby realizing the operation of the model object without waiting for data loading.

[0108] There are many possibilities for the specific implementation of releasing the data in the third storage area. For example, all the data of the second model data segment loaded in the third storage area may be released.

[0109] For example, in order to reduce the impact of occupying the third storage area on other applications and to reduce the amount of model data loaded when the model object is called again, the present application can also determine at least part of the data that needs to be released in the second model data segment loaded into the third storage area based on the current available resource status of the electronic device, and release at least part of the data in the third storage area.

[0110] Among them, the current available resource status of the electronic device includes at least the available memory resources of the processor running the model object in the electronic device, and of course can also include the resource occupancy or estimated resource demand of the application currently running in the processor, etc., without limitation.

[0111] For example, if the available resource status indicates that the available resources of the electronic device are insufficient, all the second model data segments loaded in the third storage area may be released; if the available resource status indicates that the available resources of the electronic device are sufficient, only a small portion of the second model data segments loaded in the third storage area may be released.

[0112] It is understandable that when an electronic device runs a model object, each component module in the model object will be run in sequence. However, for any component module of the model object, not all data of the component model needs to be run every time the component module is used to process data. In particular, based on different requests for calling the model object, the data in the component module that needs to be used when running the component module will also be different.

[0113] For example, taking a machine learning model such as a neural network model as an example, the model object may include multiple network layers, and each network layer may involve multiple nodes. To execute a request processing, only data from some nodes in the network layer may be required.

[0114] On this basis, in order to reduce the data of the model object that needs to be loaded, the present application can also determine, in response to the first request, the sub-model data in the model data of the model object that is required to process the first request. The sub-model data segment may include data required for at least one component module in the model object to process the first request. Accordingly, when the second model data segment stored in the first storage area is loaded into the third storage area, the second model data segment is data belonging to the sub-model data, and the running order is after the running order of the first model data segment.

[0115] There are multiple possible implementations for determining the sub-model data required by the model object to process the first request. In one possible implementation, in response to the first request, a prediction model associated with the application may be called, and based on the first request, the prediction model is used to predict the sub-model data required by the model object to process the first request. The prediction model may be a pre-trained machine learning model or other intelligent model, without limitation.

[0116] It is understandable that in actual applications, there may be multiple model objects that can be called by the application. In this case, if the application can call multiple model objects, after detecting that the application is started, at least one model object that needs to be preloaded among the multiple model objects can be determined based on the current available resource status of the electronic device.

[0117] Correspondingly, for each model object that needs to be preloaded, a first model data segment is determined from the model data of the model object stored in the first storage area, and the first model data segment is loaded into the second storage area.

[0118] Among them, due to the different composition structures and parameters of different model objects, the number of component modules that need to be preloaded after the application is started is also different for different models. For example, taking the structure of the model object including multiple layers as an example, some model objects may be suitable for preloading 5 layers of data, while some model objects can preload 10 layers of data. Therefore, the amount of data in the first model data segment of different model objects is different.

[0119] The space size of the second storage area corresponding to each model object is positively correlated with the data volume of the first model data segment of the model object. Moreover, the second storage areas corresponding to different model objects are also different.

[0120] It can be understood that, for any model object, the specific implementation of loading the first model data segment of the model object into the second storage area can refer to the relevant introduction of the previous embodiment, which will not be repeated here.

[0121] The at least one model object that needs to be preloaded can be determined in combination with the current available resource status of the electronic device and the set rules, and there is no restriction on this.

[0122] For example, if the electronic device currently has sufficient available resources, the multiple model objects are all determined as model objects that need to be preloaded. If the electronic device currently has insufficient available resources, at least one model object that needs to be preloaded is determined according to the priorities of the multiple model objects and at least one of the historical user usage data. The historical user usage data of the model object is the usage data such as the number of times, frequency, and preference of the model object in history. Of course, there may also be other data that characterizes the usage of the model object by the user, and there is no limitation on this.

[0123] Furthermore, in the case where the application can call multiple model objects, during the running of the application, it is also possible to call another model object after calling one model object. Figure 4 For explanation. Figure 4, shows another flow chart of the model operation method provided in the embodiment of the present application. The method of this embodiment may include:

[0124] S401, detecting the startup of an application, and determining a model object that can be called by the application.

[0125] S402: If the application can call multiple model objects, based on the current available resource status of the electronic device, determine at least one model object that needs to be preloaded among the multiple model objects.

[0126] S403: For each model object that needs to be preloaded, determine a first model data segment from the model data of the model object stored in the first storage area, and load the first model data segment into a second storage area corresponding to the model object.

[0127] The first model data segment is data determined according to the running order of the model data of the model object, and the data volume of the first model data segment of different model objects is different.

[0128] For any model object, the data processing speed of the second storage area is higher than that of the first storage area. For example, the first storage area belongs to a disk, and the model data of each model object is stored in a different storage area of ​​the disk. The second storage area corresponding to the model object can be a memory area allocated by the operating system from the physical memory for the application to run the model object.

[0129] The space size of the second storage area corresponding to each model object is positively correlated with the data volume of the first model data segment of the model object.

[0130] S404, in response to a first request to call a first model object, execute a first model data segment corresponding to the first model object and loaded into the second storage area, and load a second model data segment of the first model object stored in the first storage area into a third storage area.

[0131] The first model object belongs to a plurality of model objects that can be called by the application program. In this embodiment, it is taken as an example that the first model object also belongs to a model object that needs to be preloaded.

[0132] S405: Obtain a second request for calling a second model object, and execute the first model data segment loaded into the second storage area corresponding to the second model object.

[0133] S406: If the storage space of the target storage unit is insufficient, release at least part of the data in the second model data segment corresponding to the first model object cached in the third storage area, and load the second model data segment of the second model object stored in the first storage area to the fourth storage area.

[0134] In the present application, it is taken as an example that the second model object also belongs to the model object that needs to be preloaded among multiple model objects.

[0135] It can be understood that the amount of the second model data segments corresponding to the first model object that need to be released can be determined based on the current available resources of the electronic device and the amount of model data of the second model object, without specific limitation.

[0136] The third storage area and the fourth storage area both belong to the target storage unit, and the data processing speed of the target storage unit is higher than the data processing speed of the first storage area. For example, the target storage unit may be the physical memory corresponding to the processor running the application, and the third storage area and the fourth storage area are different areas in the physical memory.

[0137] In one possible implementation, since the second storage area of ​​each model object may also be the same as the third storage area, the second storage area, the third storage area and the fourth storage area of ​​each model object may all belong to the target storage unit. For example, if the target storage unit is the physical memory corresponding to the CPU, then the second storage area, the third storage area and the fourth storage area are all different areas in the physical memory.

[0138] It can be understood that if the second model object does not belong to the model objects that need to be preloaded, then when calling the second model object, it can also be determined based on actual needs whether it is necessary to release at least part of the data of the second model data segment cached in the third storage area, so that the model data of the second model object can be loaded into the target storage unit, and the specific implementation is not restricted.

[0139] Corresponding to a model operation method of the present application, the present application also provides a model operation device. Figure 5 , shows another schematic diagram of the structure of the model operation device provided by the present application. The device of this embodiment may include:

[0140] The object detection unit 501 detects the startup of an application and determines a model object that can be called by the application;

[0141] A data determining unit 502 is used to determine a first model data segment from the model data of the model object stored in the first storage area, wherein the first model data segment is data determined according to the running order of the model data of the model object;

[0142] A preloading unit 503, configured to load the first model data segment into a second storage area, wherein a data processing speed of the second storage area is higher than a data processing speed of the first storage area;

[0143] The model running unit 504 is used to run the first model data segment loaded into the second storage area in response to a first request to call the model object, and load the second model data segment of the model object stored in the first storage area into the third storage area, the first request is triggered by user input information, the data processing speed of the third storage area is higher than the data processing speed of the first storage area, and the second model data segment is data whose running order is after the running order of the first model data segment.

[0144] In a possible implementation, the model running device further includes:

[0145] A synchronous operation unit is used to sequentially operate the data in the second model data segment that has been loaded into the third storage area after completing the operation of the first model data segment.

[0146] In another possible implementation, the model running device further includes:

[0147] A data release unit is used to release at least part of the data in the second model data segment loaded in the third storage area and maintain the first model data segment loaded in the second storage area unchanged after completing the processing of the first request based on the model object, if no request to call the model object is detected within a set time period.

[0148] In yet another possible implementation, the data releasing unit includes:

[0149] A data release subunit is used to determine, after completing the processing of the first request based on the model object, if no request to call the model object is detected within a set time period, based on the current available resource status of the electronic device, at least part of the data in the second model data segment loaded into the third storage area that needs to be released, release the at least part of the data in the third storage area, and maintain the first model data segment loaded in the second storage area unchanged.

[0150] In yet another possible implementation, the data determination unit includes:

[0151] a model determination subunit, configured to determine, if the application can call multiple model objects, at least one model object that needs to be preloaded among the multiple model objects based on the current available resource status of the electronic device;

[0152] A data determination subunit is used to determine, for each model object that needs to be preloaded, a first model data segment from the model data of the model object stored in a first storage area; wherein the data volume of the first model data segment of different model objects is different, and the space size of the second storage area corresponding to each of the model objects is positively correlated with the data volume of the first model data segment of the model object.

[0153] In yet another possible implementation, the model determination subunit includes:

[0154] A first model determination subunit, configured to determine all of the plurality of model objects as model objects that need to be preloaded if the currently available resources of the electronic device are sufficient;

[0155] The second model determination subunit is used to determine at least one model object that needs to be preloaded according to at least one of the priorities of the multiple model objects and historical user usage data if the current available resources of the electronic device are insufficient.

[0156] In yet another possible implementation, the model running unit includes:

[0157] an execution subunit, configured to execute, in response to a first request for calling a first model object, a first model data segment corresponding to the first model object and loaded into the second storage area, and load the second model data segment of the first model object stored in the first storage area into a third storage area;

[0158] The model running device also includes:

[0159] A parallel operation unit, configured to obtain a second request for calling a second model object, and to execute the first model data segment loaded into the second storage area corresponding to the second model object;

[0160] a release control unit, configured to release at least part of the data in the second model data segment corresponding to the first model object cached in the third storage area, and load the second model data segment of the second model object stored in the first storage area into a fourth storage area if the storage space of the target storage unit is insufficient;

[0161] Wherein, the third storage area and the fourth storage area both belong to the target storage unit, and the data processing speed of the target storage unit is higher than the data processing speed of the first storage area;

[0162] The first model object and the second model object are both model objects that need to be preloaded among the multiple model objects.

[0163] In yet another possible implementation, the model running unit includes:

[0164] a data selection subunit, configured to determine sub-model data in the model data of the model object that is required to process the first request;

[0165] The data loading subunit is used to load the second model data segment stored in the first storage area into the third storage area, wherein the second model data segment belongs to the sub-model data.

[0166] The present application also provides an electronic device. Figure 6 As shown, it shows a schematic diagram of a composition structure of the electronic device, which at least includes: at least one processor 601 and at least one processing model that can run on the processor, and the processing model can be called by an application to perform the following operations:

[0167] Detecting the launch of an application, and determining a model object that can be called by the application;

[0168] Determine a first model data segment from the model data of the model object stored in the first storage area, wherein the first model data segment is data determined according to a running order of the model data of the model object;

[0169] Loading the first model data segment into a second storage area, wherein a data processing speed of the second storage area is higher than a data processing speed of the first storage area;

[0170] In response to a first request to call the model object, the first model data segment loaded into the second storage area is run, and the second model data segment of the model object stored in the first storage area is loaded into the third storage area, the first request is triggered by user input information, the data processing speed of the third storage area is higher than the data processing speed of the first storage area, and the second model data segment is data whose running order is after the running order of the first model data segment.

[0171] The operations performed by the processor may refer to the related introduction of the previous embodiment.

[0172] Furthermore, the electronic device also includes a memory 602;

[0173] The memory 602 is used to store programs required for the processor to perform operations.

[0174] It can be understood that the electronic device may further include a display unit 603 and an input unit 604 .

[0175] Of course, the electronic device may also have Figure 6 There may be more or fewer components, without limitation.

[0176] A computer program product is also provided in an embodiment of the present application, including computer-readable instructions. When the computer-readable instructions are executed on an electronic device, the electronic device implements any model operation method provided in the embodiment of the present application.

[0177] A computer-readable storage medium is also provided in an embodiment of the present application. The storage medium carries one or more computer programs. When the one or more computer programs are executed by an electronic device, the electronic device can implement any model operation method provided in the embodiment of the present application.

[0178] It should also be noted that the device embodiments described above are merely schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed over multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. In addition, in the drawings of the device embodiments provided by the present application, the connection relationship between the modules indicates that there is a communication connection between them, which may be specifically implemented as one or more communication buses or signal lines.

[0179] Through the description of the above implementation mode, the technicians in the field can clearly understand that the present application can be implemented by means of software plus necessary general hardware, and of course, it can also be implemented by special hardware including special integrated circuits, special CPUs, special memories, special components, etc. In general, all functions completed by computer programs can be easily implemented by corresponding hardware, and the specific hardware structure used to implement the same function can also be various, such as analog circuits, digital circuits or special circuits. However, for the present application, software program implementation is a better implementation mode in more cases. Based on such an understanding, the technical solution of the present application is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a readable storage medium, such as a computer floppy disk, a U disk, a mobile hard disk, a ROM, a RAM, a disk or an optical disk, etc., including a number of instructions to enable a computer device (which can be a personal computer, a training device, or a network device, etc.) to execute the methods described in each embodiment of the present application.

[0180] In the above embodiments, all or part of the embodiments may be implemented by software, hardware, firmware or any combination thereof. When implemented by software, all or part of the embodiments may be implemented in the form of a computer program product.

[0181] The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions may be transmitted from a website site, a computer, a training device, or a data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, training device, or data center. The computer-readable storage medium may be any available medium that a computer can store or a data storage device such as a training device, a data center, etc. that includes one or more available media integrations. The available medium may be a magnetic medium, (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive (SSD)), etc.

Claims

1. A model operation method, comprising: Detecting the launch of an application, and determining a model object that can be called by the application; Determine a first model data segment from the model data of the model object stored in the first storage area, wherein the first model data segment is data determined according to a running order of the model data of the model object; Loading the first model data segment into a second storage area, wherein a data processing speed of the second storage area is higher than a data processing speed of the first storage area; In response to a first request to call the model object, the first model data segment loaded into the second storage area is run, and the second model data segment of the model object stored in the first storage area is loaded into the third storage area, the first request is triggered by user input information, the data processing speed of the third storage area is higher than the data processing speed of the first storage area, and the second model data segment is data whose running order is after the running order of the first model data segment.

2. The model operation method according to claim 1, further comprising: After the execution of the first model data segment is completed, the data in the second model data segment that has been loaded into the third storage area is executed in sequence.

3. The model operation method according to claim 1 or 2, further comprising: After completing the processing of the first request based on the model object, if no request to call the model object is detected within a set time period, at least part of the data in the second model data segment loaded in the third storage area is released, and the first model data segment loaded in the second storage area is maintained unchanged.

4. The model running method according to claim 3, wherein releasing at least part of the data in the second model data segment loaded in the third storage area comprises: Based on the current available resource status of the electronic device, it is determined that at least part of the data in the second model data segment that has been loaded into the third storage area needs to be released, and the at least part of the data in the third storage area is released.

5. The model running method according to claim 1, wherein determining the first model data segment from the model data of the model object stored in the first storage area comprises: If the application can call multiple model objects, based on the current available resource status of the electronic device, determine at least one model object that needs to be preloaded among the multiple model objects; For each model object that needs to be preloaded, determining a first model data segment from the model data of the model object stored in the first storage area; The data volumes of the first model data segments of different model objects are different, and the space size of the second storage area corresponding to each model object is positively correlated with the data volume of the first model data segment of the model object.

6. The model running method according to claim 5, wherein the step of determining at least one model object that needs to be preloaded among the plurality of model objects based on the current available resource status of the electronic device comprises: If the currently available resources of the electronic device are sufficient, all of the multiple model objects are determined as model objects that need to be preloaded; If the currently available resources of the electronic device are insufficient, at least one model object that needs to be preloaded is determined according to at least one of the priorities of the multiple model objects and historical user usage data.

7. The model running method according to claim 5, wherein in response to a first request for calling the model object, running the first model data segment loaded into the second storage area, and loading the second model data segment of the model object stored in the first storage area into the third storage area, comprises: In response to a first request for calling a first model object, running a first model data segment corresponding to the first model object and loaded into the second storage area, and loading a second model data segment of the first model object stored in the first storage area into a third storage area; The model operation method further comprises: Obtaining a second request for calling a second model object, and executing a first model data segment loaded into a second storage area corresponding to the second model object; If the storage space of the target storage unit is insufficient, releasing at least part of the data in the second model data segment corresponding to the first model object cached in the third storage area, and loading the second model data segment of the second model object stored in the first storage area into the fourth storage area; Wherein, the third storage area and the fourth storage area both belong to the target storage unit, and the data processing speed of the target storage unit is higher than the data processing speed of the first storage area; The first model object and the second model object are both model objects that need to be preloaded among the multiple model objects.

8. The model running method according to claim 1, wherein loading the second model data segment of the model object stored in the first storage area into the third storage area comprises: Determine sub-model data in the model data of the model object that is required to process the first request; The second model data segment stored in the first storage area is loaded into the third storage area, the second model data segment belonging to the sub-model data.

9. A model operation device, comprising: An object detection unit detects the startup of an application and determines a model object that can be called by the application; a data determining unit, configured to determine a first model data segment from the model data of the model object stored in the first storage area, wherein the first model data segment is data determined according to a running order of the model data of the model object; a preloading unit, configured to load the first model data segment into a second storage area, wherein a data processing speed of the second storage area is higher than a data processing speed of the first storage area; A model running unit is used to run the first model data segment loaded into the second storage area in response to a first request to call the model object, and load the second model data segment of the model object stored in the first storage area into the third storage area, the first request is triggered by user input information, the data processing speed of the third storage area is higher than the data processing speed of the first storage area, and the second model data segment is data whose running order is after the running order of the first model data segment.

10. An electronic device, comprising: At least one processor and at least one processing model capable of running on the processor, the processing model capable of being called by an application program to perform the following operations: Detecting the launch of an application, and determining a model object that can be called by the application; Determine a first model data segment from the model data of the model object stored in the first storage area, wherein the first model data segment is data determined according to a running order of the model data of the model object; Loading the first model data segment into a second storage area, wherein a data processing speed of the second storage area is higher than a data processing speed of the first storage area; In response to a first request to call the model object, the first model data segment loaded into the second storage area is run, and the second model data segment of the model object stored in the first storage area is loaded into the third storage area, the first request is triggered by user input information, the data processing speed of the third storage area is higher than the data processing speed of the first storage area, and the second model data segment is data whose running order is after the running order of the first model data segment.

Images