Model memory management method and device based on memory recovery and electronic equipment

Abstract

The invention relates to the technical field of smart home, and discloses a model memory management method and device based on memory recovery and electronic device.The model memory management method comprises the steps that task monitoring is conducted on a target model on equipment, and a model comprehensive index is calculated according to a task monitoring result; under the condition that the model comprehensive index exceeds a preset threshold value, performing memory demand prediction on the equipment based on a memory demand prediction model to obtain a prediction result; according to the prediction result, determining to-be-rejected elements in a memory of the equipment; and removing the to-be-removed element from the memory of the equipment to realize memory recovery. According to the method, the memory recovery strategy can be dynamically adjusted and executed according to the real-time state of the model task obtained through task monitoring, the memory resources are reasonably allocated, and resource competition between tasks is avoided, so that the complex requirements of the target model for the memory in different stages are met, and the intelligent degree of model memory management is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of smart home, for example, relates to a model memory management method and device based on memory recycling, and electronic equipment. BACKGROUND

[0002] With the continuous development of artificial intelligence technology, deep learning large models have been widely applied in various fields. However, these large models usually require a large amount of computing resources and memory space, especially when running on end-side devices, they face greater challenges. In smart home scenarios, the computing power and memory resources of end-side devices (such as smart central control devices) are limited, while running large models (such as voice assistant large models) often requires a large amount of memory resources. This leads to a situation of insufficient memory when processing complex tasks or simultaneously processing multiple tasks, affecting the running efficiency and performance of the large model.

[0003] In related technologies, a memory management method based on cache replacement algorithm is disclosed, which uses common cache replacement algorithms such as Least Recently Used (LRU) algorithm, Least Frequently Used (LFU) algorithm, etc. to manage data in memory. When the memory is insufficient, some infrequently used data is selected for replacement according to the algorithm to release memory space.

[0004] In the process of implementing the embodiments of the present disclosure, it is found that at least the following problems exist in the related art:

[0005] Only the single factor of data access frequency is considered, and the characteristics of the model and the environment of the end-side device are not fully considered, which cannot adapt to the change of memory demand of the model at different stages, and the intelligent degree is low.

[0006] It should be noted that the information disclosed in the above background section is only used to strengthen the understanding of the background of the present application, and therefore can include information that does not constitute prior art known to those of ordinary skill in the art. SUMMARY

[0007] In order to have a basic understanding of some aspects of the disclosed embodiments, the following is a simple summary. The summary is not a general review, nor is it intended to determine the key / important elements or delineate the scope of protection of these embodiments, but as a prelude to the detailed description below.

[0008] The embodiments of the present disclosure provide a model memory management method and device based on memory recycling, and electronic equipment, to adapt to the complex memory demand of the model at different stages and improve the intelligent degree of memory management.

[0009] In some embodiments, the memory recycling-based model memory management method comprises: task monitoring on a target model on a device, and calculating a model comprehensive index according to the task monitoring result; in the case that the model comprehensive index exceeds a preset threshold, performing memory demand prediction on the device based on a memory demand prediction model to obtain a prediction result; determining a to-be-removed element in the memory of the device according to the prediction result; and removing the to-be-removed element in the memory of the device to realize memory recycling.

[0010] Optionally, the task monitoring on the target model on the device comprises: tracking and monitoring at least one of a task execution progress of the target model, a device resource consumption condition, and a device state based on a task monitoring system.

[0011] Optionally, the memory demand prediction model is constructed in the following manner: obtaining multi-source data; the multi-source data comprises at least one of historical task execution data of the target model, model characteristic features, device state parameters, and user behavior features; modeling the historical task execution data based on a deep learning method to capture a dynamic change trend of the memory demand; modeling the model characteristic features, the device state parameters, and the user behavior features based on a machine learning method to extract a nonlinear relationship; and combining modeling results of the deep learning method and the machine learning method to obtain the memory demand prediction model.

[0012] Optionally, the determination of the to-be-removed element in the memory of the device according to the prediction result comprises: screening elements in the memory of the device based on a cache eviction algorithm according to the prediction result to obtain a candidate removal list; evaluating each element in the candidate removal list according to an importance score function to obtain an importance score of each element; and obtaining the to-be-removed element according to the importance score of each element.

[0013] Optionally, the importance score function determines the importance score of the element according to at least one of an importance of the element to a model task, a timeliness of data, whether the element can be regenerated, and a change trend of an access frequency of the element.

[0014] Optionally, the removal of the to-be-removed element in the memory of the device comprises: adjusting a priority of the to-be-removed element; and removing a to-be-removed element with the highest priority according to the adjusted priority.

[0015] Optionally, the adjustment of the priority of the to-be-removed element comprises: adjusting the priority of the to-be-removed element according to at least one of the importance score function, a criticality of the task, and a load condition of the device.

[0016] Optionally, the memory management method further comprises: after completing the memory recycling, adjusting parameters and weight coefficients of the importance score function according to a system performance after the memory recycling and a model task execution condition to adjust a memory recycling decision.

[0017] In some embodiments, the memory recycling-based model memory management apparatus includes a processor and a memory storing program instructions, the processor being configured to execute the memory recycling-based model memory management method as described above when running the program instructions.

[0018] In some embodiments, the electronic device includes an electronic device body, and the memory recycling-based model memory management apparatus as described above is installed in the electronic device body.

[0019] The memory recycling-based model memory management method and apparatus, and the electronic device provided by the embodiments of the present disclosure can achieve the following technical effects:

[0020] In the embodiments of the present disclosure, through task monitoring, model comprehensive index calculation can be performed when the device is in a high load state and the memory occupancy rate is high, the memory usage of the model task is tracked in real time, and memory demand prediction of the device is performed when the model comprehensive index exceeds the preset threshold, so as to trigger memory recycling. Through the memory demand prediction model, the future memory demand of the device can be predicted, and memory planning and adjustment can be performed in advance. According to the prediction result, the elements to be removed in the memory of the device are dynamically determined, and the elements to be removed are recycled to release the memory. Therefore, the embodiments of the present disclosure can dynamically adjust and execute the memory recycling strategy according to the real-time state of the model task, reasonably allocate memory resources, and avoid resource competition between tasks, so that the complex demand of the target model on memory at different stages can be adapted, and the intelligent degree of model memory management is improved.

[0021] The foregoing general description and the following description are merely exemplary and explanatory, and are not intended to limit the present application. BRIEF DESCRIPTION OF DRAWINGS

[0022] One or more embodiments are exemplarily illustrated by corresponding drawings, which do not constitute a limitation on the embodiments, elements with the same reference numerals in the drawings are shown as similar elements, the drawings do not constitute a proportional limitation, and wherein:

[0023] Figure 1 is an application environment schematic diagram of a memory recycling-based model memory management method provided by the embodiments of the present disclosure;

[0024] Figure 2 is a schematic diagram of a memory recycling-based model memory management method provided by the embodiments of the present disclosure;

[0025] Figure 3 is a schematic diagram of a method for constructing a memory demand prediction model provided by the embodiments of the present disclosure;

[0026] Figure 4 is a schematic diagram of another model memory management method based on memory recycling provided by an embodiment of the present disclosure;

[0027] Figure 5 is a schematic diagram of another model memory management method based on memory recycling provided by an embodiment of the present disclosure;

[0028] Figure 6 is a schematic diagram of a model memory management device based on memory recycling provided by an embodiment of the present disclosure. DETAILED DESCRIPTION

[0029] In order to enable a person skilled in the art to more fully understand the features and technical contents of the embodiments of the present disclosure, the implementation of the embodiments of the present disclosure will be described in detail below with reference to the accompanying drawings, which are used for reference only and do not limit the embodiments of the present disclosure. In the following technical description, in order to facilitate explanation, a plurality of details are provided to provide a full understanding of the disclosed embodiments. However, one or more embodiments can still be implemented without these details. In other cases, well-known structures and devices can be simplified to facilitate the drawings.

[0030] The terms "first", "second", and the like used in the technical solutions described in the present application are used to distinguish similar objects, and do not necessarily describe a specific order or sequence. It should be understood that the data thus used can be interchanged under appropriate circumstances, so that the embodiments of the present disclosure described herein can be implemented. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion.

[0031] Unless otherwise specified, the term "a plurality of" means two or more.

[0032] In the embodiments of the present disclosure, the character " / " represents a "or" relationship between the objects before and after it. For example, A / B represents: A or B.

[0033] The term "and / or" is a description of the association between objects, which means that there can be three relationships. For example, A and / or B, which means: A or B, or, A and B, three kinds of relationship.

[0034] The term "corresponding" can refer to an association or binding relationship, A corresponding to B means that there is an association or binding relationship between A and B.

[0035] According to an aspect of the embodiments of the present application, a model memory management method based on memory recycling is provided. The model memory management method is widely applied to smart home, smart home, smart home device ecology, intelligence house ecology, and other whole-house intelligent digital control application scenarios. Optionally, in the present embodiment, the above-mentioned model memory management method can be applied to the hardware environment composed of terminal device 102 and server 104 as shown in Figure 1 As shown in Figure 1 The server 104 is connected with the terminal device 102 through the network, which can be used to provide services (such as application services, etc.) for the terminal or the client installed on the terminal, and a database can be set on the server or independently of the server, which is used to provide data storage services for the server 104, and cloud computing and / or edge computing services can be configured on the server or independently of the server, which is used to provide data operation services for the server 104.

[0036] The above-mentioned network can include but is not limited to at least one of the following: wired network, wireless network. The above-mentioned wired network can include but is not limited to at least one of the following: wide area network, metropolitan area network, local area network, and the above-mentioned wireless network can include but is not limited to at least one of the following: WIFI (Wireless Fidelity, Wireless Fidelity), Bluetooth. The terminal device 102 can not be limited to PC, mobile phone, tablet computer, smart air conditioner, smart oven, smart refrigerator, smart oven, smart oven, smart washing machine, smart water heater, smart washing equipment, smart dishwasher, smart projection equipment, smart television, smart clothesline, smart curtain, smart audio and video, smart socket, smart sound, smart sound box, smart fresh air equipment, smart kitchen and bathroom equipment, smart bathroom equipment, smart sweeping robot, smart window cleaning robot, smart mopping robot, smart air purification equipment, smart steamer, smart microwave oven, smart kitchen treasure, smart purifier, smart water dispenser, smart door lock, etc. According to the application of the terminal device 102, corresponding models can be set on the terminal device, such as voice assistant large model, network communication model, Internet of Things perception model, etc.

[0037] As shown in Figure 2 The model memory management method based on memory recycling provided by the embodiments of the present disclosure, the execution subject of the method can be the processor in the above-mentioned terminal device, and the method comprises:

[0038] S201, the processor monitors the target model on the device, and calculates the model comprehensive index according to the task monitoring result.

[0039] S202, the processor predicts the memory demand of the device based on the memory demand prediction model in the case that the model comprehensive index exceeds the preset threshold, and obtains the prediction result.

[0040] S203, the processor determines the element to be pruned in the memory of the device according to the prediction result.

[0041] S204, the processor prunes the element to be pruned in the memory of the device to realize memory recycling.

[0042] In the embodiments of the present disclosure, through task monitoring, model comprehensive index calculation can be performed when the device is in a high load state and the memory occupancy is high, the memory usage of the model task is tracked in real time, and memory demand prediction of the device is performed when the model comprehensive index exceeds the preset threshold to trigger memory recycling. Through the memory demand prediction model, the future memory demand of the device can be predicted, and memory planning and adjustment can be performed in advance. According to the prediction result, the element to be pruned in the memory of the device is dynamically determined, and the element to be pruned is recycled to release the memory. Therefore, the embodiments of the present disclosure can dynamically adjust and execute the memory recycling strategy according to the real-time state of the model task, reasonably allocate memory resources, and avoid resource competition between tasks, so as to adapt to the complex memory demand of the target model at different stages and improve the intelligent degree of model memory management.

[0043] Optionally, the task monitoring of the target model on the device includes: tracking and monitoring at least one of the task execution progress of the target model, the device resource consumption state, and the device state based on a task monitoring system.

[0044] In this embodiment, by establishing a task monitoring system, the task execution progress, resource consumption, and device state of the target model on the device can be tracked in real time, providing comprehensive data support for the calculation of the model comprehensive index. For example, for an image recognition large model, the processing time after each image input, data throughput (such as the number of images processed per second), model parameter update amount, and other indicators are monitored. At the same time, the system state of the device is closely monitored, including the utilization rate of CPU and GPU, memory occupancy, disk I / O speed, etc. If the device is in a high load state and the memory occupancy is close to the danger threshold, memory checking will be triggered, and at this time, the model comprehensive index can be calculated according to the task execution progress, resource consumption, and device state, so as to predict the risk of insufficient memory of the device in advance, and actively trigger memory recycling to improve the intelligent degree of model memory management. Through task monitoring, it can also ensure that memory recycling is triggered at the most appropriate time to avoid early or late recycling affecting model performance.

[0045] Optionally, the task execution progress includes at least one of the completion percentage of the task in the target model, the current data volume being processed, and the remaining time of the task.

[0046] Optionally, the device resource consumption condition comprises at least one of memory occupancy, CPU utilization, GPU utilization, and disk I / O speed.

[0047] Optionally, the device status comprises at least one of device temperature, battery level, storage capacity, and network connection status.

[0048] Optionally, the model comprehensive index is calculated according to the following formula:

[0049] StatusIndex=ω1×TaskProgress+ω2×ResourceConsumption+ω3×DeviceLoa d

[0050] wherein, StatusIndex is the model comprehensive index, TaskProgress is the task execution progress index, ResourceConsumption is the device resource consumption condition index, DeviceLoad is the device status index, and ω1, ω2, ω3 are corresponding weight coefficients.

[0051] In this embodiment, the model comprehensive index is used to measure the comprehensive state of the target model and the device, and when the model comprehensive index exceeds the preset threshold, it indicates that the target model and the device are in a high load state and the memory occupancy is high.

[0052] Optionally, the task execution progress index can be the completion percentage of the task in the target model. For an image recognition large model task, the task execution progress index can be the ratio of the number of processed images to the total number of images.

[0053] Optionally, the device resource consumption condition index can be the weighted sum of memory occupancy, CPU utilization, GPU utilization, etc.

[0054] Optionally, the device resource consumption condition index is calculated according to the following formula:

[0055] ResourceConsumption=ω4×MemoryOccupancy+ω5×CPUUtilization+ω6×GPUUtilization

[0056] wherein, MemoryOccupancy is the memory occupancy, CPUUtilization is the CPU utilization, GPUUtilization is the GPU utilization, and ω4, ω5, ω6 are corresponding weight coefficients.

[0057] Optionally, the device status index can be a function of device temperature, battery level, etc.

[0058] Optionally, the device state indicator is calculated according to the following formula:

[0059] DeviceLoad = ω7 x Temperature + ω8 x (1 - BatteryLevel)

[0060] wherein Temperature is the device temperature, BatteryLevel is the battery level, and ω7 and ω8 are corresponding weight coefficients.

[0061] Optionally, ω1, ω2, ω3, ω4, ω5, ω6, ω7, and ω8 can be adjusted according to the characteristics of the type and model of the device.

[0062] Optionally, the memory demand prediction model is constructed in the following manner: obtaining multi-source data; the multi-source data includes at least one of historical task execution data of a target model, model characteristic features, device state parameters, and user behavior features; based on a deep learning method, modeling the historical task execution data to capture the dynamic change trend of memory demand; based on a machine learning method, modeling the model characteristic features, device state parameters, and user behavior features to extract nonlinear relationships; and combining the modeling results of the deep learning method and the machine learning method to obtain the memory demand prediction model.

[0063] In combination with Figure 3 As shown in the drawings, the embodiments of the present disclosure provide a method for constructing a memory demand prediction model, and the execution subject of the method can also be a processor in the terminal device described above. The method comprises the following steps:

[0064] S301, the processor obtains multi-source data; the multi-source data includes at least one of historical task execution data of a target model, model characteristic features, device state parameters, and user behavior features.

[0065] S302, the processor models the historical task execution data based on a deep learning method to capture the dynamic change trend of memory demand.

[0066] S303, the processor models the model characteristic features, device state parameters, and user behavior features based on a machine learning method to extract nonlinear relationships.

[0067] S304, the processor combines the modeling results of the deep learning method and the machine learning method to obtain the memory demand prediction model.

[0068] In this embodiment, by comprehensively considering the influence of multi-source data on model memory demand, the obtained memory demand prediction model can accurately predict the memory demand of the target model in a certain future time period, avoid memory shortage or waste caused by inaccurate prediction, and improve the accuracy of memory demand prediction. By combining deep learning and machine learning, a complex memory demand prediction model can be constructed, which can dynamically adapt to changes in different task types, device states and user behaviors, and improve the intelligent degree of memory management. For the obtained memory demand prediction model, the input is a multi-dimensional vector including at least one of historical task execution data, device state parameters, model characteristic features and user behavior features, and the output is a memory demand prediction value in a certain future time period.

[0069] Optionally, the historical task execution data includes at least one of memory usage peak value, average memory consumption and memory demand trend over time of different types of tasks.

[0070] Optionally, the device state parameters include at least one of CPU utilization, GPU utilization, memory occupancy, storage capacity of the device and network connection state.

[0071] Optionally, the model characteristic features include at least one of structure, parameter quantity, layer type and size of the target model.

[0072] Optionally, the user behavior features include at least one of frequency of using the target model by the user in a certain time period, size and type of input data.

[0073] Optionally, the deep learning method includes convolutional neural network, recurrent neural network or long short-term memory network.

[0074] Optionally, the machine learning method includes random forest.

[0075] Optionally, the memory demand prediction model performs memory demand prediction according to the following formula:

[0076] PredictedMemoryDemand=f(HistoricalDataFeatures,DeviceStateParameters,ModelCharacteristicsFeatures,UserBehaviorFeatures)

[0077] wherein PredictedMemoryDemand is the memory demand prediction value, HistoricalDataFeatures is the historical task execution data, DeviceStateParameters is the device state parameter, ModelCharacteristicsFeatures is the model characteristic feature, UserBehaviorFeatures is the user behavior feature, and f is the prediction function.

[0078] Optionally, the prediction function can be a combination of a deep learning function and a machine learning function.

[0079] Optionally, the prediction function is determined according to the following formula:

[0080] f = LSTM(HistoricalDataFeatures) + RandomForest(DeviceStateParameters, ModelCharacteristicsFeatures, UserBehaviorFeatures)

[0081] wherein LSTM is a long short-term memory network function, and RandomForest is a random forest function.

[0082] In this embodiment, for the LSTM part, the historical task execution data is input, and an intermediate prediction result LSTMOutput is obtained after processing by the LSTM network. For the RandomForest part, the device state parameter, the model characteristic feature, and the user behavior feature are input, and another intermediate prediction result RandomForestOutput is obtained after processing by the RandomForest model. Finally, the two intermediate prediction results are added to obtain the final memory demand prediction value, i.e., f = LSTMOutput + RandomForestOutput.

[0083] Optionally, according to the prediction result, the element to be removed in the memory of the device is determined, including: according to the prediction result, screening the elements in the memory of the device based on a cache eviction algorithm to obtain a candidate removal list; according to an importance score function, evaluating each element in the candidate removal list to obtain an importance score of each element; and according to the importance score of each element, obtaining the element to be removed.

[0084] In combination with Figure 4 As shown in FIG. 8, the embodiments of the present disclosure provide another model memory management method based on memory recycling, including:

[0085] In S401, the processor performs task monitoring on the target model on the device, and calculates a model comprehensive index according to the task monitoring result.

[0086] In S402, the processor performs memory requirement prediction on the device based on the memory requirement prediction model, and obtains a prediction result, when the model comprehensive index exceeds a preset threshold.

[0087] In S403, the processor screens elements in the memory of the device based on a cache eviction algorithm according to the prediction result, and obtains a candidate elimination list.

[0088] In S404, the processor evaluates each element in the candidate elimination list according to an importance scoring function, and obtains an importance score of each element.

[0089] In S405, the processor obtains an element to be eliminated according to the importance score of each element.

[0090] In S406, the processor eliminates the element to be eliminated in the memory of the device to realize memory recycling.

[0091] In this embodiment, the element to be eliminated is determined based on the cache eviction algorithm and the importance scoring function according to the prediction result of the memory requirement of the device, so that the least critical memory element can be accurately selected for recycling, important data can be avoided from being mistakenly deleted, and the accuracy of memory recycling is improved.

[0092] Optionally, the cache eviction algorithm includes LRU or LFU.

[0093] Optionally, the importance scoring function determines the importance score of the element according to at least one of the importance of the element to the model task, the timeliness of the data, whether the element can be regenerated, and the access frequency trend of the element.

[0094] Optionally, the importance scoring function is determined according to the following formula:

[0095] ImportanceScore(element)=ω9×TaskImportance(element)+ω10×Timeliness(element)+ω11×Reproducibility(element)+ω12×AccessFrequencyTrend(elemen t)

[0096] wherein, ImportanceScore(element) is an importance score function, TaskImportance(element) is the importance of the element to the model task, Timeliness(element) is the timeliness degree of the data, Reproducibility(element) is the reproducibility of the element, AccessFrequencyTrend(element) is the access frequency trend of the element, and ω9, ω10, ω11, and ω12 are corresponding weight coefficients.

[0097] Optionally, the importance of the element to the model task is determined according to a task dependency graph of the target model and the role of the element in the task.

[0098] Optionally, the importance of the element to the model task is determined according to the position of the element in the computational graph of the target model and the influence degree on the result. For example, if the element is a parameter of a key layer of the target model, the importance of the element to the task is higher.

[0099] Optionally, the timeliness degree of the data is determined according to the update frequency and the validity period of the data. If the update frequency of the data is higher or the validity period of the data is shorter, the timeliness of the data is lower.

[0100] Optionally, the reproducibility of the element is determined according to whether the element can be regenerated from external storage or other data sources. If the element can be easily regenerated, the reproducibility of the element is higher.

[0101] Optionally, if the element can be regenerated, Reproducibility(element) is 0; if the element cannot be regenerated, Reproducibility(element) is 1.

[0102] Optionally, the access frequency trend of the element is determined by predicting the future access frequency trend through a time series analysis method (such as an autoregressive moving average model). If the predicted access frequency is lower, the access frequency trend of the element is lower.

[0103] Optionally, ω9, ω10, ω11, and ω12 can be adjusted according to the type of the target model and the performance of the device.

[0104] Optionally, the to-be-eliminated elements are eliminated in the memory of the device, including: adjusting the priorities of the to-be-eliminated elements; and eliminating the to-be-eliminated element with the highest priority according to the adjusted priorities.

[0105] In combination Figure 5 As shown in the figure, the embodiment of the present disclosure provides another model memory management method based on memory recycling, including:

[0106] S501, the processor performs task monitoring on the target model on the device, and calculates a model comprehensive index according to the task monitoring result.

[0107] S502, the processor performs memory requirement prediction on the device based on the memory requirement prediction model in the case where the model comprehensive index exceeds a preset threshold, and obtains a prediction result.

[0108] S503, the processor screens elements in the memory of the device based on a cache eviction algorithm according to the prediction result, and obtains a candidate elimination list.

[0109] S504, the processor evaluates each element in the candidate elimination list according to an importance score function, and obtains an importance score of each element.

[0110] S505, the processor obtains an element to be eliminated according to the importance score of each element.

[0111] S506, the processor adjusts the priority of the element to be eliminated.

[0112] S507, the processor eliminates the element to be eliminated with the highest priority in the memory of the device according to the adjusted priority, to realize memory recycling.

[0113] In this embodiment, during the memory recycling process, the elimination priority of the element is dynamically adjusted according to the real-time task execution condition and device state of the target model, which can accurately select the least critical memory element for recycling, avoid deleting important data by mistake, and improve the accuracy of memory recycling. For example, if the task of the target model enters a critical stage, the protection level of the element with high task importance is increased, and the possibility of elimination is reduced.

[0114] Optionally, the adjustment of the priority of the element to be eliminated includes: adjusting the priority of the element to be eliminated according to at least one of the importance score function, the criticality of the task, and the load condition of the device.

[0115] In this embodiment, the dynamic adjustment of the priority is realized by comprehensively considering multiple factors, which can improve the intelligent degree of memory management.

[0116] Optionally, the adjustment of the priority is represented as follows:

[0117] Priority(element)=ImportanceScore(element)×ω13×TaskCriticality+ω14×SystemLoad

[0118] wherein Priority(element) is the adjusted priority, TaskCriticality is the criticality of the task, SystemLoad is the load condition of the device, and ω13 and ω14 are corresponding weight coefficients.

[0119] Optionally, the criticality of the task is determined according to the progress, importance, and other factors of the task. For example, if the task is in a critical stage or has a high requirement for the accuracy of the result, the criticality of the task is high.

[0120] Optionally, the load condition of the device is determined according to a function of the memory occupancy, CPU utilization, GPU utilization, and the like.

[0121] Optionally, the load condition of the device is calculated according to the following formula:

[0122] SystemLoad = ω15 x MemoryOccupancy + ω16 x CPUUtilization + ω17 x GPUUtilization

[0123] wherein MemoryOccupancy is the memory occupancy, CPUUtilization is the CPU utilization, GPUUtilization is the GPU utilization, and ω15, ω16, and ω17 are corresponding weight coefficients.

[0124] Optionally, ω13, ω14, ω15, ω16, and ω17 can be adjusted according to the type of the model and the performance of the device.

[0125] Optionally, the model memory management method further includes: after completing the memory recovery, adjusting the parameters and weight coefficients of the importance scoring function according to the system performance after the memory recovery and the model task execution condition, to adjust the memory recovery decision.

[0126] In this embodiment, a feedback mechanism is introduced to continuously optimize the importance scoring function and improve the efficiency and effect of memory recovery. After completing the memory recovery, the memory recovery strategy can be continuously optimized according to the actual running condition, the adaptability of memory recovery is improved, and the memory resources can be efficiently managed in different scenarios.

[0127] The embodiments of the present disclosure are described below with the application in a smart home.

[0128] In the smart home environment, a set of smart home control systems are matched, and the system connects various smart home appliances such as smart TVs, smart speakers, smart air conditioners, smart lights, etc. through a smart central control device. At the same time, the central control device also runs a deep learning-based voice assistant large model for receiving user voice instructions and controlling various home appliance devices. As users use smart home more and more frequently, the voice assistant large model needs to handle more and more tasks, which leads to the shortage of memory resources of the central control device. Especially when processing multiple tasks or complex tasks at the same time, there may be a situation of insufficient memory, which affects the response speed and accuracy of the voice assistant. At this time, the model memory management method based on memory recycling provided by the embodiments of the present disclosure can recycle the memory resources of the central control device to adapt to the complex memory requirements of the voice assistant large model at different stages.

[0129] The central control device monitors the task execution progress, device resource consumption status and device state of the voice assistant large model in real time through the task monitoring system. For example, when a user issues a voice instruction, the central control device starts processing the instruction through the voice assistant large model, and monitors the time required to process the instruction, the data throughput (such as the amount of voice data processed per second), the model parameter update amount, etc. At the same time, the system state of the device is also monitored, including the utilization rate of CPU and GPU, the memory occupancy rate, the disk I / O speed, etc.

[0130] In the case that the central control device is in a high load state and the memory occupancy is high, such as the memory occupancy reaching 80%, memory checking is triggered. At this time, the central control device judges whether memory recovery is needed according to a model comprehensive index as described above. The model comprehensive index is: StatusIndex = ω1 × TaskProgress + ω2 × ResourceConsumption + ω3 × DeviceLoad. Wherein, TaskProgress is the completion percentage of the current processing voice instruction task; ResourceConsumption is the weighted sum of memory occupancy, CPU utilization, GPU utilization, etc., such as ResourceConsumption = ω4 × MemoryOccupancy + ω5 × CPUUtilization + ω6 × GPUUtilization, wherein MemoryOccupancy is the memory occupancy, CPUUtilization is the CPU utilization, and GPUUtilization is the GPU utilization; DeviceLoad is a function of device temperature, battery level, etc., such as DeviceLoad = ω7 × Temperature + ω8 × (1-BatteryLevel), wherein Temperature is the device temperature, and BatteryLevel is the battery level. If the StatusIndex exceeds a certain threshold, such as 0.8, memory recovery is triggered.

[0131] After triggering memory recycling, the central control device needs to perform memory demand prediction through a memory demand prediction model. The construction of the memory demand prediction model requires collecting a large amount of historical voice assistant task execution records, including the memory usage peak value, average memory consumption, and memory demand trend over time of different types of tasks. For example, when the user issues different voice instructions such as "turn on the TV", "turn up the air conditioner temperature", "play music", etc., the central control device will record the memory usage during the processing of these instructions. Using deep learning algorithms such as convolutional neural networks or recurrent neural networks, historical data is modeled to extract feature patterns of voice assistant tasks to predict future task memory demand. At the same time, consider device state parameters, in addition to basic CPU, GPU utilization and memory occupancy, also include device storage capacity, network connection status and other factors. For large voice assistant models, analyze the model's structure, parameter quantity, layer type and size, etc. If the task involves user interaction, analyze the user's behavior patterns and input habits, such as the frequency of using the voice assistant in a certain time period, the type and length of input instructions, etc. Finally, use a combination of deep learning and traditional machine learning to build a prediction model. For example, use a long short-term memory network to model time series data to capture the dynamic trend of memory demand; at the same time, use a random forest to model device state parameters and model characteristics, extracting nonlinear relationships. Define the input of the memory demand prediction model as a multi-dimensional vector, including historical task execution data, device state parameters, model characteristic features, and user behavior features, etc. The output is the memory demand prediction value in a certain time period in the future. The prediction process can be represented by the following formula: PredictedMemoryDemand = f(HistoricalDataFeatures, DeviceStateParameters, ModelCharacteristicsFeatures, UserBehaviorFeatures). Where f is the prediction function, which can be a combination of deep learning models and machine learning models, such as f = LSTM(HistoricalDataFeatures) + RandomForest(DeviceStateParameters, ModelCharacteristicsFeatures, UserBehaviorFeatures).

[0132] After obtaining the prediction results according to the memory requirement prediction model, the elements in the memory need to be evaluated according to the prediction results to determine the elements to be pruned. In combination with the cache eviction algorithm and the element importance score, a comprehensive evaluation algorithm is designed. First, the cache eviction algorithm is used to preliminarily screen the elements in the memory to obtain a candidate pruning list. For example, if the LRU algorithm is used, the least recently used elements will be selected and put into the candidate pruning list. Then, for each element in the candidate pruning list, further evaluation is performed according to its importance score. The importance score considers multiple factors, including the importance of the element to the voice assistant task, the timeliness of the data, whether it can be regenerated, the access frequency trend of the element, etc. The importance score function is defined as: ImportanceScore(element) = ω9 × TaskImportance(element) + ω10 × Timeliness(element) + ω11 × Reproducibility(element) + ω12 × AccessFrequencyTrend(element). Among them, TaskImportance(element) is the importance degree of the element to the voice assistant task, which can be determined according to the task dependency graph of the voice assistant and the role of the element in the task. For example, if an element is a parameter of a key layer of a large voice assistant model, its task importance is high. Timeliness(element) is the timeliness degree of the data, which can be determined according to the update frequency and validity period of the data. If the update frequency of the data is high or the validity period is short, the timeliness is low. Reproducibility(element) is the reproducibility of the element, which is 0 for reproducible and 1 for non-reproducible. For example, if an element can be regenerated from external storage or other data sources, its reproducibility is high. AccessFrequencyTrend(element) is the access frequency trend of the element, which can be predicted using time series analysis methods. If the predicted access frequency is low, the access frequency trend is low.

[0133] After determining the elements to be pruned in memory, the elements to be pruned in memory are pruned to realize memory recycling. During the memory recycling process, the priority of the elements is dynamically adjusted according to the real-time voice assistant task execution and device state. For example, if the voice assistant is processing an urgent task, the protection level of the elements with high importance is increased, and the possibility of being pruned is reduced. The dynamic priority adjustment can be represented using the following formula: Priority(element) = ImportanceScore(element) x ω13 x TaskCriticality + ω14 x SystemLoad. Wherein, TaskCriticality is the criticality of the voice assistant task, which can be determined according to the progress, importance and other factors of the task. For example, if the task is in a critical stage or has high accuracy requirements for the result, the criticality of the task is high. SystemLoad is the load of the device, which can be a function of memory occupancy, CPU utilization, GPU utilization, etc., such as SystemLoad = ω15 x MemoryOccupancy + ω16 x CPUUtilization + ω17 x GPUUtilization, wherein MemoryOccupancy is the memory occupancy, CPUUtilization is the CPU utilization, and GPUUtilization is the GPU utilization. In addition, a feedback mechanism can be introduced to adjust the parameters and weight coefficients of the importance scoring function according to the system performance and voice assistant task execution after memory recycling, to optimize future memory recycling decisions.

[0134] The model memory management method based on memory recycling provided by the embodiments of the present disclosure can start memory recycling in time when the device memory is about to be insufficient or the device is under high load, maximize the use of memory resources without affecting task execution, avoid system crashes or lag caused by memory depletion, and realize optimal use of resources through precise triggering mechanism, thereby improving the stability of the system. By making memory planning and adjustment in advance, the dynamic changes in different scenarios can be adapted to, whether it is the change of task type, the fluctuation of device state or the change of user behavior, the memory demand can be accurately predicted, thereby realizing more flexible memory management and avoiding delay caused by insufficient memory during task execution. During the memory recycling process, the least critical memory elements can be selected for recycling to avoid deleting important data. At the same time, the priority is dynamically adjusted according to the real-time situation to ensure efficient recycling process.

[0135] In combination with Figure 6As shown, the embodiment of the present disclosure provides a model memory management device 600 based on memory recycling, comprising a processor 601 and a memory 602. Optionally, the device can also comprise a communication interface 603 and a bus 604. Wherein the processor 601, the communication interface 603, the memory 602 can complete the communication among each other through the bus 604. The communication interface 603 can be used for information transmission. The processor 601 can call the logical instructions in the memory 602 to execute the model memory management method based on memory recycling of the above-mentioned embodiments.

[0136] In addition, the logical instructions in the memory 602 described above can be realized in the form of a software functional unit and sold or used as an independent product, which can be stored in a computer readable storage medium.

[0137] The memory 602 as a kind of computer readable storage medium can be used to store software programs, computer executable programs, such as program instructions / modules corresponding to the method in the embodiment of the present disclosure. The processor 601 executes the program instructions / modules stored in the memory 602, thereby executing function application and data processing, i.e. realizing the model memory management method based on memory recycling in the above-mentioned embodiments.

[0138] The memory 602 can include a program storage area and a data storage area, wherein the program storage area can store an operating system, at least one application required by a function; the data storage area can store data created according to the use of the terminal device, etc. In addition, the memory 602 can include a high-speed random access memory, and can also include a non-volatile memory.

[0139] The embodiment of the present disclosure provides an electronic device, comprising: an electronic device body, and the model memory management device based on memory recycling described above. The model memory management device based on memory recycling is installed in the electronic device body. The installation relationship described herein is not limited to placing in the electronic device, but also includes the installation connection with other components of the electronic device, including but not limited to physical connection, electrical connection or signal transmission connection, etc. Those skilled in the art can understand that the model memory management device based on memory recycling can be adapted to the feasible electronic device body, and then realize other feasible embodiments.

[0140] The embodiment of the present disclosure provides a computer readable storage medium, which stores computer executable instructions, and the computer executable instructions are set to execute the model memory management method based on memory recycling.

[0141] The technical solutions of the embodiments of the present disclosure can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes one or more instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method disclosed in the embodiments of the present disclosure. The aforementioned storage medium can be a non-transitory storage medium, including: a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and various media that can store program codes.

[0142] The above description and drawings sufficiently illustrate the embodiments of the present disclosure to enable one skilled in the art to practice them. Other embodiments can include structural, logical, electrical, process, and other changes. The embodiments represent only a few of the possible variations. Individual components and functions are optional unless explicitly required, and the order of operations can be changed. Parts and features of some embodiments can be included in or replace parts and features of other embodiments. Also, the words used in this application are only used to describe the embodiments and not to limit the technical solutions described in this application. As used in the technical solutions described in this application, unless the context clearly indicates otherwise, the singular form "a", "an" and "the" are intended to include the plural form as well. Similarly, as used in this application, the term "and / or" refers to any and all possible combinations of one or more associated listed items. In addition, when used in this application, the term "comprise" and its variants "comprises" and / or "comprising" and the like refer to the presence of the stated features, integers, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups of these. Without more limitations, the element defined by the phrase "comprising a" does not exclude the presence of additional identical elements in the process, method, or device that includes the stated element. In this document, each embodiment focuses on the differences from other embodiments, and the same or similar parts between various embodiments can be referred to each other. For the method, product, etc. disclosed in the embodiments, if it corresponds to the method part disclosed in the embodiments, the relevant part can be referred to the description of the method part.

[0143] Those skilled in the art can understand that the units and algorithm steps of each example described in combination with the embodiments disclosed herein can be realized by electronic hardware or a combination of computer software and electronic hardware. Whether the functions are realized in hardware or software depends on the specific application and design constraints of the technical solution. The skilled person can use different methods for each specific application to realize the described functions, but such implementation should not be considered beyond the scope of the embodiments of the present disclosure. The skilled person can clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described system, device and unit can refer to the corresponding processes in the foregoing method embodiments, which will not be repeated here.

[0144] In the embodiments disclosed herein, the disclosed methods, products (including but not limited to devices, equipment, etc.) can be implemented in other ways. For example, the above-described device embodiments are only schematic, for example, the division of the units can only be a logical function division, and actual implementation can have another division manner, for example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the units shown or discussed can be indirect coupling or communication connection through some interface, device or unit, and can be electrical, mechanical or other forms. The units described as separate components can or can not be physically separate, and the components shown as units can or can not be physical units, that is, they can be located in one place, or can be distributed on a plurality of network units. Part or all of the units can be selected according to actual needs to implement the embodiments. In addition, each functional unit in the embodiments of the present disclosure can be integrated in one processing unit, or each unit can be a physically independent unit, or two or more units can be integrated in one unit.

[0145] The computer program instructions can also be loaded onto a computer, other programmable data processing apparatus, or other processing device to cause a series of operational steps to be performed on the computer, other programmable apparatus or other processing device to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide processes for implementing the functions / acts specified in the flowchart and / or block diagram block or blocks.

Claims

1. A model memory management method based on memory recycling, characterized in that: include: Perform task monitoring on the target model on the device and calculate the model comprehensive indicators based on the task monitoring results; When the comprehensive index of the model exceeds a preset threshold, performing memory demand prediction on the device based on the memory demand prediction model to obtain a prediction result; Determining, based on the prediction result, elements to be removed from the memory of the device; The elements to be eliminated are eliminated from the memory of the device to achieve memory recovery.

2. The model memory management method based on memory recycling according to claim 1, characterized in that: Perform task monitoring on the target model on the device, including: Based on the task monitoring system, at least one of the task execution progress, device resource consumption status and device status of the target model is tracked and monitored.

3. The model memory management method based on memory recycling according to claim 1, characterized in that: Build a memory demand prediction model as follows: Acquire multi-source data; the multi-source data includes at least one of historical task execution data of the target model, model characteristics, device state parameters, and user behavior characteristics; Based on a deep learning method, the historical task execution data is modeled to capture the dynamic change trend of memory requirements; Modeling the model characteristics, the device state parameters, and the user behavior characteristics based on a machine learning method to extract nonlinear relationships; The modeling results of the deep learning method and the machine learning method are combined to obtain a memory demand prediction model.

4. The model memory management method based on memory recycling according to any one of claims 1 to 3, characterized in that: Determining, based on the prediction result, elements to be removed from the memory of the device, including: According to the prediction result, the elements in the memory of the device are screened based on a cache elimination algorithm to obtain a candidate elimination list; Evaluate each element in the candidate elimination list according to an importance scoring function to obtain an importance score for each element; According to the importance score of each element, the elements to be eliminated are obtained.

5. The model memory management method based on memory recycling according to claim 4 is characterized in that: The importance scoring function determines the importance score of an element according to at least one of the importance of the element to the model task, the timeliness of the data, whether the element can be regenerated, and the access frequency change trend of the element.

6. The model memory management method based on memory recycling according to claim 4 is characterized in that: Eliminating the element to be eliminated from the memory of the device includes: Adjusting the priority of the elements to be removed; According to the adjusted priority, the element to be removed with the highest priority is removed.

7. The model memory management method based on memory recycling according to claim 6, characterized in that: Adjusting the priority of the elements to be removed includes: The priority of the elements to be removed is adjusted according to at least one of the importance scoring function, the criticality of the task, and the load condition of the device.

8. The model memory management method based on memory recycling according to claim 4 is characterized in that: Also includes: After memory reclamation is completed, the parameters and weight coefficients of the importance scoring function are adjusted according to the system performance and model task execution status after memory reclamation to adjust the memory reclamation decision.

9. A model memory management device based on memory recycling, comprising a processor and a memory storing program instructions, characterized in that: The processor is configured to execute the model memory management method based on memory recycling according to any one of claims 1 to 8 when running the program instructions.

10. An electronic device, characterized in that: include: Electronic device body; The model memory management device based on memory recycling as described in claim 9 is installed in the electronic device body.

Images