Model adaptation method and device for streaming player terminal and storage medium

Abstract

The application relates to a model adaptation method and device for a streaming media playing terminal and a storage medium. At least two AI models are deployed in the streaming media playing terminal, the at least two AI models are used to execute the same type of AI task, and different AI models correspond to different resource consumption levels, the resource consumption level is used to represent the consumption degree of the hardware resource when the AI model runs; the method comprises the following steps: acquiring the current available hardware resource information of the streaming media playing terminal in the streaming media playing process; selecting an AI model with the highest matching degree from the at least two AI models as a target AI model according to the matching degree between the current available hardware resource information and the resource consumption levels corresponding to the at least two AI models respectively; and loading the target AI model to execute the AI task while maintaining the streaming media playing uninterrupted. According to the application, the streaming media playing terminal can adaptively match the AI model according to the hardware resource state, and the user experience can be effectively improved.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a model adaptation method, apparatus and storage medium for a streaming media playback terminal. Background Technology

[0002] Streaming media playback devices (such as mobile smart screens and smart projectors) often rely on cloud-deployed AI models when performing tasks like AI image enhancement. However, if the device loses internet access or the network becomes unstable, it will severely impact the user's viewing experience. Furthermore, even with a network connection, uploading video data to the cloud in real time for AI processing introduces additional bandwidth consumption and privacy risks. Summary of the Invention

[0003] In view of this, embodiments of this application provide a model adaptation method, apparatus, and storage medium for a streaming media playback terminal to solve at least one problem existing in the background art.

[0004] In a first aspect, embodiments of this application provide a model adaptation method for a streaming media playback terminal, wherein the streaming media playback terminal deploys at least two AI models, the at least two AI models are used to perform the same type of AI task, and different AI models correspond to different resource consumption levels, the resource consumption level being used to characterize the degree of hardware resource consumption by the AI ​​model during operation; the method includes: During streaming media playback, obtain the currently available hardware resource information of the streaming media playback terminal; Based on the degree of matching between the currently available hardware resource information and the resource consumption level corresponding to each of the at least two AI models, the AI ​​model with the highest degree of matching is selected as the target AI model from the at least two AI models; While maintaining uninterrupted streaming playback, the target AI model is loaded to perform the AI ​​task.

[0005] In conjunction with the first aspect, in an optional implementation, the resource consumption level is divided according to at least one of the following parameters: peak memory usage, average power consumption, and model inference latency; when the resource consumption level is divided according to multiple parameters, each resource consumption level corresponds to a combination of numerical ranges of each parameter.

[0006] In conjunction with the first aspect, in an optional implementation, determining the target AI model from the at least two AI models based on the degree of matching between the currently available hardware resource information and the resource consumption levels corresponding to each of the at least two AI models includes: Based on the resource consumption level corresponding to each AI model, the expected performance indicators of each AI model in the following multiple dimensions are obtained: device power consumption, memory usage, and model inference latency; Based on the remaining power, device temperature, memory usage, and processor load in the currently available hardware resource information, determine the corresponding demand weights for device power consumption, memory usage, and model inference latency, respectively. For each AI model, based on the expected performance indicators and corresponding demand weights of the AI ​​model in each dimension, a matching score is calculated between the resource consumption level of the AI ​​model and the currently available hardware resource information, wherein the matching score represents the degree of matching. The AI ​​model with the highest matching score is selected as the target AI model.

[0007] In conjunction with the first aspect, in an optional implementation, the demand weight corresponding to the dimension has a preset mapping relationship with at least one parameter in the currently available hardware resource information; Among them, the remaining power is negatively correlated with the demand weight of the device power consumption dimension, the device temperature is positively correlated with the demand weight of the device power consumption dimension, the memory utilization rate is positively correlated with the demand weight of the memory utilization dimension, and the processor load is positively correlated with the demand weight of the model inference latency dimension.

[0008] In conjunction with the first aspect, in an alternative embodiment, the method further includes: Based on the changing trends of the currently available hardware resources, the demand weights corresponding to each dimension are dynamically adjusted.

[0009] In conjunction with the first aspect, in an optional implementation, loading the target AI model to perform the AI ​​task while maintaining uninterrupted streaming playback includes: If the first AI model is not currently loaded, then the target AI model is loaded; If the first AI model is currently loaded, and the first AI model is the target AI model, then the loaded first AI model is retained. If the first AI model is currently loaded, and the first AI model is different from the target AI model, then it is determined whether the expected resource consumption of the target AI model exceeds the available resource limit for its corresponding task type. The available resource limit is determined based on the currently available hardware resource information. If the limit is exceeded, the first AI model that has already been loaded will be retained; If the limit is not exceeded, then load the target AI model and release the resources occupied by the first AI model.

[0010] In conjunction with the first aspect, in an alternative embodiment, the method further includes: Output a prompt message, which prompts the user to confirm whether to apply the target AI model; In response to the user's confirmation, the step of loading the target AI model to perform the AI ​​task is executed.

[0011] In conjunction with the first aspect, in an optional implementation, the output prompt information includes: A floating prompt is displayed in a non-critical area of ​​the streaming media playback interface. The floating prompt contains an operation entry point for receiving user confirmation or rejection of the operation.

[0012] In conjunction with the first aspect, in an optional implementation, the streaming media playback terminal currently has multiple AI tasks to be processed, and the method further includes: Obtain available model information corresponding to each AI task to be processed, wherein at least one of the AI ​​tasks to be processed corresponds to a candidate set containing at least two candidate models; Based on the currently available hardware resource information, determine the total available resources that can be used by the multiple AI tasks to be processed; Under the constraint of the total available resources, AI target models are determined for the multiple AI tasks to be processed, such that the sum of the expected resource consumption of each target AI model determined to be used does not exceed the total available resources. While maintaining uninterrupted streaming playback, each of the determined AI target models is loaded separately.

[0013] Secondly, embodiments of this application provide a model adaptation device for a streaming media playback terminal. The streaming media playback terminal deploys at least two AI models, which are used to perform the same type of AI task. Different AI models correspond to different resource consumption levels, whereby the resource consumption level characterizes the degree of hardware resource consumption during AI model operation. The device includes: The acquisition module is used to acquire the currently available hardware resource information of the streaming media playback terminal during streaming media playback. The determination module is used to select the AI ​​model with the highest matching degree from the at least two AI models as the target AI model based on the matching degree between the currently available hardware resource information and the resource consumption level corresponding to each of the at least two AI models; A loading module is used to load the target AI model to perform the AI ​​task while maintaining uninterrupted streaming playback.

[0014] Thirdly, embodiments of this application provide a computer-readable storage medium having an executable program stored thereon, wherein the executable program, when executed by a processor, implements the model adaptation method for a streaming media playback terminal as provided in any optional embodiment of the first aspect.

[0015] The model adaptation method, apparatus, and storage medium for streaming media playback terminals provided in this application deploy multiple AI models with different resource consumption levels corresponding to the same AI task type within the streaming media playback terminal. During streaming media playback, the terminal's currently available hardware resource information is acquired in real time, and the most suitable target model is automatically determined and switched based on the matching degree between the currently available hardware resource information and the model's resource consumption level. This completes the loading of the AI ​​model while maintaining uninterrupted streaming media playback. Since it executes entirely locally on the streaming media playback terminal, without relying on cloud model services or network connection status, it can still provide stable AI functionality even in weak or no network environments. Simultaneously, based on dynamic perception and adaptive matching of hardware resource status, the terminal can leverage the advantages of high-performance models when resources are abundant and automatically downgrade to low-resource-consumption models when resources are limited. This reduces video stuttering and frame drops caused by excessive AI model usage and slows down battery drain during continuous playback, effectively improving the user experience. Attached Figure Description

[0016] Figure 1 This is a flowchart illustrating a model adaptation method for a streaming media playback terminal according to an embodiment. Figure 2 for Figure 1 The flowchart of step S120 is shown below; Figure 3 for Figure 1 The flowchart of step S130 is shown below; Figure 4 This is a flowchart illustrating a model adaptation method for a streaming media playback terminal according to another embodiment; Figure 5 A schematic diagram of a model adapter for a streaming media playback terminal is shown according to one embodiment. Detailed Implementation

[0017] To make the technical solution and beneficial effects of this application more apparent and understandable, a detailed description is provided below by listing specific embodiments. The accompanying drawings are not necessarily drawn to scale, and local features may be enlarged or reduced to more clearly show the details of the local features; unless otherwise defined, the technical and scientific terms used herein have the same meanings as those in the technical field to which this application pertains.

[0018] The embodiments in this application are not exhaustive, but merely illustrative of some embodiments, and are not intended to limit the scope of protection of this disclosure. Unless otherwise specified, each step in a particular embodiment can be implemented as an independent embodiment, and the steps can be arbitrarily combined. For example, a solution after removing some steps in a particular embodiment can also be implemented as an independent embodiment, and the order of the steps in a particular embodiment can be arbitrarily interchanged. Furthermore, the optional implementation methods in a particular embodiment can be arbitrarily combined; moreover, the embodiments can be arbitrarily combined, for example, some or all steps of different embodiments can be arbitrarily combined, and a particular embodiment can be arbitrarily combined with the optional implementation methods of other embodiments.

[0019] In each embodiment of this application, unless otherwise specified or in case of logical conflict, the terminology and / or descriptions of the embodiments are consistent and can be referenced by each other. Technical features in different embodiments can be combined to form new embodiments based on their inherent logical relationships.

[0020] In the description of the embodiments of this application, it should be understood that the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Therefore, features defined with "first" and "second" may explicitly or implicitly include one or more of the stated features. In the description of the embodiments of this application, "multiple" means two or more, unless otherwise explicitly specified.

[0021] It should be noted that the user information (including but not limited to user device information, user personal information, collected data, used data, generated data, processed data, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, collected information, used information, generated information, processed information, etc.) involved in this application are all information and data authorized by the user. Furthermore, the collection, storage, use, processing, transmission, provision, disclosure, and application of related data all comply with the relevant laws, regulations, and standards of the relevant countries and regions, and necessary confidentiality measures have been taken. These measures do not violate public order and good morals, and corresponding operation entry points are provided for users to choose to authorize or refuse. For example, this system provides users with corresponding interfaces. Before obtaining relevant information, it needs to send an acquisition request to the aforementioned user through the interface, and obtain the relevant information only after receiving the aforementioned user's consent information.

[0022] This disclosure provides a model adaptation method for a streaming media playback terminal, applicable to such terminals. The streaming media playback terminal can be a mobile smart screen (commonly known as a "best friend phone"), tablet computer, smart TV, or other device with streaming media playback capabilities and requiring the execution of AI tasks.

[0023] Inside the streaming media playback terminal, at least two AI models are pre-deployed. These models are used to perform the same type of AI task, such as multiple large language models (for voice interaction response, content understanding, etc.). It's important to note that "at least two AI models for the same type of AI task" means that these models are functionally substitutable, i.e., they can all perform the same AI task (e.g., both are large language model inference). The terminal may also deploy other types of AI models (e.g., in addition to large language models, there may be AI models for image recognition). AI models belonging to different task types are not adapted to the same AI model group. When a certain type of AI task needs to be performed, the terminal will select the corresponding set of AI models for adaptation based on the task type.

[0024] At least two AI models used to perform the same type of AI task each correspond to a resource consumption level. The resource consumption level characterizes the degree to which the AI ​​model consumes terminal hardware resources during runtime. Hardware resources may include at least one of the following: central processing unit (CPU), graphics processing unit (GPU), neural network processor, memory, and power consumption. The resource consumption level can be determined based on hardware resource consumption indicators such as the model's peak memory usage, average power consumption, and model computational accuracy level (e.g., quantization bit width).

[0025] Taking large language model-type AI tasks deployed on terminals as an example, different models vary in their functional support and resource consumption, thus corresponding to different resource consumption levels. For example: the basic model supports basic question answering and short text generation, with peak memory usage typically not exceeding 1.5GB, corresponding to the first resource consumption level; the standard model supports multi-turn dialogue and contextual understanding, with moderate memory usage, corresponding to the second resource consumption level; the enhanced model supports image understanding and local knowledge base retrieval, with peak memory usage reaching 4GB, corresponding to the third resource consumption level.

[0026] like Figure 1 As shown, the model adaptation method for streaming media playback terminals provided in this application includes steps S110 to S130.

[0027] S110: During streaming media playback, obtain information on the currently available hardware resources of the streaming media playback terminal.

[0028] The streaming media playback process refers to the process by which the terminal is decoding and rendering video or audio streams, such as in application scenarios where users need to maintain continuous image and sound output, like watching online videos, live content, or following fitness classes.

[0029] Here, the currently available hardware resource information refers to the remaining hardware resources available for the AI ​​task of the same type to be executed, after deducting the hardware resources occupied by the streaming media decoding and playback task, necessary system background tasks, and other AI tasks currently running in the streaming media playback terminal (if any). This currently available hardware resource information may include, but is not limited to, the remaining battery percentage, the internal temperature of the device (such as battery temperature or processor junction temperature), the load percentage of the processor (CPU / GPU / NPU), and the memory usage rate.

[0030] In some examples, the streaming media playback terminal can periodically acquire the currently available hardware resource information in a way that presets the sampling period, or trigger the acquisition in response to a hardware state change event.

[0031] Step S120: Based on the matching degree between the currently available hardware resource information and the resource consumption level corresponding to each of the at least two AI models, select the model with the highest matching degree from the at least two AI models as the target AI model.

[0032] In some examples, the triggering of step S120 may be based on periodic triggering using a timer or threshold triggering based on hardware state changes. The threshold triggering conditions include: the change in any parameter of the currently available hardware resource information (such as remaining battery power, device temperature, or memory usage) exceeds a preset threshold, or the amount of available resources represented by the currently available hardware resource information is lower than a preset safety threshold.

[0033] It is understandable that even if the streaming media playback terminal currently has an AI model running, if it detects a sudden increase in resource consumption due to system background services, newly launched applications by the user, or other sudden tasks, resulting in the current available hardware resources falling below the preset safety threshold, step S120 will be triggered.

[0034] Furthermore, to suppress logical conflicts and computational overhead caused by repeated triggering from multiple trigger sources within a short period of time, the system can set a trigger debouncing mechanism, such as merging multiple trigger requests received within a time window, or forcibly specifying a minimum time interval between two consecutive matching evaluations.

[0035] In some examples, the matching degree can be calculated in multiple ways. In one exemplary implementation, a rule mapping table can be preset, defining the adaptation tendency relationship between different hardware resource ranges and different resource consumption level models. For example, when the remaining battery power is below a preset threshold (e.g., 20%), the matching degree evaluation value of the high resource consumption level model is lowered, while the matching degree evaluation value of the low resource consumption level model is raised; when the device temperature exceeds a preset temperature threshold (e.g., 45℃), the low resource consumption level model is preferentially adapted; when the processor load exceeds a preset load threshold (e.g., 80%), the model with lower processor resource requirements is preferentially adapted. The streaming media playback terminal calculates a comprehensive matching score for each candidate model and determines the model with the highest score as the target AI model. This application does not limit the specific matching score calculation method used, as long as the optimal adapted model can be selected based on the quantitative relationship between the currently available hardware resource information and the model resource consumption level.

[0036] After determining the target AI model, the streaming media playback terminal compares the target AI model with the currently running AI model (if any). If they are the same, the current running state is maintained and no model loading operation is performed; if they are different, the subsequent loading process can proceed.

[0037] Step S130: While maintaining uninterrupted streaming playback, load the target AI model to execute the AI ​​task.

[0038] The target AI model is loaded into memory while maintaining uninterrupted streaming playback.

[0039] It should be noted that model loading refers to reading the model weight file and computation graph structure data already deployed on the local storage medium of the streaming media playback terminal into the runtime memory. After loading is complete, it can be managed and invoked by a specific process or thread, making the model enter an executable state. Correspondingly, model unloading refers to removing the model data from the runtime memory to release the memory space it occupies, but the model file remains intact in the terminal's persistent storage medium, meaning its deployment state is unaffected.

[0040] Furthermore, while the hardware resources consumed by streaming media playback decoding are typically relatively fixed, the overall available hardware resources on the terminal change in real time depending on factors such as remaining battery power, device temperature, the number of concurrently running tasks, and background system activity. Matching the level of hardware resource consumption (i.e., resource consumption level) with the currently available overall resources ensures uninterrupted playback, prevents sudden drops in battery life, and avoids overheating.

[0041] In some examples, the process of loading the AI ​​model is prohibited from being performed again within a preset time window after each successful loading. This reduces the need for repeated model loading by the terminal due to frequent fluctuations in hardware resource information (such as battery levels fluctuating around a threshold), thus improving system stability.

[0042] In the technical solution provided in this application, the currently available hardware resource information excludes the resource share occupied by streaming media decoding tasks and other necessary system tasks. It actually represents the remaining hardware resources available for AI tasks. Based on this, the terminal selects a target AI model from at least two AI models whose resource consumption level matches the currently available hardware resource information.

[0043] Therefore, when loading the target AI model, the hardware resource consumption required for its operation is adapted to the currently available hardware resources. This ensures that the sum of the additional resource consumption from model operation and the inherent resource consumption of the ongoing streaming media decoding task is always less than the total resource limit provided by the terminal hardware, maintaining a dynamically variable spare resource margin. The size of this spare resource margin can be adaptively adjusted according to real-time changes in the currently available hardware resource information, and it is configured to accommodate the resource requirements of necessary background tasks or anticipated sudden tasks. In scenarios where no other tasks are running, the spare resource margin represents unoccupied remaining hardware resources that can be readily accessed by any subsequent tasks.

[0044] In some examples, as previously described, step S120 can be configured to trigger re-execution in response to the amount of available resources, as represented by the currently available hardware resource information, falling below a preset safety threshold. In this case, if the target AI model is already running, re-executing step S120 will perform a matching evaluation based on the updated available hardware resource information and determine a new target AI model. Accordingly, step S130 performs a switching operation, replacing the currently running AI model with the new target AI model. In this way, the terminal can prioritize the supply of processor time slices and memory bandwidth for the streaming media decoding task, thereby keeping the total system resource usage within the total resource limit and ensuring that the continuity of the streaming media playback process is not affected. In cases of extreme resource scarcity and no model with a lower resource consumption level is available, the inference thread of the target AI model can be temporarily suspended.

[0045] In summary, the model adaptation method for streaming media playback terminals provided in this application deploys multiple AI models with different resource consumption levels corresponding to the same AI task type within the streaming media playback terminal. During streaming media playback, it acquires the terminal's currently available hardware resource information in real time and automatically determines and switches to the most suitable target model based on the matching degree between the currently available hardware resource information and the model's resource consumption level. This completes the loading of the AI ​​model while maintaining uninterrupted streaming media playback. Since the model adaptation process is executed entirely locally on the streaming media playback terminal, without relying on cloud model services or network connection status, it can still provide stable AI functionality even in weak or no network environments. Furthermore, based on dynamic perception and adaptive matching of hardware resource status, the terminal can leverage the advantages of high-performance models when resources are abundant and automatically downgrade to low-resource-consumption models when resources are limited. This reduces video stuttering and frame drops caused by excessive AI model usage and slows down battery drain during continuous playback, effectively improving the user experience.

[0046] In some embodiments, in order to facilitate the quantitative classification and comparison of the resource consumption levels of different AI models, the resource consumption levels are divided according to at least one of the following parameters: peak memory usage, average power consumption, and model inference latency; when the resource consumption levels are divided according to multiple parameters, each resource consumption level corresponds to a combination of numerical ranges of each parameter.

[0047] Peak memory usage refers to the maximum amount of physical memory occupied by the AI ​​model in the terminal's running memory from the start of loading until the completion of a single or continuous inference task. This metric can be used to measure the upper limit of the instantaneous pressure the model puts on the terminal's memory resources. Average power consumption refers to the average energy consumption rate per unit time calculated by the AI ​​model during continuous execution of inference tasks, by obtaining the remaining battery level of the streaming media playback terminal and combining it with the execution duration and number of executions of the inference task. Model inference latency refers to the time interval between the AI ​​model receiving a single inference request and outputting a complete inference result. This metric can be obtained by performing multiple single inferences in a standard test environment and calculating the average time consumption.

[0048] In some embodiments, such as Figure 2 As shown, step S120 above may further include the following steps: S121: Based on the resource consumption level corresponding to each AI model, obtain the expected performance indicators of each AI model in the following multiple dimensions: device power consumption, memory usage, and model inference latency.

[0049] The aforementioned expected performance metrics can be obtained during model training or offline testing and pre-stored in the terminal. Specifically, based on the resource consumption level and model configuration of each AI model, the power consumption value, memory usage value, and latency reference value reflecting the model's inference latency performance can be read from a preset mapping relationship.

[0050] S122: Based on the remaining power, device temperature, memory usage, and processor load in the currently available hardware resource information, determine the respective demand weights for device power consumption, memory usage, and model inference latency.

[0051] The demand weight corresponding to any dimension reflects the sensitivity of the current hardware resource status to that dimension.

[0052] In some embodiments, the demand weights corresponding to dimensions have a preset mapping relationship with at least one parameter in the currently available hardware resource information; wherein, the remaining power is negatively correlated with the demand weight of the device power consumption dimension, that is, the lower the remaining power, the higher the demand weight of the corresponding device power consumption dimension; the device temperature is positively correlated with the demand weight of the device power consumption dimension, that is, the higher the device temperature, the higher the demand weight of the corresponding memory usage dimension; the memory utilization rate is positively correlated with the demand weight of the memory usage dimension, that is, the higher the memory utilization rate, the higher the demand weight of the corresponding memory usage dimension; the processor load is positively correlated with the demand weight of the model inference latency dimension, that is, the higher the processor load, the higher the demand weight of the corresponding inference latency dimension.

[0053] In some examples, the demand weights can be determined using a piecewise linear function. For instance, the power consumption weight is set to 0.2 when the remaining battery level is above 50%, and 0.8 when it is below 20%, with a linear variation in between. The specific settings can be flexibly adjusted based on actual hardware characteristics and expert experience.

[0054] S123: For each AI model, based on the expected performance indicators and corresponding demand weights of the AI ​​model in each dimension, calculate the matching score between the resource consumption level of the AI ​​model and the currently available hardware resource information, wherein the matching score represents the degree of matching.

[0055] For each AI model, the streaming media playback terminal performs a weighted sum of the expected performance metrics of the AI ​​model across each dimension and the corresponding required weights for that dimension, to obtain a matching score between the AI ​​model and the currently available hardware resources. The higher the matching score, the more suitable the model is for the current hardware resources of the streaming media playback terminal.

[0056] When calculating the matching score, negative metrics such as power consumption, memory usage, and inference latency, where smaller values ​​are better, are first normalized and converted into positive scores. Specifically, for each negative metric, the minimum and maximum values ​​of that metric in the candidate model library are calculated. The measured values ​​are then converted into scores between 0 and full marks using a linear mapping function, where the minimum metric corresponds to a full mark, the maximum metric corresponds to 0 marks, and intermediate values ​​are calculated linearly proportionally. Finally, the positive scores of each metric are weighted and summed to obtain the final matching score.

[0057] S124: Select the AI ​​model with the highest matching score as the target AI model.

[0058] In this embodiment, by obtaining the expected indicators of different AI models in multiple resource and performance dimensions, and assigning dynamic demand weights to each dimension based on the available hardware resources of the terminal, the target model is determined by weighted matching. This allows the model selection to be adaptively adjusted as hardware conditions change, thereby balancing the processing power of AI tasks with the terminal resource overhead while ensuring the continuity of streaming media playback, which is beneficial to improving the operational stability of the terminal.

[0059] In some embodiments, the method may further include: dynamically adjusting the demand weights corresponding to each dimension based on the changing trends of currently available hardware resource information.

[0060] For example, trend data of available hardware resource information within a preset historical time window can be obtained; based on the trend data, weight correction factors for each dimension can be generated; the weight correction factors can be fused with the basic demand weight determined based on the current instantaneous hardware resource information to obtain the final demand weight.

[0061] Specifically, the trend data includes, but is not limited to, the rate of decrease of remaining battery power, the rate of increase of device temperature, the variance of processor load fluctuation, and / or the growth slope of memory occupancy. For example, in response to the rate of decrease of remaining battery power exceeding a first preset threshold, or the rate of increase of device temperature exceeding a second preset threshold, a first correction factor for increasing the demand weight of the power consumption dimension can be generated; in response to the fluctuation of processor load exceeding a third preset threshold, a second correction factor for increasing the demand weight of the model inference latency dimension can be generated. The weight correction factors are then fused with the basic demand weights determined based on the current instantaneous hardware resource information to obtain the final demand weights. For example, the first or second correction factor can be used as a multiplicative coefficient on the corresponding basic demand weights.

[0062] For example, a streaming media playback terminal continuously records hardware sensor data using a 30-second sliding time window. When it detects that the remaining battery power drops from 40% to 20% within a continuous 5-minute period (i.e., the rate of decrease is ≥4% / minute), even though the current remaining battery power is still within the relatively safe threshold of 20%, the system determines that the device is in a high-power consumption expected state. At this time, when calculating the final demand weight of the power consumption dimension, a correction coefficient greater than 1 (e.g., 1.2 times) is applied. Similarly, when the device temperature rises by more than 2°C within 10 seconds, even though the current temperature has not reached the frequency reduction threshold, the system determines that the risk of heat accumulation has increased, and simultaneously increases the power consumption weight and the processor load demand weight.

[0063] In this embodiment, the demand weights corresponding to different dimensions are adaptively adjusted by utilizing the changing trends of currently available hardware resource information, so that the model adaptation can have a certain predictive ability and thus be more adaptable to the resource fluctuation state of the terminal.

[0064] In some embodiments, such as Figure 3 As shown, step S130 above may further include the following steps: S131: Determine whether a first AI model with the same task type as the target AI model has been loaded. If yes, proceed to either step S133 or step S134; if no, proceed to step S132. S132: Load the target AI model; S133: If the first AI model is the target AI model, then the loaded first AI model is retained; S134: If the first AI model is different from the target AI model, then determine whether the expected resource consumption of the target AI model exceeds the available resource limit of its corresponding task type. The available resource limit is determined based on the current available hardware resource information. If it exceeds the limit, then execute step S135. If it does not exceed the limit, then execute step S136. S135: Keep the first AI model that has been loaded; S136: Load the target AI model and release the resources occupied by the first AI model.

[0065] Specifically, step S131 confirms whether a first AI model belonging to the same task type as the target AI model has already been loaded. If it is confirmed that no model of the same type has been loaded, step S132 reads the relevant data of the target AI model deployed in the terminal from local storage into the running memory to complete the model instantiation. This memory loading operation is performed in a background thread independent of the decoding thread, so the core computing resources required for video decoding are not affected. After loading is completed, the target AI model is then enabled to process subsequent AI tasks, and the entire process does not interfere with the continuous output of the video.

[0066] If step S131 confirms that a first AI model of the same type has already been loaded, and step S133 determines that the newly determined target AI model is the same model as the first AI model, then the system will not make any changes and will continue to use the already loaded first AI model, thereby avoiding the extra power consumption and memory overhead caused by repeatedly loading the same model.

[0067] If step S133 determines that the target AI model is different from the currently loaded first AI model, then proceed to step S134. In step S134, the system determines an allowed resource limit for this task type based on the currently available hardware resource information, and determines whether the resource consumption required for the target AI model to run exceeds this limit. If the determination result is that it exceeds the limit, then step S135 is executed, that is, the loading of the target AI model is abandoned, the already loaded first AI model continues to run, and corresponding prompt information can be output to the user. This can prevent the forced loading of a high-consumption model when there are insufficient remaining resources, thereby avoiding the risk of the video playback process being terminated due to the operating system's low memory reclamation mechanism triggered by a sudden memory shortage.

[0068] If step S134 determines that the expected resource consumption of the target AI model does not exceed the available resource limit for that task type, then step S136 is executed. The expected resource consumption of the target AI model is determined based on the resource consumption level corresponding to the target AI model. Specifically, the terminal pre-stores a mapping relationship between each resource consumption level and parameters such as peak memory usage and average power consumption. The system reads the corresponding parameter value from this mapping relationship according to the resource consumption level to which the target AI model belongs, and uses it as the expected resource consumption of the target AI model.

[0069] In step S136, the system loads the target AI model from local storage into runtime memory. This loading process is executed asynchronously in the background, during which the first AI model continues to process received inference requests. After the target AI model completes memory loading and initialization, newly generated inference requests are directed to the target AI model. After the first AI model completes processing all existing requests, its occupied memory resources are released. This ensures that the video decoding thread always has a stable operating environment, ensuring that the AI ​​service responds smoothly and without interruption or errors during the transition from the first AI model to the target AI model.

[0070] This embodiment, through the above steps, can complete the loading of the AI ​​model without interrupting streaming media playback, thereby avoiding playback abnormalities caused by forced loading.

[0071] In some embodiments, the method further includes the following steps: outputting a prompt message to prompt the user to confirm whether to use the target AI model; and performing the loading step in response to the user's confirmation operation.

[0072] In this embodiment, a floating prompt can be displayed in a non-critical area of ​​the streaming media playback interface. This floating prompt includes an operation entry point for receiving user confirmation or rejection. Alternatively, it can be output via a notification bar message.

[0073] For example, when the terminal detects low battery and switches from a high-performance model to a power-saving model, a small card can be displayed in the lower right corner of the playback interface before the switch: "Battery only 15% remaining, it is recommended to switch to power-saving picture quality model, click to confirm." The card provides "Confirm" and "Ignore" buttons. The floating prompt includes a one-click confirmation entry for convenient user operation. The terminal only performs the loading operation after the user clicks confirm; if the user clicks ignore or there is no response within a preset time, the current AI model remains unchanged, and the loading operation is not performed. Additionally, for terminals with voice interaction capabilities, the voice assistant can also ask the user whether to apply the target AI model.

[0074] The aforementioned embodiments primarily address model adaptation scenarios for a single AI task. In practical use, a terminal may run multiple different types of AI tasks simultaneously, such as playing videos (requiring image quality enhancement models) while simultaneously performing fitness exercises (requiring posture recognition models). In this case, multiple different types of models will simultaneously consume hardware resources. If each task independently selects a model (i.e., each selects the model with the highest matching degree), it may cause the total memory usage or total power consumption after the selected models are combined to exceed the actual hardware capacity of the terminal, thereby affecting the smoothness of streaming media playback and system stability.

[0075] To address the resource coordination issues in multi-task concurrent scenarios, this embodiment provides a model adaptation method for streaming media playback terminals, such as... Figure 4 As shown, the method includes the following steps: S210: Obtain available model information for each AI task to be processed, wherein at least one AI task to be processed corresponds to a candidate set containing at least two candidate models.

[0076] Specifically, for multiple AI tasks currently pending, the system iterates through each task type to determine the available models for each. For task types that support dynamic adaptation, there is a candidate set containing at least two candidate models, with different candidate models having different resource consumption levels; for task types that only support fixed models, there is only one specific AI model.

[0077] S220: Based on the currently available hardware resource information, determine the total available resources that can be used by multiple AI tasks to be processed.

[0078] Based on the currently available hardware resource information obtained in the preceding steps, the system calculates the total available resources that can be shared by all AI tasks to be processed, after deducting resources already occupied or requiring reservation for fixed tasks such as streaming media decoding. This total available resource can be specifically represented by the upper limit of remaining available memory, the upper limit of tolerable additional power consumption, etc.

[0079] S230: Under the constraint of total available resources, determine the AI ​​target model for multiple AI tasks to be processed, such that the sum of the expected resource consumption of each target AI model determined to be used does not exceed the total available resources.

[0080] For AI tasks with candidate sets, the system selects from their respective candidate sets; for AI tasks that correspond only to a fixed model, the target model is that fixed model. The system performs joint solution under the constraint of total available resources, ensuring that the total resource consumption of each target AI model ultimately determined does not exceed the upper limit of total available resources determined in step S220.

[0081] When there are multiple candidate model combinations that satisfy the total available resource constraint, the comprehensive matching degree corresponding to each candidate model combination is calculated, and each candidate model in the candidate model combination with the highest comprehensive matching degree is determined as the AI ​​target model corresponding to each AI task to be processed; wherein, the comprehensive matching degree is determined according to the matching score of each candidate model in the combination under the corresponding task dimension and the preset priority weight of each AI task to be processed.

[0082] S240: Load the determined AI target models respectively while maintaining uninterrupted streaming playback.

[0083] After determining the target model for each AI task to be processed, the system loads the relevant data of each target model from local storage into the runtime memory. For tasks that already have an old model of the same type but differ between the old and new models, the system releases the memory resources occupied by the old model after loading the new model; for tasks with the same target model as the already loaded model, the original loading state is maintained without change. The entire loading process is executed in the background, without affecting the normal operation of the video decoding thread, thus enabling concurrent execution and resource coordination of multiple different types of AI tasks while ensuring the continuity of streaming media playback.

[0084] Through the above steps, this embodiment can coordinate and allocate limited hardware resources in multi-task concurrent scenarios, alleviate the resource over-limit problem caused by the independent adaptation of each task, and ensure smooth and stable streaming media playback while taking into account the availability and response quality of various AI services.

[0085] The various embodiments or implementation methods described in this specification are presented in a progressive manner. Each embodiment focuses on the differences from other embodiments, and the same or similar parts between the embodiments can be referred to each other.

[0086] Figure 5 This is a schematic diagram illustrating the structure of a model adaptation device for a streaming media playback terminal according to one embodiment. The streaming media playback terminal deploys at least two AI models, which are used to perform the same type of AI task. Different AI models correspond to different resource consumption levels, which characterize the degree of hardware resource consumption by the AI ​​model during operation. Figure 5 As shown, the model adaptation device 100 includes: The acquisition module 101 is used to acquire the currently available hardware resource information of the streaming media playback terminal during streaming media playback. The determining module 102 is used to select the AI ​​model with the highest matching degree from the at least two AI models as the target AI model based on the matching degree between the currently available hardware resource information and the resource consumption level corresponding to each of the at least two AI models; The loading module 103 is used to load the target AI model to perform the AI ​​task while maintaining uninterrupted streaming playback.

[0087] In some embodiments, the resource consumption level is divided according to at least one of the following parameters: peak memory usage, average power consumption, and model inference latency; when the resource consumption level is divided according to multiple parameters, each resource consumption level corresponds to a combination of numerical ranges of each parameter.

[0088] In some embodiments, the determining module 102 is configured to: Based on the resource consumption level corresponding to each AI model, the expected performance indicators of each AI model in the following multiple dimensions are obtained: device power consumption, memory usage, and model inference latency; Based on the remaining power, device temperature, memory usage, and processor load in the currently available hardware resource information, determine the corresponding demand weights for device power consumption, memory usage, and model inference latency, respectively. For each AI model, based on the expected performance indicators and corresponding demand weights of the AI ​​model in each dimension, a matching score is calculated between the resource consumption level of the AI ​​model and the currently available hardware resource information, wherein the matching score represents the degree of matching. The AI ​​model with the highest matching score is selected as the target AI model.

[0089] In some embodiments, the demand weight corresponding to the dimension has a preset mapping relationship with at least one parameter in the currently available hardware resource information; Among them, the remaining power is negatively correlated with the demand weight of the device power consumption dimension, the device temperature is positively correlated with the demand weight of the device power consumption dimension, the memory utilization rate is positively correlated with the demand weight of the memory utilization dimension, and the processor load is positively correlated with the demand weight of the model inference latency dimension.

[0090] In some embodiments, the apparatus further includes an adjustment module, the adjustment module being configured to: Based on the changing trends of the currently available hardware resources, the demand weights corresponding to each dimension are dynamically adjusted.

[0091] In some embodiments, the loading module is used for: Determine if a first AI model with the same task type as the target AI model has already been loaded; If the first AI model is not currently loaded, then the target AI model is loaded; If the first AI model is currently loaded, and the first AI model is the target AI model, then the loaded first AI model is retained. If the first AI model is currently loaded, and the first AI model is different from the target AI model, then it is determined whether the expected resource consumption of the target AI model exceeds the available resource limit for its corresponding task type. The available resource limit is determined based on the currently available hardware resource information. If the limit is exceeded, the first AI model that has already been loaded will be retained; If the limit is not exceeded, then load the target AI model and release the resources occupied by the first AI model.

[0092] In some embodiments, the device further includes an output module; The output module is used to output prompt information, which prompts the user to confirm whether to apply the target AI model; The loading module is used to execute the step of loading the target AI model in response to the user's confirmation operation, so as to perform the AI ​​task.

[0093] In some embodiments, the output module is used for: A floating prompt is displayed in a non-critical area of ​​the streaming media playback interface. The floating prompt contains an operation entry point for receiving user confirmation or rejection of the operation.

[0094] In some embodiments, the streaming media playback terminal currently has multiple AI tasks pending processing; The acquisition module is also used to: acquire a set of candidate AI models corresponding to each AI task to be processed, wherein different candidate AI models in each set of candidate AI models correspond to different resource consumption levels; The determining module is further configured to: determine the total resources available for all AI tasks based on the currently available hardware resource information; select a target AI model for each AI task from the candidate AI model set corresponding to each AI task, such that the sum of the expected resource consumption of all selected target AI models does not exceed the total resources, and the overall matching degree of all selected target AI models is optimal. The loading module is also used to load the target models corresponding to each AI task while maintaining uninterrupted streaming playback.

[0095] The model adaptation device provided in this application embodiment belongs to the same application concept as the model adaptation method for streaming media playback terminals provided in the above embodiments of this application. It can execute the model adaptation method for streaming media playback terminals provided in any of the above embodiments of this application, and has the corresponding functional modules and beneficial effects for executing the model adaptation method for streaming media playback terminals. Technical details not described in detail in this embodiment can be found in the specific implementation of the model adaptation method for streaming media playback terminals provided in the above embodiments of this application, and will not be repeated here.

[0096] This application also provides an electronic device, including a processor, a memory, and an executable program stored in the memory and executable by the processor. When the processor runs the executable program, it performs the steps of the model adaptation method for a streaming media playback terminal as provided in any of the foregoing embodiments.

[0097] This application also provides a computer-readable storage medium storing an executable program thereon, which, when executed by a processor, implements the steps of the model adaptation method for a streaming media playback terminal provided in any of the foregoing embodiments.

[0098] For ease of understanding, the following focuses on explaining the terminology used in this embodiment: In this application embodiment, the processor is a circuit with signal processing capabilities. In one implementation, the processor can be a circuit with instruction read and execute capabilities, such as a Central Processing Unit (CPU), a microprocessor, a Graphics Processing Unit (GPU) (which can be understood as a type of microprocessor), or a Digital Signal Processor (DSP). In another implementation, the processor can implement certain functions through the logical relationships of hardware circuits. The logical relationships of the aforementioned hardware circuits are fixed or reconfigurable. For example, the processor is a hardware circuit implemented using an Application-Specific Integrated Circuit (ASIC) or a Programmable Logic Device (PLD), such as an FPGA. In a reconfigurable hardware circuit, the process of the processor loading a configuration document and configuring the hardware circuit can be understood as the process of the processor loading instructions to implement the functions of some or all of the above units or modules. In addition, it can also be a hardware circuit designed for artificial intelligence, which can be understood as an ASIC, such as a Neural Network Processing Unit (NPU), a Tensor Processing Unit (TPU), a Deep Learning Processing Unit (DPU), etc.

[0099] The computer-readable storage medium provided in this embodiment can execute the model adaptation method for streaming media playback terminals described in the above embodiments. Its implementation principle and technical effects are similar to those in the above embodiments, and will not be repeated here.

[0100] The aforementioned computer-readable storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. The readable storage medium can be any available medium accessible to a general-purpose or special-purpose computer.

[0101] An exemplary readable storage medium is coupled to a processor, enabling the processor to read information from and write information to the readable storage medium. Of course, the readable storage medium can also be a component of the processor. The processor and the readable storage medium can reside in an Application Specific Integrated Circuit (ASIC). Alternatively, the processor and the readable storage medium can exist as discrete components in an electronic device or a host device.

[0102] Those skilled in the art will understand that all or part of the steps of the above-described method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When executed, the program performs the steps of the above-described method embodiments; and the aforementioned storage medium includes various media capable of storing program code, such as ROM, RAM, magnetic disks, or optical disks.

[0103] The various embodiments or implementation methods described in this specification are presented in a progressive manner. Each embodiment focuses on the differences from other embodiments, and the same or similar parts between the embodiments can be referred to each other.

[0104] In the description of this specification, references to "one embodiment," "some embodiments," "illustrative embodiment," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with an embodiment or example is included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0105] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application.

Claims

1. A model adaptation method for a streaming media playback terminal, characterized in that, The streaming media playback terminal is equipped with at least two AI models, which are used to perform the same type of AI task. Different AI models correspond to different resource consumption levels, whereby the resource consumption level characterizes the degree of hardware resource consumption during AI model operation. The method includes: During streaming media playback, obtain the currently available hardware resource information of the streaming media playback terminal; Based on the degree of matching between the currently available hardware resource information and the resource consumption level corresponding to each of the at least two AI models, the AI ​​model with the highest degree of matching is selected as the target AI model from the at least two AI models; While maintaining uninterrupted streaming playback, the target AI model is loaded to perform the AI ​​task.

2. The model adaptation method according to claim 1, characterized in that, The resource consumption level is defined based on at least one of the following parameters: peak memory usage, average power consumption, and model inference latency; when the resource consumption level is defined based on multiple parameters, each resource consumption level corresponds to a combination of numerical ranges for each parameter.

3. The model adaptation method according to claim 1, characterized in that, Based on the degree of matching between the currently available hardware resource information and the resource consumption levels corresponding to the at least two AI models, the AI ​​model with the highest degree of matching is selected as the target AI model from the at least two AI models, including: Based on the resource consumption level corresponding to each AI model, the expected performance indicators of each AI model in the following multiple dimensions are obtained: device power consumption, memory usage, and model inference latency; Based on the remaining power, device temperature, memory usage, and processor load in the currently available hardware resource information, determine the corresponding demand weights for device power consumption, memory usage, and model inference latency, respectively. For each AI model, based on the expected performance indicators and corresponding demand weights of the AI ​​model in each dimension, a matching score is calculated between the resource consumption level of the AI ​​model and the currently available hardware resource information, wherein the matching score represents the degree of matching. The AI ​​model with the highest matching score is selected as the target AI model.

4. The model adaptation method according to claim 3, characterized in that, There is a preset mapping relationship between the demand weight corresponding to the dimension and at least one parameter in the currently available hardware resource information; Among them, the remaining power is negatively correlated with the demand weight of the device power consumption dimension, the device temperature is positively correlated with the demand weight of the device power consumption dimension, the memory utilization rate is positively correlated with the demand weight of the memory utilization dimension, and the processor load is positively correlated with the demand weight of the model inference latency dimension.

5. The model adaptation method according to claim 3, characterized in that, The method further includes: Based on the changing trends of the currently available hardware resources, the demand weights corresponding to each dimension are dynamically adjusted.

6. The model adaptation method according to claim 1, characterized in that, The step of loading the target AI model to execute the AI ​​task while maintaining uninterrupted streaming playback includes: Determine if a first AI model with the same task type as the target AI model has already been loaded; If the first AI model is not currently loaded, then the target AI model is loaded; If the first AI model is currently loaded, and the first AI model is the target AI model, then the loaded first AI model is retained. If the first AI model is currently loaded, and the first AI model is different from the target AI model, then it is determined whether the expected resource consumption of the target AI model exceeds the available resource limit for its corresponding task type. The available resource limit is determined based on the currently available hardware resource information. If the limit is exceeded, the first AI model that has already been loaded will be retained; If the limit is not exceeded, then load the target AI model and release the resources occupied by the first AI model.

7. The model adaptation method according to any one of claims 1 to 6, characterized in that, The method further includes: Output a prompt message, which prompts the user to confirm whether to apply the target AI model; In response to the user's confirmation, the step of loading the target AI model to perform the AI ​​task is executed.

8. The model adaptation method according to claim 7, characterized in that, The output prompt information includes: A floating prompt is displayed in a non-critical area of ​​the streaming media playback interface. The floating prompt contains an operation entry point for receiving user confirmation or rejection of the operation.

9. The model adaptation method according to claim 1, characterized in that, The streaming media playback terminal currently has multiple AI tasks awaiting processing, and the method further includes: Obtain available model information corresponding to each AI task to be processed, wherein at least one of the AI ​​tasks to be processed corresponds to a candidate set containing at least two candidate models; Based on the currently available hardware resource information, determine the total available resources that can be used by the multiple AI tasks to be processed; Under the constraint of the total available resources, AI target models are determined for the multiple AI tasks to be processed, such that the sum of the expected resource consumption of each target AI model determined to be used does not exceed the total available resources. While maintaining uninterrupted streaming playback, each of the determined AI target models is loaded separately.

10. A model adaptation device for a streaming media playback terminal, characterized in that, The streaming media playback terminal is equipped with at least two AI models, which are used to perform the same type of AI task. Different AI models correspond to different resource consumption levels, which characterize the degree of hardware resource consumption by the AI ​​model during operation. The device includes: The acquisition module is used to acquire the currently available hardware resource information of the streaming media playback terminal during streaming media playback. The determination module is used to select the AI ​​model with the highest matching degree from the at least two AI models as the target AI model based on the matching degree between the currently available hardware resource information and the resource consumption level corresponding to each of the at least two AI models; A loading module is used to load the target AI model to perform the AI ​​task while maintaining uninterrupted streaming playback.

11. A computer-readable storage medium, characterized in that, It stores an executable program, which, when executed by a processor, implements the model adaptation method for a streaming media playback terminal as described in any one of claims 1 to 9.

Images