Optimized control of elements of semiconductor devices utilizing multimodal machine-learned models

Abstract

Aspects of optimized control of elements of semiconductor devices utilizing multimodal machine-learned models are disclosed. An optimizer unit may optimize the operation of elements of a semiconductor device utilizing a multimodal machine-learned model. The optimizer unit may determine optimal operations of element(s) of the semiconductor device based on a maximum performance state of the element(s), a minimum performance state of the element(s), and a performance residency state of the element(s). The maximum performance state may be determined based on a thermal control policy and thermal aspects of the element(s) as well as thermal aspects of an enclosure of the semiconductor device. The minimum performance state may be determined based on a latency policy and latency of the element(s). The maximum performance state may ensure thermal control of the element(s) and the minimum performance state may ensure a maximum latency of the element(s).

Description

OPTIMIZED CONTROL OF ELEMENTS OF SEMICONDUCTOR DEVICES UTILIZING MULTIMODAL MACHINE-LEARNED MODELSBACKGROUND

[0001] Semiconductor devices are widely used throughout the world in various electronic devices. It is estimated that almost 80% of the world's population owns a mobile phone, which is one type of electronic device. One semiconductor device used within electronic devices is a system-on-a-chip (SoC). which may include various elements, such as a central processing unit (CPU), a graphic processing unit (GPU), an accelerated processing unit (APU), an audio processing unit, and a tensor processing unit (TPU).

[0002] The various elements of a semiconductor device (e.g., CPU, GPU) within an electronic device may cause a rise in temperature within the semiconductor device, which may apply stress or reduce performance of the semiconductor device when the temperature is too high. Electronic devices may include fixed thermal control policies that are applied to elements of the semiconductor device. Electronic devices may include other fixed control policies such as latency policies and / or pow er consumption policies. Fixed control policies may be applied to one or more elements of a semiconductor device in an effort to provide adequate performance while also controlling elements to prevent potential thermal issues. However, fixed control policies may not provide the optimal performance of an electronic device that includes the semiconductor device. For example, an electronic device may employ a multimodal machine-learned model, which may consume a large amount of resources of an SoC of the electronic device. Thus, a fixed thermal control policy may not be adequate to prevent a thermal issue. Likewise, a fixed latency control policy if too severe may provide a poor user experience due to delays in the multimodal machine-learned model providing results to a user. Further, a fixed powder consumption policy may not adequately prevent the multimodal machine-learned model from draining a battery of the electronic device.SUMMARY

[0003] This document describes systems and techniques directed at optimized control of elements of semiconductor devices utilizing multimodal machine-learned models, which may overcome or reduce the disadvantages of applying fixed thermal and latency control policies. For example, an optimizer unit may be configured to optimize the operations of elements for a semiconductor device utilizing a multimodal machine-learned model based on control policies of the semiconductor device, performance state residency of the elements, and feedback (e.g., operational information) from the elements.

[0004] Aspects of optimized control of elements of semiconductor devices utilizing multimodal machine-learned models are disclosed. For example, an optimizer unit may optimize the operation of one or more elements of a semiconductor device utilizing a multimodal machine-learned model. The optimizer unit may determine optimal operations of an element based on a maximum performance state of the element, a minimum performance state of the element, and a performance residency state of the element. The maximum performance state may be determined based on a thermal control policy, thermal aspects (e.g., temperature) of the element, and thermal aspects (e.g., temperature) of an enclosure of the semiconductor device. The minimum performance state may be determined based on a latency policy and latency of the element. The maximum performance state may ensure thermal control of the element of the semiconductor device and the minimum performance state may ensure a maximum latency of the element of the semiconductor device.

[0005] In some aspects, the techniques described herein relate to a method that includes receiving, at an optimizer unit, a first performance state residency of a first element of a semiconductor device, and the first performance state residency indicates a performance history' of the first element. The method includes receiving, at the optimizer unit, a first maximum performance state of the first element from a thermal governor, the first maximum performance state determined based on a first thermal target, a first temperature of the first element, and anenclosure temperature. The method includes receiving, at the optimizer unit, a first minimum performance state of the first element from a latency governor, the first minimum performance state determined based on a first latency target and a first latency of the first element.

[0006] The method includes determining, by the optimizer unit, first operating parameters of the first element, the first operating parameters determined based on the first performance state residency of the first element, the first maximum performance state, and the first minimum performance state. The method includes applying the first operating parameters to the first element, the applying effective to transition the first element to the first operating parameters.

[0007] In some aspects, the techniques described herein relate to a system that includes a first element of a semiconductor device, the first element having a first temperature and a first performance state. The system includes a latency calculator coupled with the first element. The first element provides the first performance state to the latency calculator and the latency¬ calculator is configured to determine a first latency of the first element based on the first performance state. The system includes a policy manager coupled with the first element. The first element provides the first performance state of the policy manager. The policy manager is configured to determine a first thermal target based on a thermal policy of the semiconductor device and the first performance state of the first element and to determine a first latency target based on a latency policy of the semiconductor device and the first performance state of the first element.

[0008] The system includes an enclosure temperature sensor coupled with a thermal governor. The enclosure temperature sensor is configured to provide an enclosure temperature to the thermal governor. The thermal governor is coupled with the policy manager and the first element. The first element is configured to provide the first temperature to the thermal governor and the thermal governor is configured to determine a first maximum performance state of the first element, the first maximum performance state of the first element determined based on thefirst thermal target provided by the policy manager, the first temperature of the first element, and the enclosure temperature.

[0009] The system includes a latency governor coupled with the latency calculator and the policy manager. The latency governor is configured to determine a first minimum performance state of the first element, the first minimum performance state of the first element determined based on the first latency target provided by the policy manager and the first latency of the first element provided by the latency calculator. The system includes an optimizer unit coupled with the thermal governor, the latency governor, and the first element. The optimizer unit is configured to determine and apply first operating parameters of the first element. The first operating parameters are determined based on a first performance state residency of the first element, the first maximum performance state, and the first minimum performance state, the first performance state residency of the first element indicating a performance history of the first element.

[0010] This Summary is provided to introduce simplified concepts of optimized control of elements of semiconductor devices utilizing multimodal machine-learned models, the concepts of which are further described below in the Detailed Description and Drawings. This Summary' is not intended to identify essential features of the claimed subject matter, nor is it intended for use in determining the scope of the claimed subject matter.BRIEF DESCRIPTION OF THE DRAWINGS

[0011] The details of one or more aspects of systems and techniques directed at optimized control of elements of semiconductor devices utilizing multimodal machine-learned models are described in this document with reference to the following drawings, in which the use of same numbers in different instances may indicate similar features or components.

[0012] FIG. 1 illustrates an example system in which aspects of optimized control of elements of semiconductor devices utilizing multimodal machine-learned models can be implemented.

[0013] FIG. 2 illustrates an example system in which aspects of optimized control of elements of semiconductor devices utilizing multimodal machine-learned models can be implemented.

[0014] FIG. 3 illustrates an example controller framework that may be implemented within an optimized controller for semiconductor devices utilizing multimodal machine-learned models.

[0015] FIG. 4 illustrates an example system in which aspects of optimized control of elements of semiconductor devices utilizing multimodal machine-learned models can be implemented.

[0016] FIG. 5 illustrates an example operating environment in which aspects of optimized control of elements of semiconductor devices utilizing multimodal machine-learned models can be implemented.

[0017] FIG. 6 illustrates an integrated circuit component in which aspects of optimized control of elements of semiconductor devices utilizing multimodal machine-learned models can be implemented.

[0018] FIG. 7 illustrates an example electronic device having internal hardware configurations for optimized control of elements of semiconductor devices utilizing multimodal machine-learned models in accordance with one or more implementations.

[0019] FIG. 8 is a flowchart of a method for optimized control of elements of semiconductor devices utilizing multimodal machine-learned models.DETAILED DESCRIPTIONOverview

[0020] Electronic devices may include one or more semiconductor devices that apply fixed control policies (e.g., thermal control policies, latency control policies, power consumption control policies) to the various elements of the semiconductor device. The fixed control policiesmay be implemented to prevent the operation of an element from creating thermal issues (e.g., overheating), latency issues, and / or power consumption issues (e.g., depletion of a battery) of the electronic device. The electronic device may utilize a multimodal machine-learned model, which consumes resources of the electronic device. A fixed latency policy may cause the multimodal machine-learned model to continuously, or almost continuously, operate on the electronic device to ensure that a user does not experience a delay (e.g., latency, lag) when accessing the multimodal machine-learned model. However, the continuous, or almost continuous, operating of the multimodal machine-learned model may potentially cause thennal issues and / or power consumption issues.

[0021] Conversely, the application of a fixed thermal control policy and / or a fixed power consumption policy may cause the semiconductor device within the electronic device to cause the multimodal machine-learned model to transition to a standby mode or even to shut down when not actively engaged by the electronic device to conserve power and / or prevent thermal issues. This may lead to unacceptable latency when the multimodal machine-learned model is once again accessed. Fixed control policies are unable to optimize the various elements of the semiconductor device within an electronic device to provide an optimal user experience.

[0022] To this end, this document describes systems and techniques directed at optimized control of elements of semiconductor devices utilizing multimodal machine-learned models. An optimizer unit may be configured to optimize the operation of elements of a semiconductor device utilizing a multimodal machine-learned model. Elements of the semiconductor device communicate operational aspects (e.g., temperature, operating state) to a policy manager and a latency calculator. The policy manager determines thermal targets and latency targets based on the operational aspects in view of control policies of the semiconductor device, and the latency calculator determines latency of the elements.

[0023] A thermal governor may determine a maximum performance state of the elements of the semiconductor device based on the thermal targets, the temperatures of the elements, andan enclosure temperature. A latency governor may determine a minimum performance state of the elements of the semiconductor device based on the latency targets and the latencies of the elements. An optimizer unit determines the optimal performance of the elements of the semiconductor device based on the maximum performance state, the minimum performance state, and performance state residencies of the elements. The optimizer unit then applies the optimal performance state to control the operation of each element to optimize performance of the elements of the semiconductor device utilizing the multimodal machine-learned model.

[0024] The following discussion describes operating environments, techniques that may be employed in the operating environments, and example methods. Although techniques using and apparatuses for an adaptive controller scheduling framework for semiconductor devices are described, it is to be understood that the subject of the appended claims is not necessarily limited to the specific features or methods described. Rather, the specific features and methods are disclosed as example implementations and reference is made to the operating environment by way of example only.Example Systems and Operational Schemes

[0025] The operation of elements of a semiconductor device utilizing a multimodal machine-learned model may be optimized for the usage of the multimodal machine-learned model. The optimizer unit may be configured to optimize the operation of the various elements based on control policies of the semiconductor device, feedback (e.g., information about current operational aspects) from the various elements, and performance history of the various elements.

[0026] FIG. 1 illustrates an example system 100 in which aspects of optimized control of elements of semiconductor devices utilizing multimodal machine-learned models can be implemented. The system 100 includes an element 102 of a semiconductor device. The element 102 may be any operational component of the semiconductor device. For example, the element 102 may be a component that may increase in temperature due to operation of the element 102.For example, the element 102 may be a processor that may increase in temperature during processing. The processor may be a CPU, a GPU, a TPU, or the like. The element 102 may instead be anon-processing element, such as a sensor or display. For example, the sensor may be an optical sensor, radar sensor, proximity sensor, or the like. The element 102 may be any component of a semiconductor device that consumes power, has a latency, has a performance state residency, or the like, and / or has a temperature that may rise due to use, continued use, and / or repeated cycling of the component. The element 102 may be configured to communicate a present, or cunent. temperature 104 and / or present, or current, performance state 106 to various other components of the system 100.

[0027] The element 102 is coupled with a policy manager 108 of the system 100. The performance state 106 of the element 102 is communicated to both the policy manager 108 and the latency calculator 114. The policy manager 108 includes various control policies of the semiconductor device. For example, the policy manager 108 may include a themial control policy to be applied to various components (e.g., element 102) of the semiconductor device and a latency control policy to be applied to various components (e.g., element 102) of the semiconductor device. The policy manager 108 may include other control policies such as, for example, a power consumption policy to be applied to various components (e.g., element 102) of the semiconductor device as would be appreciated by one of ordinary7skill in the art having the benefit of this disclosure.

[0028] The policy manager 108 is configured to determine athermal target 110 based on the thermal control policy of the semiconductor device and the performance state 106 of the element 102. The policy manager 108 is also configured to determine a latency target 112 based on the latency control policy of the semiconductor device and the performance state 106 of the element 102. The policy manager 108 may be one or more processing units programmed to utilize the control policies of the semiconductor device to generate various targets for controlling the operation of various components of a semiconductor device as discussed herein. In another aspect,the policy manager 108 may be one or more controllers that utilize control policies and properties of one or more components of a semiconductor device to generate targets (e.g., thresholds) of various operating properties of the components of the semiconductor device. The policy manager 108 may be one or more logic devices configured to determine targets (e.g.. thresholds) of various operating properties of a component of the semiconductor device based on control policies of the semiconductor device and the operating state of the component.

[0029] The element 102 is also coupled with a latency calculator 114. The performance state 106 of the element 102 is also communicated to the latency calculator 114. The latency calculator 114 is configured to determine a present, or current, latency 116 of the element 102 based on the performance state 106. The latency calculator 114 may be one or more processing units programmed to determine a latency of a component of the semiconductor device based on a performance state of the component. In another aspect, the latency calculator 114 may be one or more controllers that utilize a performance state of a component to determine latency. In some implementations, the latency calculator 114 may be one or more logic devices configured to determine a latency of a component based on the operating state of the component.

[0030] The system 100 includes a thermal governor 118 coupled with the policy manager 108. The thermal governor 118 is coupled with an enclosure temperature sensor 120 that provides a current, or present, temperature 122 of an enclosure of the semiconductor device. For example, the enclosure may be a housing of a mobile electronic device such as a mobile phone. The thermal governor 118 is also coupled with the element 102 of the semiconductor device and receives the temperature 104 of the element 102. The thermal governor 118 is configured to determine a maximum performance state 124 of the element 102. The maximum performance state 124 is determined based on the enclosure temperature 122 provided by the enclosure temperature sensor 120, the temperature 104 of the element 102, and the thermal target 110 provided to the thermal governor 118 by the policy manager 108. The maximum performance state 124 may indicate a ceiling performance state for the element 102 to ensure the temperature 104 of the element 102does not exceed the thermal target 110 provided by the policy manager 108. The ceiling performance state for the element 102 may be calculated so that the temperature 104 of the element 102 does not exceed a threshold instantaneous temperature or a temperature threshold for a timescale for the element 102. For example, the threshold may be set for an average temperature 104 of the element 102 over a specified period of time as would be appreciated by one of ordinary skill in the art having the benefit of this disclosure.

[0031] The thermal governor 118 may be one or more processing units configured to determine the maximum performance state 124. In another implementation, the thermal governor 118 may be one or more thermal controllers programmed to determine the maximum performance state 124 based on thermal aspects of a component of the semiconductor device. The thermal governor 118 may include one or more thermal controllers configured to control the thermal aspects of the semiconductor device. The thermal governor 118 may be one or more logic units configured to generate the maximum performance state 124 based on received inputs from the policy manager 108, the element 102, and the enclosure temperature sensor 120.

[0032] The system 100 includes a latency governor 126 coupled with the latency calculator 114 and the policy manager 108. The latency governor 126 receives the latency target 112 from the policy manager 108 and the latency 116 of the element 102 from the latency calculator 114. The latency governor 126 is configured to determine a minimum performance state 128 of the element 102. The minimum performance state 128 is determined based on the latency 116 provided by the latency calculator 114 and the latency target 112 provided to the latency governor 126 by the policy manager 108. The minimum performance state 128 may indicate a floor performance state for the element 102 to ensure the latency 116 of the element 102 does not exceed the latency target 112 provided by the policy manager 108. The floor performance state for the element 102 may be calculated so that the latency 116 of the element 102 does not exceed the maximum allowed latency, according to the latency policies of the semiconductor device, while the semiconductor device is utilizing a multimodal machine-learnedmodel or another program. The latency target 112 may be calculated based on instantaneous latency or average latency over a specified period of time as would be appreciated by one of ordinary skill in the art having the benefit of this disclosure.

[0033] The latency governor 126 may be one or more processing units configured to determine the minimum performance state 128 to ensure latency of a component of the semiconductor device does not exceed a maximum latency set forth in latency control policies of the semiconductor device. In another implementation, the latency governor 126 may be one or more controllers programmed to detennine the minimum performance state 128 based on latency of a component of the semiconductor device. The latency governor 126 may include one or more controllers configured to control latency of the semiconductor device. The latency governor 126 may be one or more logic units configured to generate the minimum performance state 128 based on received inputs from the policy manager 108 and the latency calculator 114.

[0034] The system 100 includes an optimizer unit 130 coupled with the thermal governor 118, the latency governor 126, and the element 102 of the semiconductor device. The optimizer unit 130 receives the maximum performance state 124 from the thermal governor 118 and the minimum performance state 128 from the latency governor 126. The optimizer unit 130 may be configured to determine the optimal performance state of the element 102 based on the received maximum performance state 124 and received minimum performance state 128. The optimizer unit 130 applies operating parameters 132 to the element 102. The operating parameters 132 applied to the element 102 are effective to transition the operation of the element 102 to the operating parameters 132. In one implementation, the optimizer unit 130 may apply the operating parameters 132 to one or more controllers configured to control the operation of the element 102 as would be appreciated by one of ordinary skill in the art having the benefit of this disclosure. The optimizer unit 130 is configured to optimize the performance of the element 102 based on the control policies (e.g., thermal control policy, latency control policy, power consumption policy, and the like) of the semiconductor device to ensure the operation of the element 102 is optimizedto utilize programs, such as a multimodal machine-learned model, without violating the control policies of the semiconductor device.

[0035] The optimizer unit 130 may also receive a performance state residency 134 of the element 102. The performance state residency 134 of the element 102 is a history of the performance state 106 of the element 102. A machine-learned model may determine the history’ of the performance state 106 of the element 102. For example, the machine-learned model may be coupled with the element 102 and determine the history’ of the performance state 106 of the element 102 by monitoring the performance state 106 over a specified time period. In some implementations, the machine-learned model may monitor the element 102 continuously or may monitor the element 102 at specified intervals during a specified time period. The performance state residency 134 may indicate that the performance of the element 102 may warrant exceeding the maximum performance state 124 determined by the thermal governor 118. For example, the performance state residency 134 (e.g., performance state history) of the element 102 may indicate that the element 102 has not exceeded a thermal threshold in the past. Thus, it may be warranted to increase the operating parameters 132 for the element 102 to exceed the determined maximum performance state 124.

[0036] The optimizer unit 130 may be one or more processing units configured to determine the optimal performance of a component of a semiconductor device to ensure the latency of a component of the semiconductor device does not violate the various control policies of the semiconductor device. In another implementation, the optimizer unit 130 may be one or more controllers programmed to determine the optimal performance state of a component of the semiconductor device based on the inputs received from the thermal governor 118, the latency governor 126, and the performance state residency 134. In an aspect, the optimizer unit 130 may be one or more logic units configured to optimize the performance of the element 102. In these ways, the system 100 is configured to ensure that the operation of the element 102 is controlled and optimized in real time based on control policies of the semiconductor device for the operationof various programs such as a multimodal machine-learned model. FIG. 2 illustrates an example system that controls and optimizes the operation of components of a semiconductor device for the operation of a multimodal machine-learned model.

[0037] FIG. 2 illustrates an example system 200 in which aspects of optimized control of elements of semiconductor devices utilizing multimodal machine-learned models can be implemented. The system 200 includes a first element 102-1 and a second element 102-2 of a semiconductor device. As discussed above, the first element 102-1 and the second element 102-2 may be any operational component of the semiconductor device. For example, the element 102 may be a processor such as a CPU, GPU, TPU. or the like. The first element 102-1 may be configured to communicate various operating parameters (e.g., a first temperature 104-1. a performance state 106-1) to various other components of the system 200. Likewise, the second element 102-2 may be configured to communicate various operating parameters (e.g., a second temperature 104-2, a second performance state 106-2) to various other components of the system 200.

[0038] A policy manager 108 and a latency calculator 114 are each coupled with the first element 102-1 and the second element 102-2, and the first performance state 106-1 and the second performance state 106-2 are communicated to both the policy manager 108 and the latency calculator 114. The policy manager 108 includes various control policies of the semiconductor device. For example, the policy manager 108 may include athermal control policy and a latency control policy to be applied to various components (e.g., first element 102-1, second element 102-2) of the semiconductor device. The policy manager 108 may include other control policies such as, for example, a power consumption policy to be applied to various components (e.g., first element 102-1, second element 102-2) of the semiconductor device.

[0039] The policy manager 108 is configured to determine a first thermal target 110-1 for the first element 102-1 and a second thermal target 110-2 for the second element 102-2. The first thermal target 110-1 and the second thermal target 110-2 are each based on the thermal controlpolicy of the semiconductor device and the performance states (e.g., first performance state 106-1 , second performance state 106-2) of the elements (e.g., first element 102-1, second element 102-2) of the semiconductor device. The first element 102-1 and the second element 102-2 are shown for illustrative purposes, and the policy manager 108 may be coupled with more or fewer elements of the semiconductor device as would be appreciated by one of ordinary skill having the benefit of this disclosure.

[0040] The policy manager 108 may also be configured to determine a first latency target 112-1 for the first element 102-1 and a second latency target 112-2 for the second element 102-2. As discussed above, the latency targets (e.g., first latency target 112-1, second latency target 112-2) are based on the latency control policy of the semiconductor device and the performance states (e.g.. first perfomiance state 106-1, second performance state 106-2) of the elements (e.g., first element 102-1, second element 102-2).

[0041] The latency calculator 114 is configured to determine a first latency 116-1 of the first element 102-1 based on the first performance state 106-1. Likewise, the latency calculator 114 is configured to determine a second latency 116-2 of the second element 102-2 based on the second performance state 106-2.

[0042] The system 200 includes a thermal governor 118 coupled with the policy manager 108. The thermal governor 118 is also coupled with an enclosure temperature sensor 120 that provides a current, or present, temperature 122 of an enclosure of the semiconductor device. The thermal governor 118 is also coupled with the first element 102-1 and the second element 102-2 of the semiconductor device. The first element 102-1 sends the first temperature 104-1 to the thermal governor 118. Likewise, the second element 102-2 sends the second temperature 104-2 to the thermal governor 118. The thermal governor 118 is configured to determine a first maximum performance state 124-1 of the first element 102-1 determined based on the enclosure temperature 122 provided by the enclosure temperature sensor 120, the first temperature 104-1 of the first element 102-1, and the first thermal target 110-1 provided to the thermal governor 118 bythe policy manager 108. Likewise, the thermal governor 118 is configured to determine a second maximum performance state 124-2 of the second element 102-2 determined based on the enclosure temperature 122 provided by the enclosure temperature sensor 120, the second temperature 104-2 of the second element 102-2. and the second thermal target 110-2 provided to the thermal governor 118 by the policy manager 108.

[0043] The system 200 includes a latency governor 126 coupled with the latency calculator 114 and the policy manager 108. The latency governor 126 receives the first latency target 112-1 from the policy manager 108 and the first latency 116-1 of the first element 102-1 from the latency calculator 114. Likewise, the latency governor 126 receives the second latency target 112-2 from the policy manager 108 and the second latency 116-2 of the second element 102-2 from the latency calculator 114.

[0044] The latency governor 126 is configured to determine a first minimum performance state 128-1 of the first element 102-1 and a second minimum perfomiance state 128-2 of the second element 102-2. The first minimum performance state 128-1 is determined based on the first latency 116-1 provided by the latency calculator 114 and the first latency target 112-1 provided to the latency governor 126 by the policy manager 108. The second minimum performance state 128-2 is determined based on the second latency 116-2 provided by the latency calculator 114 and the second latency target 112-2 provided to the latency governor 126 by the policy manager 108.

[0045] The system 200 includes an optimizer unit 130 coupled with the thermal governor 118, the latency governor 126, the first element 102-1, and the second element 102-2. The optimizer unit 130 receives the first maximum performance state 124-1 and second maximum performance state 124-2 from the thermal governor 118 and the first minimum performance state 128-1 and second minimum performance state 128-2 from the latency governor 126. The optimizer unit 130 may be configured to determine the optimal performance state of the first element 102-1 based on the received first maximum performance state 124-1 and the received firstminimum performance state 128-1. Likewise, the optimizer unit 130 may be configured to determine the optimal performance state of the second element 102-2 based on the received second maximum performance state 124-2 and the received second minimum performance state 128-2.

[0046] Upon determining the optimal performance state of the first element 102-1, the optimizer unit 130 applies first operating parameters 132-1 to the first element 102-1. The optimizer unit 130 applies second operating parameters 132-2 to the second element 102-2 upon determining the optimal performance state of the second element 102-2. The first and second operating parameters 132-1, 132-2 applied to the first element 102-1 and the second element 102-2 are effective to transition the operation of the first element 102-1 to the first operating parameters 132-1 and the operation of the second element 102-2 to the second operating parameters 132-2. As discussed above, the optimizer unit 130 is configured to optimize the performance of the elements (e.g., first element 102-1, second element 102-2) of the semiconductor device based on the control policies (e.g., thermal control policy, latency control policy, power consumption policy, and the like) of the semiconductor device to ensure that the elements (e.g., first element 102-1, second element 102-2) are optimized to utilizes programs, such as a multimodal machine-learned model, without violating the control policies of the semiconductor device.

[0047] The optimizer unit 130 may also receive a first performance state residency 134-1 of the first element 102-1 and a second performance state residency 134-2 of the second element 102-2. A machine-learned model may determine a history' of the performance states 106-1, 106-2 of the elements (e.g., first element 102-1, second element 102-2). The performance state residencies (e.g., first performance state residency 134-1, second performance state residency 134-2) may indicate that the performance of one or more of the elements (e.g., first element 102-1, second element 102-2) may warrant exceeding the maximum performance states (e.g., first maximum performance state 124-1, second maximum performance state 124-2) determined by the thermal governor 118 as discussed above. The systems (e.g., system 100, system 200) disclosed herein may include one or more controllers to control the elements (e.g., element 102,first element 102-1, second element 102-2) of the semiconductor device. Likewise, a component (e.g., thermal governor 118) of the systems (e.g., system 100, system 200) may include one or more controllers. The controller may be a controller framework 300 as shown in FIG. 3.

[0048] FIG. 3 illustrates an example controller framework 300 that may be implemented within a system for semiconductor devices utilizing multimodal machine-learned models. The controller framework 300 includes a controller 302 coupled to an element 102 of a semiconductor device. The controller 302 may be coupled to one or more elements 102 and may be configured to control various operational aspects of the element 102 as would be appreciated by one of ordinary skill in the art having the benefit of this disclosure. For example, the controller 302 may be a thermal controller configured to control a temperature of the element 102.

[0049] The controller framework 300 includes a policy target 304 coupled with the controller 302. The policy target 304 may include various targets based on control policies of the semiconductor device. For example, the policy target 304 may include a thermal target based on a thermal policy or a power consumption target based on a power consumption target. Controller parameters 306 may be coupled with the controller 302. The controller parameters 306 may be configured to provide operating limits to the controller 302. A filter 308 may be coupled with the element 102 to provide feedback (e.g., temperature, latency, power consumption) from the element 102 to the controller 302. Filter parameters 310 may be applied to the filter 308 to determine the feedback provided to the controller 302 by the filter 308. The controller framework 300 may be used in systems that control and optimize the operation of components of a semiconductor device for the operation of a multimodal machine-learned model. FIG. 4 illustrates a system 400 that may be used in systems that control and optimize the operation of components of a semiconductor device for the operation of a multimodal machine-learned model.

[0050] FIG. 4 illustrates an example system 400 in which aspects of optimized control of elements of semiconductor devices utilizing multimodal machine-learned models can be implemented. Similarly to the system 100 of FIG. 1, an element 102 provides a temperature 104to a thermal governor 118 and a performance state 106 to a policy manager 108 and a latency calculator 114. The element 102 may also provide different operation parameters to the policy manager 108 and the latency calculator 114. For example, the element 102 may indicate a current, or present, power consumption 402. The policy manager 108 may indicate a power consumption target 404 to a power consumption governor 406 based on power consumption policies of the semiconductor device. The power consumption governor 406 may provide a power perfonnance state 408 of the first element 102 to an optimizer unit 130.

[0051] As discussed herein, the thermal governor 118 provides a maximum performance state 124 of the element 102 based on a thermal target 110 provided by the policy manager 108, an enclosure temperature 122 provided by an enclosure temperature sensor 120, and the temperature 104 of the element 102. Likewise, a latency governor 126 provides a minimum performance state 128 of the element 102 based on a latency target 112 provided by the policy manager 108 and latency 116, provided by the latency calculator 114, of the element 102. The optimizer unit 130 determines operating parameters 532 to be applied to the element 102 based on the received power performance state 408, the received maximum perfonnance state 124, the received minimum perfonnance state 128, and a received performance state residency 134 of the element 102. The performance state residency 134 may be received by a machine-learned model as discussed above. The operating parameters 532 are applied to the element 102 to optimize the operation of the element 102. For example, the operation of the element 102 may be optimized to utilize a multimodal machine-learned model as discussed herein rather than just relying on fixed operational targets.Example Environments and Electronic Devices

[0052] FIG. 5 illustrates an example operating environment 500 in which aspects of optimized control of elements of semiconductor devices utilizing multimodal machine-learned models can be implemented. As illustrated, an SoC integrated circuit (IC) device 502 may bemounted to a printed circuit board (PCB) 504, which may be included as part of a computing device that implements one or more security protocols. As non-hmiting examples, the computing device may be a smartphone 506, a personal digital assistant 508, a tablet 510, a laptop 512, or a workstation 514.

[0053] The SoC IC device 502 may include various elements 102 (e.g., GPU. CPU. TPU) that may cause a temperature event (e.g.. a sudden increase in temperature) within the SoC IC device 502 due to repeated and / or continued use. For example, a user may repeatedly launch, use, and cancel an application on an electronic device that utilizes the SoC IC device 502. Uikewise, various elements 102 (e.g., GPU, CPU, TPU) may experience lag or latency while using a multimodal machine-learned model or other programs. The SoC IC device 502 may include one or more policy managers 108, latency calculators 114, thermal governors 118, latency governors 126, and optimizer units 130. The thermal governors 118 may include one or more thermal controllers configured to thermally control the one or more elements 102. The one or more policy managers 108, latency calculators 114, thermal governors 118, latency governors 126, and optimizer units 130 may be configured to optimize the one or more elements 102 according to various control policies of the semiconductor device.

[0054] The elements 102 provide operational information to various components. For example, the elements 102 may provide temperatures 104 to the thermal governors 118 and performance states 106 of the elements 102 to the policy managers 108 and latency calculators 114. Based on control policies of the semiconductor device and operational information from the elements 102, the policy managers 108 indicate thermal targets to the thermal governors 118 and latency targets to the latency governors 126. Based on the operational information from the elements 102, the latency calculators 114 provides latency of the elements 102 to the latency governors 126.

[0055] The thermal governors 118 provide maximum performance states of the elements 102 to the optimizer units 130 and the latency governors 126 provide minimum performance statesof the elements 102 to the optimizer units 130. The optimizer units 130 determine optimal operating parameters of the elements 102 based on the received maximum performance states, the received minimum performance states, and received performance state residencies 134 of the elements. The optimal operating parameters are applied by the optimizer units 130 to the elements 102 to transition the elements 102 to the optimal operating parameters.

[0056] Although the SoC IC device 502 is described in the context of a single SoC IC device including the element(s) 102, policy manager(s) 108. latency calculator(s) 114, thermal govemor(s) 118, latency govemor(s) 126, and optimizer unit(s) 130, a combination of discrete IC devices may perform the same functions. For example, a discrete processor IC device (e.g., element(s) 102, policy manager(s) 108, latency calculator(s) 114, thermal govemor(s) 118, latency govemor(s) 126, and optimizer unit(s) 130) may work in combination with a discrete non-volatile memory IC device having the elements to perform one or more functions described herein.

[0057] FIG. 6 illustrates an integrated circuit component implemented as an SoC 600 that can implement various aspects of an adaptive controller scheduling framework for semiconductor devices. The SoC 600 may be a single chip including components that are fabricated on the same semiconductor substrate. Alternatively, the SoC 600 may be a number of such chips that are epoxied together. The SoC 600 can be implemented in any suitable device, such as a smartphone, a cellular phone, a netbook, a tablet computer, a server, a wireless router, a network-attached storage, a camera, a smart appliance, a printer, a set-top box, or any other suitable type of device. Although described with reference to an SoC, the entities of FIG. 6 may also be implemented as an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or the like.

[0058] The SoC 600 can be integrated with electronic circuitry, including the components described in the operating system listed herein. The SoC 600 can also include an integrated data bus (not shown) that couples the various components of the SoC 600 for data communication between the components. The integrated data bus or other components of the SoC 600 may beexposed or accessed through an external port, such as a joint test action group (JTAG) port. For example, components of the SoC 600 may be tested, configured, or programmed (e.g., flashed) through the external port at different stages of manufacture.

[0059] In this example, the SoC 600 includes computer-readable media 602, one or more processors 604, element(s) 102, policy manager(s) 108. latency calculator(s) 114, thermal govemor(s) 118, latency govemor(s) 126, optimizer unit(s) 130. and I / O units 606. The optimizer unit(s) 130 can be configured to optimize the operation of element(s) 102 of a semiconductor device utilizing a multimodal machine-learned model as described herein.

[0060] The elements 102 are configured to provide operational information (e.g., temperature, performance state) to various components (e.g., policy manager(s) 108, latency calculator(s) 114, thermal govemor(s) 118)) of the SoC 600. Based on control policies, the policy’ manager(s) 108 indicate thermal target(s) to the thermal govemor(s) 118 and latency target(s) to the latency govemor(s) 126. The latency calculator(s) 114 are configured to provide latency of the element(s) 102 to the latency govemor(s) 126.

[0061] The optimizer units 130 detennine optimal operating parameters of the elements 102 based on received maximum performance states, received minimum performance states, and received performance state residencies 134 of the elements 102. The optimal operating parameters are applied by the optimizer units 130 to the elements 102 to transition the elements 102 to the optimal operation parameters. The computer-readable media 602 may be stored in computer-readable storage media, including one or more non-transitory storage devices such as a randomaccess memory’ (RAM), dynamic random access memory (DRAM), non-volatile random access memory’ (NVRAM), or static random access memory’ (SRAM), read-only memory’ (ROM), or flash memory, a hard drive, a solid-state drive (SSD), or any type of media suitable for storing electronic instructions, each coupled with a computer system bus.

[0062] The computer-readable media 602 of the SoC 600 may’ include executable code for optimized control of elements of semiconductor devices utilizing multimodal machine-learnedmodels. One or more of the processor(s) 604 operably coupled to computer-readable storage media having computer-readable media 602 may execute instructions for optimizing controllers for semiconductor devices utilizing multimodal machine-learned models.

[0063] FIG. 7 illustrates an example environment 700 of an example electronic device 702 that includes optimized control of elements of semiconductor devices utilizing multimodal machine-learned models in accordance with one or more implementations. The electronic device 702 may include additional components and interfaces omitted from FIG. 7 for the sake of clarity. The electronic device 702 is illustrated with various non-limiting example electronic devices 702, including wireless earbuds 702-1, a smart display associated with a home-automation and control system 702-2, a desktop computer 702-3, a tablet 702-4, a laptop 702-5, a television 702-6, a computing watch 702-7, computing glasses 702-8, a gaming system 702-9, a microwave 702-10, a smart thermostat interface 702-11, and an automobile having computing capabilities 702-12. Other devices may also be used, such as wired earbuds, a security' camera, a trackpad, a drawing pad, a netbook, an e-reader, other forms of home-automation and control systems, a wall display, a virtual-reality headset, another vehicle (e.g., an e-bike or plane), and other home appliances, to name just a few examples. Note that the electronic device 702 may be wearable, non-wearable but mobile, or relatively immobile (e.g., desktops and appliances), all without departing from the scope of the present teachings.

[0064] The electronic device 702 includes a housing 704, which defines at least one internal cavity within which one or more of a plurality of electronic components may be disposed. In implementations, a mechanical frame may define one or more portions of the housing 704. As an example, a mechanical frame can include plastic or metallic walls that define portions of the housing 704. In additional implementations, a mechanical frame may support one or more portions of the housing 704. As an example, one or more exterior housing components (e.g., plastic panels) can be attached to the mechanical frame (e.g., a chassis). In so doing, the mechanical frame physically supports the one or more exterior housing components, which defineportions of the housing 704. In implementations, the mechanical frame and / or the exterior housing components may be composed of crystalline or non-crystalline solids. In implementations, the housing 704 may be sealed through the inclusion of one or more displays (e.g., at least one display 716). defining at least one internal cavity.

[0065] The electronic device 702 may further include one or more processors 706. The processor(s) 706 can include, as non-limiting examples, an SoC, an application processor (AP), a CPU. or a GPU. The processor(s) 706 generally execute commands and processes utilized by the electronic device 702 and an operating system installed thereon. For example, the processor(s) 706 may perform operations to display graphics of the electronic device 702 on the one or more displays 716 and can perform other specific computational tasks.

[0066] The electronic device 702 may also include computer-readable storage media (CRM) 708. The CRM 708 may be a suitable storage device configured to store device data of the electronic device 702, user data, and multimedia data. The CRM 708 may store an operating system 710 that generally manages hardware and software resources (e.g., the applications) of the electronic device 702 and provides common services for applications stored on the CRM 708. The operating system 710 and the applications are generally executable by the processor(s) 706 to enable communication and user interaction with the electronic device 702. One or more processors 706, such as a GPU, perform operations to display graphics of the electronic device 702 on the one or more displays 716 and can perform other specific computational tasks. The processors 706 can be single-core or multiple-core processors.

[0067] The electronic device 702 may also include input / output (I / O) ports 712. The I / O ports 712 allow the electronic device 702 to interact with other devices or users. The I / O ports 712 may include any combination of internal or external ports, such as universal serial bus (USB) ports, audio ports, serial advanced technology attachment (SATA) ports, peripheral component interconnect standard (PCI)-express based ports or card-slots, secure digital input / output (SDIO) slots, and / or other legacy ports.

[0068] The electronic device 702 may further include one or more sensors 714. The sensor(s) 714 can include any of a variety of sensors, such as an audio sensor (e.g., a microphone), a touch-input sensor (e.g., a touchscreen), an image-capture device (e.g., a camera, video-camera), proximity sensors (e.g., capacitive sensors), an under-display fingerprint sensor, or an ambient light sensor (e.g., photodetector). In implementations, the electronic device 702 includes one or more of a front-facing sensor(s) and a rear-facing sensor(s). An element 102 of the semiconductor device may be the various components of the electronic device 702. For example, the element 102 may be processor(s) 706, computer-readable storage media 708, I / O ports 712, sensors 714, display(s) 716, a battery 722, or the like. The operating system 710 and / or various processor(s) 706 of the electronic device 702 include operating instructions to enable optimized control of elements of semiconductor devices utilizing multimodal machine-learned models to control the operational transition of various components of the electronic device 702 to ensure optimal performance for utilizing multimodal machine-learned models.

[0069] The electronic device 702 may include the one or more displays 716, one or more cover layers 718, and one or more display panels 720. The cover lay er(s) 718 may be implemented as any of a variety of transparent materials including polymers (e.g., plastic, acrylic) or glasses.

[0070] The electronic device 702 further includes a battery 722. In implementations, the battery7722 is a rechargeable battery' that is configured to store and supply electrical energy'. The rechargeable battery' 722 may be any suitable rechargeable battery', such as a lithium-ion (Li-ion) battery'.Example Method

[0071] FIG. 8 is a flowchart that illustrates a method 800 for an adaptive controller scheduling framework for semiconductor devices, which includes operations 802 through 810.

[0072] At step 802, a first state performance residency of a first element of a semiconductor device is received at an optimizer unit, the first performance state residencyindicating a performance history of the first element. For example, an optimizer unit (e.g., optimizer unit 130) receives a performance state residency (e.g., performance state residency 134) that indicates a performance history of an element (e.g., element 102).

[0073] At step 804, a first maximum performance state of the first element is received at the optimizer unit from a thermal governor, the first maximum performance state determined based on a first thermal target, a first temperature of the first element, and an enclosure temperature. For example, the optimizer unit (e.g., optimizer unit 130) receives a maximum performance state (e.g., maximum performance state 124) from athermal governor (e.g.. thermal governor 118). The thermal governor (e.g., thennal governor 118) bases the maximum performance state (e.g., maximum performance state 124) on a temperature (e.g., temperature 104) of the element (e.g., element 102), a thermal target (e.g., thermal target 110) received from a policy’ manager (e.g., policy manager 108), and an enclosure temperature (e.g., enclosure temperature 122) received from an enclosure temperature sensor (e.g., enclosure temperature sensor 120).

[0074] At step 806, a first minimum performance state of the first element is received at the optimizer unit from a latency governor, the first minimum performance state determined based on a first latency target and a first latency of the first element. For example, the optimizer unit (e.g., optimizer unit 130) receives a minimum performance state (e.g., minimum performance state 128) from a latency governor (e.g., latency governor 126). The latency governor (e.g., latency governor 126) determines the minimum performance state (e.g., minimum performance state 128) based on a latency target (e.g., latency target 112) received from the policy manager (e.g., policy manager 108) and a latency (e.g., latency 116) from a latency calculator (e.g., latency calculator 114).

[0075] At step 808, the optimizer unit determines first operating parameters of the first element, the first operating parameters determined based on the first performance state residency of the first element, the first maximum performance state, and the first minimum performance state. For example, the optimizer unit (e.g., optimizer unit 130) determines operating parameters(e.g., operating parameters 132) of the element (e.g., element 102). The operating parameters (e.g., operating parameters 132) may be determined based on the performance state residency (e.g., performance state residency 134) of the element 102, the maximum performance state (e.g., maximum performance state 124), and the minimum performance state (e.g., minimum performance state 128).

[0076] At step 810, the first operating parameters are applied to the first element, the applying effective to transition the first element to the first operating parameters. For example, the optimizer unit (e.g.. optimizer unit 130) applies the operating parameters (e.g., operating parameters 132) to the element (e.g., element 102). The application of the operating parameters (e.g., operating parameters 132) is effective to transition the element (e g., element 102) to the operating parameters (e.g., operating parameters 132). For example, the operating parameters may specify a specific performance state in which the element is to operate in. A performance state may be defined in any suitable way. For example, a performance state may indicate a specific clock frequency and / or a specific voltage level to be associated with the element. The method 800 is effective to optimize controllers for semiconductor devices utilizing multimodal machine-learned models.

[0077] For the methods described herein and the associated flowchart(s) and flow diagram(s), the orders in which operations are shown and / or described are not intended to be construed as a limitation. Instead, any number or combination of the described method operations can be combined in any order to implement a given method or an alternative method, including by combining operations from the flowchart or diagram and the earlier-described techniques into one or more methods. Operations may also be omitted from or added to the described methods. Further, described operations can be implemented in fully or partially overlapping manners.Example Aspects and Implementations of Optimized Control of Elements of Semiconductor Devices Utilizing Multimodal Machine-learned models

[0078] In the following, some example aspects and implementations are descnbed: Example Aspect 1. A method comprising receiving, at an optimizer unit, a first performance state residency of a first element of a semiconductor device, the first perfonnance state residency indicating a performance history of the first element: receiving, at the optimizer unit, a first maximum performance state of the first element from a thermal governor, the first maximum performance state determined based on a first thermal target, a first temperature of the first element, and an enclosure temperature; receiving, at the optimizer unit, a first minimum performance state of the first element from a latency governor, the first minimum performance state determined based on a first latency target and a first latency of the first element; determining, by the optimizer unit, first operating parameters of the first element, the first operating parameters determined based on the first performance state residency of the first element, the first maximum performance state, and the first minimum performance state; and applying the first operating parameters to the first element, the applying effective to transition the first element to the first operating parameters.

[0079] Example Aspect 2. The method of example aspect 1 , wherein the first operating parameters are between the first minimum performance state and the first maximum performance state. That is, applying the first operating parameters to the first element is effective to cause the first element of the semiconductor device to operate between the first minimum performance state and the first maximum performance state.

[0080] Example Aspect 3. The method of example aspects 1 or 2, wherein the first operating parameters are above the first maximum performance state based on the first performance state residency of the first element indicating prior performance states of the first element warrant exceeding the first maximum perfonnance state. That is, applying the first operating parameters to the first element is effective to cause the first element of thesemiconductor device to operate above the first maximum performance state based on the first performance state residency of the first element indicating prior performance states of the first element warrant exceeding the first maximum performance state.

[0081] Example Aspect 4. The method of any one of example aspects 1 to 3, wherein a machine-learned model determines the prior perfonnance states of the first element warrant exceeding the first maximum perfonnance state.

[0082] Example Aspect 5. The method of any one of example aspects 1 through 4, further comprising receiving, at the thennal governor, the enclosure temperature from a temperature sensor; receiving, at the thermal governor, the first temperature from the first element; and receiving, at the thennal governor, the first thermal target from a policy manager, the first thermal target determined based on a first performance state of the first element and a thermal policy of the semiconductor device.Example Aspect 6: The method of any one of example aspects 1 through 5, further comprising receiving, at the latency governor, the first latency of the first element from a latency calculator, the first latency determined based on the first performance state of the first element; and receiving, at the latency governor, the first latency target from the policy manager, the first latency target determined based on the first performance state of the first element and a latency policy of the semiconductor device.Example Aspect 7. The method of any one of example aspects 1 through 6, further comprising receiving, at the policy manager, the first performance state of the first element; determining the first thermal target based on the first performance state of the first element and the thermal policy of the semiconductor device; and determining the first latency target based on the first performance state of the first element and the latency policy of the semiconductor device.

[0083] Example Aspect 8. The method of any one of example aspects 1 through 7, further comprising receiving, at the latency calculator, the first perfonnance state of the first element; and determining the first latency based on the first performance state of the first element.

[0084] Example Aspect 9. The method of any one of example aspects 1 through 8, further comprising receiving, at the optimizer unit, a power performance state of the first element from a power governor, the power performance state determined based on a power target, wherein the first operating parameters are determined based on the first performance state residency of the first element, the first maximum performance state, the first minimum performance state, and the power perfonnance state.

[0085] Example Aspect 10. The method of any one of example aspects 1 through 9, wherein a policy manager determines the power target based on a first performance state of the first element and a power policy of the semiconductor device.

[0086] Example Aspect 11. The method of any one of example aspects 1 through 10, further comprising receiving, at the optimizer unit, a second performance state residency of a second element of the semiconductor device, the second performance state residency indicating a performance history of the second element; receiving, at the optimizer unit, a second maximum performance state of the second element from the thermal governor, the second maximum performance state determined based on a second thermal target, a second temperature of the second element, and the enclosure temperature; receiving, at the optimizer unit, a second minimum performance state of the second element from the latency governor, the second minimum performance state determined based at least in part on a second latency target and a second latency of the second element; determining, by the optimizer unit, second operating parameters of the second element, the second operating parameters determined based on the second performance state residency of the second element, the second maximum performance state, and the second minimum performance state; and applying the second operating parameters to the second element, the applying effective to transition the second element to the second operating parameters.

[0087] Example Aspect 12. The method of any one of example aspects 1 through 11, further comprising receiving, at the thermal governor, the enclosure temperature from atemperature sensor; receiving, at the thermal governor, the first temperature from the first element and the second temperature from the second element; and receiving, at the thermal governor, the first thermal target and the second thermal target from a policy manager, the first thermal target determined based on a first performance state of the first element and a thermal policy of the semiconductor device and the second thermal target determined based on a second performance state of the second element and the thermal policy of the semiconductor device.

[0088] Example Aspect 13. The method of any one of example aspects 1 through 12, further comprising receiving, at the latency governor, the first latency of the first element and a second latency of the second element from a latency calculator, the first latency determined based on the first performance state of the first element and the second latency determined based on the second performance state of the second element; and receiving, at the latency governor, the first latency target and the second latency target from the policy manager, the first latency target determined based on the first performance state of the first element and a latency policy of the semiconductor device and the second latency target determined based on the second performance state of the second element and the latency policy of the semiconductor device.

[0089] Example Aspect 14. A non-transitory computer-readable memory storing instructions, which, when executed by one or more processors, cause the one or more processors to execute the method of any one of example aspects 1 to 13.

[0090] Example Aspect 15. An apparatus configured to perform the method of any one of example aspects 1 to 13.

[0091] Example Aspect 16. A system comprising a first element of a semiconductor device, the first element having a first temperature and a first performance state; a latency calculator coupled with the first element, the first element providing the first performance state to the latency calculator and the latency calculator configured to determine a first latency of the first element based on the first performance state; a policy manager coupled with the first element, the first element providing the first performance state of the policy manager, the policy managerconfigured to determine a first thermal target based on a thermal policy of the semiconductor device and the first performance state of the first element and to determine a first latency target based on a latency policy of the semiconductor device and the first performance state of the first element; an enclosure temperature sensor coupled with a thermal governor, the enclosure temperature sensor configured to provide an enclosure temperature to the thermal governor; the thermal governor being coupled with the policy manager and the first element, the first element configured to provide the first temperature to the thermal governor, the thermal governor configured to determine a first maximum performance state of the first element, the first maximum performance state of the first element determined based on the first thermal target provided by the policy manager, the first temperature of the first element, and the enclosure temperature; a latency governor coupled with the latency calculator and the policy manager, the latency governor configured to determine a first minimum performance state of the first element, the first minimum performance state of the first element determined based on the first latency target provided by the policy manager and the first latency of the first element provided by the latency calculator; and an optimizer unit coupled with the thermal governor, the latency governor, and the first element, the optimizer unit configured to determine and apply first operating parameters of the first element, the first operating parameters determined based on a first performance state residency of the first element, the first maximum performance state, and the first minimum perfonnance state, the first performance state residency of the first element indicating a performance history' of the first element.

[0092] Example Aspect 17. The system of example aspect 16, wherein applying the first operating parameters to the first element is effective to transition the first element to the first operating parameters.

[0093] Example Aspect 18. The system of example aspects 16 or 17, wherein the first operating parameters are between the first minimum performance state and the first maximum performance state.

[0094] Example Aspect 19. The system of any one of example aspects 16 to 18, wherein the first operating parameters are above the first maximum performance state based on the first performance state residency of the first element indicating prior performance states of the first element warrant exceeding the first maximum performance state.

[0095] Example Aspect 20. The system of any one of example aspects 16 through 19, wherein a machine-learned model determines the prior perfonnance states of the first element warrant exceeding the first maximum performance state.

[0096] Example Aspect 21. The system of any one of example aspects 16 through 20, further comprising a power governor coupled with the policy manager and the optimizer unit, wherein the policy manager determines a power target based on the first performance state of the first element and a power policy of the semiconductor device and the power governor is configured to determine a power performance state and the first operating parameters are detennined, by the optimizer unit, based on the first performance state residency of the first element, the first maximum performance state, the first minimum performance state, and the power performance state.

[0097] Example Aspect 22. The system of any one of example aspects 16 through 21, further comprising a second element of the semiconductor device, the second element being coupled with the thermal governor, the policy manager, the latency calculator, and the optimizer unit, the second element having a second temperature and a second performance state.

[0098] Example Aspect 23. The system of any one of example aspects 16 through 22, wherein the latency calculator is configured to determine a second latency of the second element based on the second performance state; the policy manager is configured to determine a second thermal target based on the thermal policy of the semiconductor device and the second performance state of the second element and to determine a second latency target based on the latency policy of the semiconductor device and the second performance state of the second element; the thermal governor is configured to determine a second maximum performance stateof the second element, the second maximum performance state of the second element determined based on the second thermal target provided by the policy manager, the second temperature of the second element, and the enclosure temperature; the latency governor is configured to determine a second minimum performance state of the second element, the second minimum performance state of the second element determined based on the second latency target provided by the policy manager and the second latency of the second element provided by the latency calculator; and the optimizer unit is configured to determine and apply second operating parameters of the second element, the second operating parameters determined based on a second performance state residency of the second element, the second maximum performance state, and the second minimum perfonnance state, the second performance state residency of the second element indicating a performance history of the second element.

[0099] Example Aspect 24. The system of any one of example aspects 16 through 23, wherein applying the second operating parameters to the second element is effective to transition the second element to the second operating parameters.

[0100] Example Aspect 25. The system of any one of example aspects 16 through 24, wherein the second operating parameters are betw een the second minimum performance state and the second maximum performance state.

[0101] Example Aspect 26. The system of any one of example aspects 16 through 25, wherein the second operating parameters are above the second maximum performance state based on the second performance state residency of the second element indicating prior performance states of the second element warrant exceeding the second maximum performance state.

[0102] Example Aspect 27. The system of any one of example aspects 16 through 26, wherein a machine-learned model determines the prior performance states of the second element warrant exceeding the second maximum performance state.

[0103] Example Aspect 28. The system of any one of example aspects 16 through 27, further comprising a controller coupled with the first element.

[0104] Example Aspect 29. The system of any one of example aspects 16 through 28, wherein the controller is coupled with a filter, controller parameters, and a policy target, the filter configured to provide feedback to the controller, the controller parameters configured to provide operating limits to the controller, and the policy target configured to provide one or more targets to the controller, the one or more targets based on control policies of the semiconductor device.

[0105] Example Aspect 30. The system of any one of example aspects 16 through 29, further comprising filter parameters coupled with the filter, the filter parameters configured to control the feedback provided to the controller by the filter.

[0106] Example Aspect 31. The system of any one of example aspects 16 through 30, further comprising one or more controllers within the thennal governor.

[0107] Example Aspect 32. The system of any one of example aspects 16 through 31, wherein the one or more controllers are each coupled with a filter, controller parameters, and a policy target, the filter configured to provide feedback to the controller, the controller parameters configured to provide operating limits to the controller, and the policy target configured to provide one or more targets to the controller, the one or more targets based on control policies of the semiconductor device.

[0108] Example Aspect 33. The sy stem of any one of example aspects 16 through 32, further comprising filter parameters coupled with the filter, the filter parameters configured to control the feedback provided to the controller by the filter.

[0109] Example Aspect 34. The method, apparatus, non-transitory computer-readable memory. or system of any preceding example aspect, wherein the first element comprises a central processing unit (CPU), a graphic processing unit (GPU), an accelerated processing unit (APU), an audio processing unit, and a tensor processing unit (TPU), and / or wherein the second element comprises a central processing unit (CPU), a graphic processing unit (GPU), an accelerated processing unit (APU), an audio processing unit, and a tensor processing unit (TPU).

[0110] Example Aspect 35. The method, apparatus, non-transitory computer-readable memory, or system of any preceding example aspect, wherein the first performance state residency of the first element indicating that prior performance states of the first element warrant exceeding the first maximum performance state comprises the first performance state residency of the first element indicating that the first element has not exceeded a thermal threshold in the past, and / or the second performance state residency of the second element indicating that prior perfonnance states of the second element warrant exceeding the second maximum performance state comprises the second performance state residency of the second element indicating that the second element has not exceeded a thermal threshold in the past.Conclusion

[0111] Unless context dictates otherwise, use herein of the word “or” may be considered use of an “inclusive or,” or a term that pennits inclusion or application of one or more items that are linked by the word “or” (e.g., a phrase “A or B” may be interpreted as permitting just “A,” as permitting just “B,” or as permitting both “A” and “B”). Also, as used herein, a phrase referring to “at least one of’ a list of items refers to any combination of those items, including single members. For instance, “at least one of a, b, or c” can cover a, b, c, a-b, a-c, b-c, and a-b-c, as well as any combination with multiples of the same element (e.g., a-a, a-a-a, a-a-b, a-a-c, a-b-b, a-c-c, b-b, b-b-b, b-b-c, c-c, and c-c-c, or any other ordering of a, b, and c). Further, items represented in the accompanying figures and terms discussed herein may be indicative of one or more items or terms, and thus reference may be made interchangeably to single or plural forms of the items and terms in this written description.

[0112] Although implementations for optimized control of elements of semiconductor devices utilizing multimodal machine-learned models have been described in language specific to certain features and / or methods, the subject of the appended claims is not necessarily limited to the specific features or methods described. Rather, the specific features and methods are disclosedas example implementations for optimized control of elements of semiconductor devices utilizing multimodal machine-learned models.

Claims

CLAIMSWhat is claimed is:

1. A method comprising:receiving, at an optimizer unit, a first performance state residency of a first element of a semiconductor device, the first performance state residency indicating a performance history of the first element;receiving, at the optimizer unit, a first maximum performance state of the first element from a thermal governor, the first maximum performance state determined based on a first thermal target, a first temperature of the first element, and an enclosure temperature;receiving, at the optimizer unit, a first minimum performance state of the first element from a latency governor, the first minimum performance state determined based on a first latency target and a first latency of the first element;determining, by the optimizer unit, first operating parameters of the first element, the first operating parameters determined based on the first performance state residency of the first element, the first maximum performance state, and the first minimum performance state; and applying the first operating parameters to the first element, the applying effective to transition the first element to the first operating parameters.

2. The method of claim 1 , wherein applying the first operating parameters to the first element is effective to cause the first element of the semiconductor device to operate between the first minimum performance state and the first maximum performance state.

3. The method of claim 1, wherein applying the first operating parameters to the first element is effective to cause the first element of the semiconductor device to operate above the first maximum performance state based on the first performance state residency of the first element indicating prior performance states of the first element warrant exceeding the first maximum perfonnance state.

4. The method of claim 3, wherein a machine-learned model determines that the prior performance states of the first element warrant exceeding the first maximum performance state.

5. The method of claim 1. further comprising:receiving, at the thermal governor, the enclosure temperature from a temperature sensor; receiving, at the thermal governor, the first temperature from the first element; and receiving, at the thermal governor, the first thermal target from a policy manager, the first thermal target determined based on a first performance state of the first element and a thermal policy of the semiconductor device.

6. The method of claim 5, further comprising:receiving, at the latency governor, the first latency of the first element from a latency calculator, the first latency determined based on the first performance state of the first element; andreceiving, at the latency governor, the first latency target from the policy manager, the first latency target determined based on the first performance state of the first element and a latency policy of the semiconductor device.

7. The method of claim 6, further comprising:receiving, at the policy manager, the first performance state of the first element; determining the first thermal target based on the first performance state of the first element and the thermal policy of the semiconductor device; anddetermining the first latency target based on the first performance state of the first element and the latency policy of the semiconductor device.

8. The method of claim 7. further comprising:receiving, at the latency calculator, the first performance state of the first element; and determining the first latency based on the first performance state of the first element.

9. The method of claim 1, further comprising:receiving, at the optimizer unit, a power performance state of the first element from a power governor, the power performance state determined based on a power target, wherein the first operating parameters are determined based on the first performance state residency of the first element, the first maximum performance state, the first minimum performance state, and the power performance state.

10. The method of claim 9, wherein a policy manager determines the power target based on a first performance state of the first element and a power policy of the semiconductor device.

11. The method of claim 1, further comprising:receiving, at the optimizer unit, a second performance state residency of a second element of the semiconductor device, the second performance state residency indicating a performance history of the second element;receiving, at the optimizer unit, a second maximum performance state of the second element from the thermal governor, the second maximum performance state determined based on a second thermal target, a second temperature of the second element, and the enclosure temperature;receiving, at the optimizer unit, a second minimum perfbnnance state of the second element from the latency governor, the second minimum performance state determined based at least in part on a second latency target and a second latency of the second element;determining, by the optimizer unit, second operating parameters of the second element, the second operating parameters determined based on the second performance state residency of the second element, the second maximum performance state, and the second minimum performance state; andapplying the second operating parameters to the second element, the applying effective to transition the second element to the second operating parameters.

12. The method of claim 11, further comprising:receiving, at the thermal governor, the enclosure temperature from a temperature sensor; receiving, at the thermal governor, the first temperature from the first element and the second temperature from the second element; andreceiving, at the thermal governor, the first thermal target and the second thermal target from a policy manager, the first thermal target determined based on a first performance state of the first element and a thermal policy of the semiconductor device and the second thermal target determined based on a second performance state of the second element and the thermal policy of the semiconductor device.

13. The method of claim 12, further comprising:receiving, at the latency governor, the first latency of the first element and a second latency of the second element from a latency calculator, the first latency determined based on the first performance state of the first element and the second latency determined based on the second performance state of the second element; andreceiving, at the latency governor, the first latency target and the second latency target from the policy manager, the first latency target determined based on the first performance state of the first element and a latency policy of the semiconductor device and the second latency target determined based on the second performance state of the second element and the latency policy of the semiconductor device.

14. A computer program product comprising computer-executable instructions that, when executed by a computing device, cause the computing device to perform any one of the methods of claims 1 to 13.

15. An apparatus configured to perform the method of any one of claims 1 to 13.

Images