Method, apparatus and program for estimation from multiple measurements

Abstract

Aspects of the present disclosure include a method, apparatus, and non-transitory computer-readable storage medium for estimating the complexity of an audio rendering program. One apparatus includes a processing circuit that executes a first plurality of instances of the audio rendering program on an audio evaluation platform to perform a first measurement of resource consumption. The processing circuit executes a second plurality of instances of the audio rendering program on the audio evaluation platform to perform a second measurement of resource consumption. A first number of instances in the first plurality is different from a second number of instances in the second plurality. The processing circuit estimates the complexity of the audio rendering program based on the first measurement, the second measurement, the first number of instances in the first plurality, and the second number of instances in the second plurality.

Description

[Technical field]

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS This application claims the benefit of priority to U.S. Provisional Application No. 63 / 178,660, entitled "ESTIMATION THROUGH MULTIPLE MEASUREMENTS," filed on April 23, 2021, which claims the benefit of priority to U.S. Provisional Application No. 17 / 488,419, entitled "ESTIMATION THROUGH MULTIPLE MEASUREMENTS," filed on September 29, 2021. The disclosures of the prior applications are incorporated herein by reference in their entireties.

[0002] This disclosure describes embodiments generally related to estimation in noisy environments. [Background technology]

[0003] The discussion of the background art provided herein is intended to provide a general context for the present disclosure. The work of the inventors named herein is not admitted, expressly or impliedly, as prior art to the present disclosure to the extent that their work is described in this Background section, and aspects of the description that may not otherwise qualify as prior art at the time of filing. Summary of the Invention [Problem to be solved by the invention]

[0004] Estimating quantities in noisy environments is a challenging task as measurements are often contaminated by noise. In some related examples, a self-comparison method has been proposed for complexity estimation of immersive audio renderers. In the self-comparison method, complexity is measured by comparing the central processing unit (CPU) usage for one instance and N instances of an application program. [Means for solving the problem]

[0005] An aspect of the disclosure provides an apparatus for estimating the complexity of an audio rendering program. One apparatus includes a processing circuit that executes a first plurality of instances of the audio rendering program on an audio evaluation platform to perform a first measurement of resource consumption. The processing circuit executes a second plurality of instances of the audio rendering program on the audio evaluation platform to perform a second measurement of resource consumption. A first number of the first plurality of instances is different from a second number of the second plurality of instances. The processing circuit estimates the complexity of the audio rendering program based on the first measurement, the second measurement, the first number of the first plurality of instances, and the second number of the second plurality of instances.

[0006] In one embodiment, the first number of instances of the first plurality is equal to the second number of instances of the second plurality minus one.

[0007] In one embodiment, the processing circuit determines a consumption difference between the first measured value and the second measured value. The processing circuit determines an instantaneous difference between a first number of instances of the first plurality and a second number of instances of the second plurality. The processing circuit estimates a complexity of the audio-rendering program based on the consumption difference and the instantaneous number difference.

[0008] In one embodiment, the processing circuitry executes a third plurality of instances of the audio rendering program on the audio evaluation platform to perform a third measurement of resource consumption, and the processing circuitry estimates a complexity of the audio rendering program based on a linear regression model applied to the first measurement, the second measurement, and the third measurement.

[0009] In one embodiment, the resource includes one of a central processing unit (CPU) runtime, a graphics processing unit (GPU) runtime, a CPU occupancy, or a GPU occupancy.

[0010] In one embodiment, the consumption of a resource is one of a runtime of the resource or an occupancy percentage of the resource.

[0011] In one embodiment, the audio rendering platform is one of a CPU-based platform or a GPU-based platform.

[0012] Aspects of the present disclosure provide a method for estimating complexity of an audio rendering program. In one method, a first measurement of resource consumption is performed by executing a first plurality of instances of the audio rendering program on an audio evaluation platform. A second measurement is performed on the resource consumption by executing a second plurality of instances of the audio rendering program on the audio evaluation platform. A first number of instances of the first plurality is different from a second number of instances of the second plurality. The program complexity is estimated based on the first measurement, the second measurement, the first number of instances of the first plurality, and the second number of instances of the second plurality.

[0013] Aspects of the present disclosure also provide a non-transitory computer-readable medium storing instructions that, when executed by at least one processor, cause the at least one processor to perform any one or combination of methods for estimating the complexity of an audio rendering program.

[0014] Further features, nature and various advantages of the disclosed subject matter will become more apparent from the following detailed description and the accompanying drawings. [Brief description of the drawings]

[0015] [Figure 1] 1 illustrates an example of six degrees of freedom according to an embodiment of the present disclosure. [Diagram 2] 1 illustrates an exemplary flow chart according to one embodiment of the present disclosure. [Diagram 3]FIG. 1 is a schematic diagram of a computer system according to one embodiment of the present disclosure. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0016] I. Complexity Estimation The Moving Picture Experts Group (MPEG) has proposed a suite of standards including immersive audio, immersive video, and system support. The suite of standards can support virtual reality (VR) or augmented reality (AR) presentations that allow users to navigate and interact with an environment using six degrees of freedom (6 DoF). Figure 1 shows an example of six degrees of freedom according to one embodiment of the present disclosure. In Figure 1, the six degrees of freedom can be represented by spatial navigation (x, y, z) and the user's head orientation (yaw, pitch, roll).

[0017] In an immersive audio or video environment, accurately estimating the complexity of a program running on a device (e.g., a VR or AR device) is made difficult by the presence of noise.

[0018] In some relevant examples, self-comparison methods can be used to estimate the complexity of a program. For example, in an audio scene, the complexity of an audio rendering program can be estimated by its central processing unit (CPU) usage (runtime) when the audio rendering program is running on the CPU. The audio rendering program may be Max / MSP, a visual programming language for music and multimedia.

[0019] In one example, the audio rendering program can be run on a standard test bench in the MPEG standard, such as an Audio Evaluation Platform (AEP). The audio rendering program can be run on the AEP as plug-in software. In a first measurement, the audio rendering program can be run on the AEP with one instance of the plug-in software, and the measured CPU workload is A1. In a second measurement, the audio rendering program can be run on the AEP with multiple (N) instances of the plug-in software, and the measured CPU workload is A2. N Therefore, the complexity of an audio rendering program is A plugin =(A N It can be estimated by A -A1) / (N-1). plugin can be used for comparative complexity estimation.

[0020] The premise of the self-comparison method is that measurement value A N The noise of measurement A1 is equal to the noise of measurement A1. N The noise in A1 and A2 may be close in one case, but may be quite different in another case. With this finding, the present disclosure includes a method for estimating the complexity with multiple measurements to obtain a better estimate in noisy environments.

[0021] The disclosed estimation method is not limited to complexity measures and is applicable to the estimation of a variety of variables in noisy environments.

[0022] It should be noted that the methods contained in this disclosure can be used individually or in combination. The methods can be used in part or in whole.

[0023] In this disclosure, the complexity measurement may be the resource consumption when an instance of an audio rendering program is running on a given platform, such as a CPU-based platform, a graphics processing unit (GPU)-based platform, etc. The resource consumption may be measured in terms of resource runtime, resource occupancy percentage, etc.

[0024] In one embodiment, the first measurement may be performed by executing a first plurality (M1) of instances of the plug-in software at the AEP, and the resource consumption may be measured as A M1 A second measurement can be performed by running a second multiple (M2) instances of the plug-in software on the AEP, and the same resource consumption is A M2 The complexity of an audio rendering program is A = (A M1 -A M2 ) / (M1-M2), where M1 is assumed to be greater than M2. In this embodiment, M1 and M2 are both positive integers, and M1 is not equal to M2.

[0025] In one embodiment, M1 and M2 are both greater than one.

[0026] In one embodiment, M2=M1-1. Thus, a first measurement can be performed by running M1 instances of the plug-in software on the AEP, and the resource consumption is A M1 The second measurement can be performed by running M1-1 instances of the plug-in software on the AEP, and the same resource consumption is A M1-1 In this embodiment, M1 is a positive integer equal to or greater than 2. And the complexity of the audio rendering program is A=A M1 -A M1-1 It can be estimated that:

[0027] In one embodiment, the first measurement may be performed by running M1 instances of the plug-in software on the AEP, and the resource consumption may be AM1 The second measurement can be performed by running M2 instances of the plug-in software on the AEP, and the same resource consumption is A M2 The third measurement can be performed by running M3 instances of the plug-in software on the AEP, and the same resource consumption is A M3 In this embodiment, M1, M2, and M3 are different positive integers. i( i=M1, M2, or M3), and the estimated complexity A is A i = i*A+B. Then, the estimated complexity A can be calculated by linear regression as follows:

number

[0028] In one embodiment, a plurality of measurements A 1、 A2~A k can be executed, and A i The plug-in software L i The measured consumption of the same resource for L instances. i is a positive integer, i = 1, 2, ~k, k is a positive integer greater than or equal to 2. The measured consumption A i and the estimated complexity A is A i =L i * A + B is assumed to fit a linear regression model. Then, the estimated complexity A can be obtained by linear regression as follows:

number

[0029] II. Flowchart Figure 2 shows a flow chart outlining an exemplary process (200) according to one embodiment of the present disclosure. In various embodiments, the process (200) is performed by a processing circuit, such as the processing circuit shown in Figure 3. In some embodiments, the process (200) is implemented with software instructions, such that the processing circuit performs the process (200) when the processing circuit executes the software instructions.

[0030] The process (200) may generally begin at step (S210), where the process (200) executes a first plurality of instances of an audio rendering program on an audio evaluation platform to perform a first measurement of resource consumption. The process (200) then proceeds to step (S220).

[0031] In step (S220), the process (200) executes, on the audio evaluation platform, a second plurality of instances of the audio rendering program to perform a second measurement of resource consumption, where a first number of the first plurality of instances is different from a second number of the second plurality of instances. Then, the process (200) proceeds to step (S230).

[0032] In step (S230), the process (200) estimates a complexity of the audio-rendering program based on the first measure, the second measure, the first number of instances in the first plurality, and the second number of instances in the second plurality, after which the process (200) ends.

[0033] In one embodiment, the first number of instances of the first plurality is equal to the second number of instances of the second plurality minus one.

[0034] In one embodiment, the process (200) determines a consumption difference between a first measurement and a second measurement. The process (200) determines an instantaneous difference between a first number of instances of the first plurality and a second number of instances of the second plurality. The process (200) estimates the program complexity based on the consumption difference and the instantaneous number difference.

[0035] In one embodiment, the process (200) executes a third plurality of instances of the audio rendering program on the audio evaluation platform to perform a third measurement of resource consumption. The process (200) estimates a complexity of the audio rendering program based on a linear regression model applied to the first measurement, the second measurement, and the third measurement.

[0036] In one embodiment, the resource includes one of a central processing unit (CPU) runtime, a graphics processing unit (GPU) runtime, a CPU occupancy, or a GPU occupancy.

[0037] In one embodiment, the consumption of a resource is one of a runtime of the resource or an occupancy percentage of the resource.

[0038] In one embodiment, the platform is one of a CPU-based platform or a GPU-based platform.

[0039] III. Computer Systems The techniques described above may be implemented as computer software using computer readable instructions physically stored on one or more computer readable media. For example, Figure 3 illustrates a computer system (300) suitable for implementing some embodiments of the disclosed subject matter.

[0040] Computer software may be coded using any suitable machine code or computer language that may undergo assembly, compilation, linking, or similar mechanisms to produce code containing instructions that may be executed directly, or through interpretation and execution of microcode, such as by one or more computer central processing units (CPUs) and graphics processing units (GPUs).

[0041] The instructions may be executed on various types of computers or components thereof, including, for example, personal computers, tablet computers, servers, smartphones, gaming devices, Internet of Things devices, and the like.

[0042] 3 of the computer system (300) are also exemplary of course and are not intended to suggest any limitation as to the scope of use or functionality of the computer software implementing the embodiments of the present disclosure, nor should the arrangement of components be interpreted as having a dependency or requirement relating to any one or combination of components illustrated in the exemplary embodiment of the computer system (300).

[0043] The computer system (300) may include several human interface input devices. Such human interface input devices may respond to input by one or more human users, for example, via tactile input (e.g., keystrokes, swipes, data glove movements), audio input (e.g., voice, clapping), visual input (e.g., gestures), or olfactory input (not depicted). Human interface devices may also be used to capture certain media that are not necessarily directly related to conscious human input, such as sound (e.g., speech, music, ambient sounds), images (e.g., scanned images, photographic images obtained from a still image camera), and video (e.g., two-dimensional video, three-dimensional video including stereoscopic video, etc.).

[0044] The input human interface devices may include one or more (only one of each is shown) of a keyboard (301), a mouse (302), a trackpad (303), a touch screen (310), a data glove (not shown), a joystick (305), a microphone (306), a scanner (307), and a camera (308).

[0045] The computer system (300) may also include several human interface output devices. Such human interface output devices may stimulate one or more of the senses of a human user, for example, through haptic output, sound, light, and smell / taste. Such human interface output devices may include haptic output devices (e.g., haptic feedback may be included via a touch screen (310), data gloves (not shown), or joystick (305), although there may also be haptic feedback devices that do not function as input devices), audio output devices (such as speakers (309), headphones (not shown)), visual output devices (such as screens (310), including CRT screens, LCD screens, plasma screens, OLED screens, each with or without touch screen input capability, each with or without haptic feedback capability, some of which may be capable of two-dimensional visual output or three-dimensional or higher output via means such as stereographic output, virtual reality glasses (not shown), holographic displays, and smoke tanks (not shown)), and printers (not shown). These visual output devices (such as the screen (310)) can be connected to the system bus (348) through a graphics adapter (350).

[0046] The computer system (300) may also include human accessible storage devices and their associated media, such as optical media, including CD / DVD ROM / RW (320) with media (321) such as CDs / DVDs, thumb drives (322), removable hard drives or solid state drives (323), legacy magnetic media such as tapes or floppy disks (not shown), and dedicated ROM / ASIC / PLD based devices such as security dongles (not shown).

[0047] Those skilled in the art should also understand that the term "computer-readable medium" as used in connection with the presently disclosed subject matter does not encompass transmission media, carrier waves, or other transitory signals.

[0048] The computer system (300) may also include a network interface (354) with one or more communication networks (355). The one or more communication networks (355) may be, for example, wireless, wired, optical. The one or more communication networks (355) may further be local, wide area, metropolitan, vehicular and industrial, real-time, delay tolerant, and the like. Examples of the one or more communication networks (355) include local area networks such as Ethernet, wireless LAN, cellular networks including GSM, 3G, 4G, 5G, and LTE, wired or wireless TV wide area digital networks including cable TV, satellite TV, and terrestrial broadcast TV, and vehicular and industrial networks including CANBus, and the like. Certain networks typically require an external network interface adapter attached to a specific general-purpose data port (e.g., a USB port on the computer system (300)) or peripheral bus (349), while other networks are typically integrated into the core of the computer system (300) by attaching to a system bus described below (e.g., an Ethernet interface to a PC computer system or a cellular network interface to a smartphone computer system). Using any of these networks, the computer system (300) can communicate with the other party. Such communication may be one-way receive only (e.g., broadcast TV), one-way transmit only (e.g., CANbus to certain CANbus devices), or bidirectional, for example, with other computer systems using local or wide-area digital networks. Specific protocols and protocol stacks may be used with each of these networks and network interfaces, as described above.

[0049] The aforementioned human interface devices, human accessible storage devices, and network interfaces may be attached to the core (340) of the computer system (300).

[0050] The cores (340) may include specialized programmable processing devices in the form of one or more central processing units (CPUs) (341), graphics processing units (GPUs) (342), field programmable gate areas (FPGAs) (343), task-specific hardware accelerators (344), etc. These devices may be connected via a system bus (348) along with read-only memory (ROM) (345), random access memory (346), and internal mass storage devices (347) such as internal hard drives or SSDs that are not user accessible. In some computer systems, the system bus (348) may be directly operable in the form of one or more physical plugs that allow expansion with additional CPUs, GPUs, etc. Peripheral devices may be attached directly to the core's system bus (348) or via a peripheral bus (349). Peripheral bus architectures include PCI, USB, etc.

[0051] The CPU (341), GPU (342), FPGA (343), and accelerator (344) can execute certain instructions that may combine to constitute the computer code described above. The computer code can be stored in a ROM (345) or a RAM (346). Transient data can also be stored in the RAM (346), while immutable data can be stored, for example, in an internal mass storage device (347). The use of a cache memory, which can be closely associated with one or more of the CPU (341), GPU (342), mass storage device (347), ROM (345), RAM (346), etc., can enable fast storage and retrieval in any of the memory devices.

[0052] The computer-readable medium can bear computer code for performing various computer-implemented operations. The medium and computer code may be those specially designed and constructed for the purposes of the present disclosure, or they may be of the available kind well known to those skilled in the computer software arts.

[0053] By way of example and not limitation, the architecture, and in particular the computer system (300) having the core (340), can provide functionality as a result of one or more processors (including CPUs, GPUs, FPGAs, accelerators, etc.) executing software embodied in one or more tangible computer-readable media. Such computer-readable media can be media associated with mass storage devices that a user may directly manipulate, as described above, or even specific storage devices of the core (340) that are non-transitory storage devices, such as the core internal mass storage (347) or ROM (345). Software implementing various embodiments of the present disclosure can be stored in such devices and executed by the core (340). The computer-readable media can include one or more memory devices or chips, depending on the particular needs. The software can enable the core (340), and in particular the processors (including CPUs, GPUs, FPGAs, etc.) in the core (340), to execute certain processes or certain parts of certain processes described in the present application, including the definition of data structures stored in RAM (346) and the modification of such data structures according to the processes defined by the software. Additionally or alternatively, the computer system may achieve functionality as a result of logic hardwired or otherwise embodied in circuitry (e.g., accelerator (344)) that may operate in place of or together with software to perform certain processes or certain portions of certain processes described herein. Where appropriate, references to software may encompass logic and vice versa. Where appropriate, references to computer-readable media may encompass circuitry (such as integrated circuits (ICs)) that stores software for execution, circuitry that embodies logic for execution, or both. The present disclosure encompasses any suitable combination of hardware and software.

[0054] While this disclosure has described several exemplary embodiments, there are modifications, substitutions, and various substitute equivalents which are within the scope of this disclosure. It will thus be appreciated that those skilled in the art will be able to devise numerous systems and methods which, although not explicitly shown or described herein, embody the principles of the present disclosure and are therefore within its spirit and scope. [Explanation of symbols]

[0055] 200 Processing 300 Computer Systems 301 Keyboard 302 Mouse 303 Trackpad 305 Joystick 306 Mike 307 Scanner 308 Camera 309 Speaker 310 Touch Screen 320 Medium 321 CD / DVD ROM / RW 322 Thumb Drive 323 Solid State Drive 340 cores 341 Central Processing Unit (CPU) 342 Graphics Processing Unit (GPU) 343 Field Programmable Gate Area (FPGA) 344 Hardware Accelerator 345 Read-Only Memory (ROM) 346 Random Access Memory (RAM) 347 Internal Mass Storage 348 System Bus 349 Surrounding Bus 350 Graphics Adapter 354 Network Interface 355 Communication Network

Claims

1. 1. A method for estimating a complexity A of an audio rendering program, the method comprising the steps of: performing k measurements on an audio evaluation platform, each measurement including executing L i instances of the audio rendering program to perform an i th measurement of resource consumption to obtain a measurement A i , where 1≦i≦k, k is a positive integer greater than or equal to 2, and L i are positive integers and different from each other; [0010] A step of estimating the complexity A by a linear regression model: A method comprising:

2. The method of claim 1, wherein k=2 and L 1 =L 2 −1.

3. k = 2, and the step of estimating the complexity A by the linear regression model comprises: determining a first difference between measurements A 1 and A 2 ; determining a second difference between L 1 and L 2 ; and dividing the first difference by the second difference to obtain an estimate of the complexity A.

4. The method of claim 1 , wherein the resource comprises one of a central processing unit (CPU) runtime, a graphics processing unit (GPU) runtime, a CPU occupancy, or a GPU occupancy.

5. The method of claim 1 , wherein the consumption of the resource is one of a run time of the resource or an occupancy percentage of the resource.

6. The method of claim 1 , wherein the audio evaluation platform is one of a CPU-based platform or a GPU-based platform.

7. A processing circuit configured to carry out the method according to any one of claims 1 to 6. An apparatus comprising:

8. A program comprising instructions which, when executed by at least one processor, cause said at least one processor to perform the method of any one of claims 1 to 6.

Images