System and method for optimizing bill of material cost and power performance for graphics processing unit (GPU) cores

Abstract

The invention provides a system and a method for optimizing BoM cost of a graphics processing unit (GPU) core. The system (100) includes a GPU (101) having a typical computing unit and a GPU instruction set, but having a video SIMD instruction lacking saturation logic, in which a video codec encoder / decoder module is implemented. A loop deblocking filter and a post-processing filter module (102) are implemented in the GPU. Removal of logical hardware for achieving saturation in video SIMD instructions helps to reduce BoM costs, and the steps of the present invention help to achieve bit-accurate results without saturation logic. The power consumed when the saturation logic is not used is less than the power consumed when the saturation logic is used, thereby providing an added value for platform SoC designers and manufacturers to use a low-BoM cost GPU with better power performance.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims priority to Indian patent application serial number IN 202341044078 filed on June 30, 2023, the entire contents of which are incorporated herein by reference. Technical Field

[0003] The present invention relates to a system and method for optimizing the bill of materials (BoM) cost and power performance of a graphics processing unit (GPU) core. More specifically, the present invention relates to optimizing the BoM cost of a GPU core by implementing a video codec encoder, decoder, loop filter / post-processing filter (deblocking filter) on a GPU with video single instruction multiple data (SIMD) and executing control software in a central processing unit (CPU) on a platform system-on-chip (SoC), wherein there is no hard-wired saturation logic in the video SIMD instructions. Background Art

[0004] Platform SoCs are usually composed of CPU cores, DSP cores, and GPU cores. Platform SoCs are used to build different types of handheld devices, such as mobile phones, drones, battery management systems, etc. Computationally intensive video codecs are implemented on the DSP, sometimes with the assistance of hardware coprocessors.

[0005] There are various video compression standards such as MPEG-1, MPEG-2, MPEG-4, H.264, H.265, H.266, AV1, VP8, VP10 and more are coming soon. The codec runs on the DSP in the platform SoC and consumes a lot of power. Recent trends explore designs that implement the video codec on the GPU and run the control software on the CPU.

[0006] The high load of the processor in video communication causes a large amount of power consumption, resulting in a sharp decrease in the battery life of the device used in video communication.

[0007] The GPU core instruction set architecture (ISA) has saturation logic built into the video SIMD instructions. The saturation logic in each instruction costs hundreds / thousands of gates, which increases the chip area and BoM cost of the GPU core. The GPU core ISA has SIMD-only extensions and lacks saturation logic except for three instructions, namely ADD, SUB, and a dedicated SAT instruction. The BoM cost of the resulting GPU core is much lower than that of a GPU with saturation logic. It is challenging to obtain bit-accurate results and power-optimal solutions on a GPU that contains this video SIMD instruction without saturation logic. The present invention achieves the results of implementing video codec encoders, decoders, and loop filter modules on a GPU by using GPU core video SIMD instructions without saturation logic, while achieving better power performance and reducing BoM costs.

[0008] US Patent No. 10390309, entitled "System and method for optimizing power consumption in mobile devices", discloses a method and apparatus for optimizing power consumption in mobile devices through suitable instruction set architecture feature changes and optimized implementation of voice codecs. However, the solution is intended to target mainly the use case of voice calls and the CPU.

[0009] US Patent No. 11330526, entitled "System and method for optimizing power consumption in video communication in mobile devices", discloses a method and apparatus for optimizing power consumption in mobile devices through appropriate instruction set architecture feature changes and optimized implementation of video codecs. However, the solution aims at power optimization of video call usage using CPU and DSP cores. Summary of the invention

[0010] The present invention overcomes the deficiencies in the related art and provides a system and method for optimizing the BoM cost and power performance of a GPU in a platform SoC for a mobile device.

[0011] The system includes a CPU in the platform SoC and a GPU with video SIMD instructions. Digital video data with a resolution of 8 bits and 10 bits is provided by the CPU to the GPU for encoding / decoding. The control software is implemented on the CPU, while the computationally intensive encoder, decoder, and filter modules are implemented on the GPU.

[0012] In an embodiment of the invention, the digital video signal is encoded according to the compression standard H.264 or any other standard suitable for the application. Various encoding tools (such as intra prediction, motion compensation, filtering) are implemented in a GPU containing video SIMD instructions without saturation logic. The current consumption in this GPU is less than the current consumption in the case where the GPU contains video SIMD instructions with saturation logic, and has the additional advantage of reducing the BoM cost of the GPU.

[0013] The system also includes a video codec decoder. The video codec decoder module is configured to decode the compressed video signal. The decoded video signal is then post-processed using a deblocking filter module. All of these modules are implemented in the GPU core.

[0014] Therefore, the present invention provides a method for optimizing the BoM cost of a GPU while saving power consumption in a video call in a production device compared to a GPU containing video SIMD instructions with saturated logic.

[0015] The GPU core in the platform SoC is replaced with a GPU core with a smaller chip area, thereby reducing the BoM cost of the GPU. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] The above and other features of the embodiments will become more apparent when the following detailed description of the embodiments is read in conjunction with the accompanying drawings, in which like reference numerals refer to like elements.

[0017] Figure 1 A block diagram of a system for implementing a video codec in a platform SoC having a CPU and a GPU according to one embodiment of the present invention is shown.

[0018] Figure 2 A method for optimizing the BoM cost of a GPU according to one embodiment of the present invention is shown.

[0019] Figure 3 A method for optimizing the BoM cost of a GPU while implementing a video codec in a platform SoC according to one embodiment of the present invention is shown. DETAILED DESCRIPTION

[0020] The description of this theme will now be referred to in detail, one or more examples of this theme are shown in the accompanying drawings. Each example is provided to explain the theme, rather than to limit it. Various changes and modifications obvious to those skilled in the art to which the present invention belongs are considered to be within the spirit, scope and consideration of the present invention.

[0021] In order to more clearly and concisely describe and point out the claimed subject matter of the present invention, the following definitions are provided for specific terms used in the following written description.

[0022] The present invention provides a system and method for optimizing the BoM cost and power performance of a GPU. The system includes a CPU with all the control software required for a video codec. The video encoder, decoder, and filtering are implemented on a GPU with video SIMD instructions. Implementing the video codec and post-processing modules (deblocking filters) in a GPU containing video SIMD instructions without saturation logic instead of in a GPU with saturation logic results in a reduction of up to thousands of logic gates in the platform SoC.

[0023] Figure 1 A block diagram of a system for implementing a video codec in a mobile device according to an embodiment of the present invention is shown. In a preferred embodiment, the system includes a CPU core (101) which is responsible for the control software for encoding / decoding. The encoder, decoder, loop filter / post-processing module (deblocking filter) are implemented on a GPU (102) with video SIMD instructions.

[0024] Various encoding tools of the digital video compression standard specification, namely intra prediction, motion estimation / inter prediction, motion compensation, transform, quantization, filtering, are implemented on a GPU with video SIMD instructions. Compared to implementation on a GPU with saturation logic in the video SIMD instructions, current consumption in the SoC is reduced, while BoM cost is reduced (102).

[0025] Figure 2 In an embodiment of the present invention, implementing the video codec and post-processing (deblocking filter) module on the GPU core results in an efficient implementation of the video codec without increasing current consumption, while reducing the BoM cost of the platform SoC. The architecture of the SoC includes a CPU and a GPU including video SIMD instructions without saturation logic. Therefore, in the present invention, a platform SoC with a lower BoM cost is constructed while still providing good power consumption in the GPU.

[0026] Figure 3A method for optimizing power consumption in a video codec module in a GPU according to one embodiment of the present invention is shown. In a preferred embodiment, the method starts with the step of receiving and recording an original video at step 301 .

[0027] At step 302, the video frame samples are encoded by the video codec encoder module. The video encoder is implemented on a GPU containing video SIMD instructions without saturation logic. At step 303, the deblocking filter and other post-processing are implemented on the GPU core. Compared with the implementation using video SIMD instructions with saturation logic, the current consumption in running the video encoder is better, and the limitation of lacking saturation logic in the instructions is overcome to achieve bit-accurate results.

[0028] At step 304, the compressed video signal is decoded by the video codec decoder (102).

[0029] At step 305, the decoded video is deblocked and post-processed to obtain an output video frame (102).

[0030] The creative steps of the encoder, decoder, loop / post-processing (deblocking) module implemented on a GPU with video SIMD instructions are now described. Video frame samples are usually 8 bits. Today, 10-bit video samples are supported. In the intra prediction module, the size of the intra block can vary from 4×4 to 32×32 depending on the video standard. The reference sample is also 8 bits or 10 bits and is filtered. The output of the reference sample is also 8 bits or 10 bits and can be applied within the 16-bit span of a 32-bit register. The filtering operation is a 3rd order (tap) filter, followed by reduction, and the intermediate result is also within the 16-bit span. Therefore, it is safe to turn off saturation, and the instruction set can have MAC instructions in which saturation is not embedded, and SIMD optimization can be performed. Therefore, the input of the transform module in the intra prediction module is 9 bits or 11 bits. The transform module is implemented to ensure that the intermediate results and output results do not span the 16-bit span. Therefore, saturation can be turned off again for UI, and MAC instructions that are not embedded with saturation are useful, and SIMD optimization can be performed. In the inter-block coding tool, the input pixel value is 8 bits or 10 bits. Motion estimation can be performed efficiently and correctly based on the sum of absolute differences (SAD) so that it does not span the 16-bit span. The input to the transformation of the inter-coded block is 9 bits or 11 bits, and this can also be processed within the 16-bit span of the intermediate result / final result. Therefore, saturation can be turned off, enabling the instruction set architecture (ISA) to have a MAC that does not embed saturation in the instruction, and SIMD optimization can be performed, thereby saving bill of materials (BOM) costs and saving power in optimization. In the deblocking module, the size of the block can vary from 4×4 to 32×32 according to the video standard. The filtering operation is a 3rd order filter, and the intermediate result is also within the 16-bit span. Therefore, it is safe to turn off saturation, and the instruction set can be constructed without embedding saturation therein, and SIMD optimization can be performed, thereby saving bill of materials (BOM) costs and providing better power performance, just like in a GPU with saturation logic in the deblocking filter.

[0031] Therefore, the present invention provides a method for optimizing the BoM cost of a GPU.

[0032] As used in this application, the terms "component" and "system" are intended to refer to computer-related entities, hardware, a combination of hardware and software, software, or software that is executed. For example, a component can be, but is not limited to, a process, a processor, an object, an executable file, a thread of execution, a program, and / or a computer running on a processor. By way of illustration, both an application running on a server and a server can be a component. One or more components can reside in a process and / or a thread of execution, and a component can be located on a computer and / or distributed between two or more computers.

[0033] Generally, program modules include routines, programs, components, data structures, etc. that perform specific tasks or implement specific abstract data types. In addition, those skilled in the art will recognize that the present invention can be practiced with other computer system configurations, including single-processor or multi-processor computer systems, minicomputers, mainframe computers, and personal computers, handheld computing devices, microprocessor-based or programmable consumer electronic devices, etc., each of which can be operably coupled to one or more associated devices.

[0034] The illustrated aspects of the present invention can also be practiced in distributed computing environments where certain tasks are performed by remote processing devices that are linked through a communications network. In a distributed computing environment, program modules may be located in both local and remote memory storage devices.

[0035] Computers typically include various computer-readable media. Computer-readable media can be any available media that can be accessed by a computer and includes volatile and nonvolatile media, removable media, and non-removable media. By way of example and not limitation, computer-readable media can include computer storage media and communication media. Computer storage media include volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storing information, such as computer-readable instructions, data structures, program modules, or other data. Computer storage media include, but are not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disk (DVD) or other optical disk storage devices, cassettes, magnetic tapes, disk storage devices or other magnetic storage devices, or any other media that can be used to store desired information and can be accessed by a computer.

[0036] Communication media are typically embodied as computer-readable instructions, data structures, program modules or other data in a modulated data signal (such as a carrier wave or other transport mechanism), and include any information delivery media. The term "modulated data signal" means a signal that has one or more characteristics set or changed in a manner that encodes the information in the signal. By way of example and not limitation, communication media include wired media, such as a wired network or a direct wired connection, and wireless media, such as acoustic, radio frequency (RF), infrared and other wireless media. Combinations of any of the above should also be included within the scope of computer-readable media.

[0037] Software includes applications and algorithms. Software can be implemented in a smartphone, tablet or personal computer, in the cloud, on a wearable device, or other computing or processing device. Software can include records, diaries, tables, games, recordings, communications, SMS messages, websites, charts, interactive tools, social networks, VOIP (Voice over Internet Protocol), emails, and videos. Software includes applications and algorithms. Software can be implemented in a smartphone, tablet or personal computer, in the cloud, on a wearable device, or other computing or processing device. Software can include records, diaries, tables, games, recordings, communications, SMS messages, websites, charts, interactive tools, social networks, VOIP (Voice over Internet Protocol), emails, and videos.

[0038] In some embodiments, some or all of the functions or (one or more) processes described herein and performed by a computer program are formed by computer-readable program code and embodied in a computer-readable medium. The phrase "computer-readable program code" includes any type of computer code, including source code, object code, executable code, firmware, software, etc. The phrase "computer-readable medium" includes any type of medium that can be accessed by a computer, such as a read-only memory (ROM), a random access memory (RAM), a hard drive, a compact disk (CD), a digital video disk (DVD), or any other type of memory.

[0039] All publications and patent applications mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication or patent application was specifically and individually indicated to be incorporated by reference.

[0040] Although the present invention has been described in conjunction with various embodiments, it will be appreciated that the present invention is capable of further modification. This application is intended to cover any variation, use or adaptation of the present invention, which generally follows the principles of the present invention and includes deviations from the present disclosure within known and conventional practices in the art to which the present invention belongs.

Claims

1. A system (100) for optimizing bill of materials (BoM) cost and power performance of a graphics processing unit (GPU) core, the system (100) comprising: a central processing unit (CPU) core (101), the central processing unit (CPU) core is used to execute the control code; b. a video codec encoder and decoder module, a loop filter / post-processing module, wherein the loop filter / post-processing module includes a deblocking filter (102), wherein the video codec encoder and decoder module, the filtering module (102) is implemented in a GPU core with video SIMD instructions; c. Saturation in the video codec encoder, decoder, loop filter / post-processing module (deblocking filter) is turned off; and d. The GPU's arithmetic logic unit (ALU) is designed so that video SIMD instructions do not have saturation logic in key instructions, including multiply and accumulate (MAC) instructions and shift instructions.

2. The system according to claim 1, wherein: The video codec modules (101, 102) include MPEG-1, MPEG-2, MPEG-4, H.264, H.265, H.266, AV1, VP8, and VP10 standard video codecs.

3. The system according to claim 1, wherein: The BoM cost of the GPU core is reduced compared to a GPU core containing video SIMD instructions with saturation logic to implement video codec encoder and decoder modules, loop / post-processing (deblocking filter).

4. A method for optimizing the BoM cost and power performance of a GPU core, the method comprising the following steps: a. Execute the control code in the CPU core (101); b. Implementing a video codec encoder, decoder, loop / post-processing filter module, wherein the loop / post-processing filter module includes a deblocking filter (102), wherein: The filtering module (102) is implemented in a GPU core having video SIMD instructions; c. Turn off saturation in the video codec encoder, decoder, loop filtering / post-processing module (deblocking filter); and d. The ALU of the GPU is designed so that video SIMD instructions do not have saturation logic in key instructions, including but not limited to MAC instructions and shift instructions.

5. The method according to claim 4, wherein: The video codec modules (101, 102) include MPEG-1, MPEG-2, MPEG-4, H.264, H.265, H.266, AVI, VP8, and VP10 standard video codecs.

Images