A method, device, storage medium and program product for executing a rotation operator
By dividing the image into multiple pixel blocks and performing rotation operations in the on-chip cache, the problem of high memory access pressure of the rotation operator is solved, and performance is improved.
Patent Information
- Application Number
- CN202410579441.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-05-10
- Publication Date
- 2025-08-19
- Estimated Expiration
- 2044-05-10
AI Technical Summary
In the prior art, frequent access to video memory during the execution of the rotary operator results in greater memory access pressure and poor performance.
The pending images in the video memory are divided into multiple raw pixel blocks and loaded into the on-chip cache for rotation operations. The data is buffered using the on-chip cache, and data transmission is optimized through memory merge access and cache line alignment.
The frequency of accessing video memory is reduced, the memory access pressure is reduced, and the performance of the image rotation operator is improved.
Smart Images
Figure CN118505513B_ABST
Abstract
Description
Technical Field
[0001] The embodiments of the present application relate to the field of artificial intelligence technology, and in particular to a method, device, storage medium, and program product for executing a rotation operator. Background Art
[0002] Artificial intelligence models typically refer to neural network models trained to perform inference and prediction, such as image inference models and speech inference models. In practical applications, artificial intelligence chips are used to implement the operations of artificial intelligence models.
[0003] In technical fields like machine vision and image processing, AI chips often need to rotate the entire input image before processing it. To achieve this rotation, the relevant rotation operator loads the input image onto the chip at a pixel granularity, performs the rotation operation at that pixel granularity, and writes the result to the video memory.
[0004] However, during the execution of the above rotation operator, the video memory needs to be frequently accessed, which results in a large memory access pressure and further leads to poor performance of the rotation operator. Summary of the Invention
[0005] Embodiments of the present application provide a method, device, storage medium, and program product for executing a rotation operator, which are used to reduce memory access pressure during the execution of the rotation operator and improve the performance of the rotation operator.
[0006] In one aspect, an embodiment of the present application provides a method for executing a rotation operator, comprising:
[0007] The image to be processed in the video memory is divided into a plurality of original pixel blocks, and the following operations are performed on the plurality of original pixel blocks:
[0008] loading a raw pixel block from the video memory into an on-chip cache;
[0009] Reading a plurality of pixel points included in the original pixel block from the on-chip cache, and performing rotation operations on the original coordinates of the plurality of pixel points respectively to obtain target coordinates of the plurality of pixel points;
[0010] Writing the plurality of pixel points to the on-chip cache respectively according to target coordinates of the plurality of pixel points to obtain a target pixel block;
[0011] The target pixel block is written out from the on-chip cache to the display memory.
[0012] Optionally, among the multiple pixel points included in the one original pixel block, there are at least two pixel points whose positions in the video memory are continuous;
[0013] The step of loading an original pixel block from the video memory to an on-chip cache memory includes:
[0014] A memory merge access method is used to load an original pixel block from the video memory into an on-chip cache.
[0015] Optionally, among the multiple pixel points included in the target pixel block, there are at least two pixel points whose positions written to the video memory are continuous;
[0016] Writing the target pixel block from the on-chip cache to the video memory includes:
[0017] The target pixel block is written out from the on-chip cache to the video memory using a memory merge access method.
[0018] Optionally, reading the plurality of pixel points included in the original pixel block from the on-chip cache and performing rotation operations on the original coordinates of the plurality of pixel points to obtain target coordinates of the plurality of pixel points includes:
[0019] Loading the plurality of pixels included in the original pixel block from the on-chip cache into a register in a cache line aligned manner;
[0020] A rotation operation is performed on each pixel point in the register to obtain target coordinates of the multiple pixel points.
[0021] Optionally, reading the plurality of pixel points included in the original pixel block from the on-chip cache and performing rotation operations on the original coordinates of the plurality of pixel points to obtain target coordinates of the plurality of pixel points includes:
[0022] For the multiple pixel points included in the original pixel block, perform the following operations respectively:
[0023] Loading a pixel from the on-chip cache into a register;
[0024] A rotation operation is performed on the pixel point in the register to obtain a target coordinate of the pixel point.
[0025] Optionally, writing the plurality of pixel points to the on-chip cache respectively according to the target coordinates of the plurality of pixel points to obtain the target pixel block includes:
[0026] According to the target coordinates of the multiple pixel points, the multiple pixel points are respectively written out from the register to the on-chip cache to obtain a target pixel block.
[0027] Optionally, the on-chip cache includes: an input cache for storing the one original pixel block, and an output cache for storing the target pixel block.
[0028] In one aspect, an embodiment of the present application provides a device for executing a rotation operator, comprising:
[0029] The division module is used to divide the image to be processed in the video memory into multiple original pixel blocks, and perform the following operations on the multiple original pixel blocks:
[0030] A loading module, configured to load an original pixel block from the video memory into an on-chip cache;
[0031] a calculation module, configured to read a plurality of pixel points included in the original pixel block from the on-chip cache, and perform rotation operations on the original coordinates of the plurality of pixel points to obtain target coordinates of the plurality of pixel points; and write the plurality of pixel points to the on-chip cache according to the target coordinates of the plurality of pixel points to obtain a target pixel block;
[0032] A write module is used to write the target pixel block from the on-chip cache to the video memory.
[0033] Optionally, among the multiple pixel points included in the one original pixel block, there are at least two pixel points whose positions in the video memory are continuous;
[0034] The loading module is specifically used for:
[0035] A memory merge access method is used to load an original pixel block from the video memory into an on-chip cache.
[0036] Optionally, among the multiple pixel points included in the target pixel block, there are at least two pixel points whose positions written to the video memory are continuous;
[0037] The writing module is specifically used for:
[0038] The target pixel block is written out from the on-chip cache to the video memory using a memory merge access method.
[0039] Optionally, the calculation module is specifically configured to:
[0040] Loading the plurality of pixels included in the original pixel block from the on-chip cache into a register in a cache line aligned manner;
[0041] A rotation operation is performed on each pixel point in the register to obtain target coordinates of the multiple pixel points.
[0042] Optionally, the calculation module is specifically configured to:
[0043] For the multiple pixel points included in the original pixel block, perform the following operations respectively:
[0044] Loading a pixel from the on-chip cache into a register;
[0045] A rotation operation is performed on the pixel point in the register to obtain a target coordinate of the pixel point.
[0046] Optionally, the calculation module is specifically configured to:
[0047] According to the target coordinates of the multiple pixel points, the multiple pixel points are respectively written out from the register to the on-chip cache to obtain a target pixel block.
[0048] Optionally, the on-chip cache includes: an input cache for storing the one original pixel block, and an output cache for storing the target pixel block.
[0049] On the one hand, an embodiment of the present application provides a computer device, including a memory, a processor chip, and a computer program stored in the memory and executable on the processor chip. When the processor chip executes the computer program, the steps of the above-mentioned method for executing the rotation operator are implemented.
[0050] On the one hand, an embodiment of the present application provides a computer-readable storage medium, which stores a computer program that can be executed by a computer device. When the computer program runs on the computer device, the computer device executes the steps of the execution method of the above-mentioned rotation operator.
[0051] On the one hand, an embodiment of the present application provides a computer program product, which includes a computer program stored on a computer-readable storage medium, and the computer program includes program instructions. When the program instructions are executed by a computer device, the computer device executes the steps of the execution method of the above-mentioned rotation operator.
[0052] In an embodiment of the present application, the image to be processed in the video memory is divided into multiple original pixel blocks, each original pixel block includes multiple pixel points, and then the entire original pixel block is loaded into the on-chip cache, thereby increasing the granularity of data transmission between on-chip and off-chip (i.e., between the video memory and the on-chip cache), reducing the frequency of accessing the video memory, and thus reducing memory access pressure.
[0053] Secondly, the pixels in the original pixel block are read from the on-chip cache to perform the rotation operation, and the pixels are written to the on-chip cache according to the rotated target coordinates. When the rotation operation is completed for the entire original pixel block, the rotated target pixel block is written from the on-chip cache to the video memory. In other words, this application uses the on-chip cache to buffer data and utilizes the high on-chip bandwidth to perform the image rotation operation, thereby improving the performance of the image rotation operator. BRIEF DESCRIPTION OF THE DRAWINGS
[0054] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.
[0055] Figure 1 A schematic diagram of the structure of a processor chip provided in an embodiment of the present application;
[0056] Figure 2 A flowchart of a method for executing a rotation operator provided in the related art;
[0057] Figure 3 A flowchart of a method for executing a rotation operator is provided for an embodiment of the present application;
[0058] Figure 4A A schematic diagram of an image format is provided for an embodiment of the present application;
[0059] Figure 4B A schematic diagram of an image format is provided for an embodiment of the present application;
[0060] Figure 5A A flowchart of an image segmentation method is provided for an embodiment of the present application;
[0061] Figure 5B A flowchart of an image segmentation method is provided for an embodiment of the present application;
[0062] Figure 6A A flowchart of a method for executing a rotation operator is provided for an embodiment of the present application;
[0063] Figure 6B A flowchart of a method for executing a rotation operator is provided for an embodiment of the present application;
[0064] Figure 7 A schematic diagram of the structure of an execution device for a rotation operator provided in an embodiment of the present application;
[0065] Figure 8A schematic diagram of the structure of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION
[0066] In order to make the purpose, technical solutions and beneficial effects of the present invention more clearly understood, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.
[0067] refer to Figure 1 , which is a structural diagram of a processor chip applicable to an embodiment of the present application. The processor chip 100 includes at least: a video memory 101, a plurality of programmable multiprocessors 102, wherein the programmable multiprocessor 102 can be a streaming processing cluster (SPC for short).
[0068] Each programmable multiprocessor 102 includes an on-chip cache 103 and multiple computing units 104 . The multiple computing units 104 within the programmable multiprocessor 102 share the on-chip cache 103 .
[0069] Each computing unit 104 includes multiple execution units 105 , and each execution unit 105 includes registers 106 for multiple threads.
[0070] Video memory 101 can be high-bandwidth memory (HBM) or other types of memory. On-chip cache 103 is a temporary memory with a smaller capacity than video memory 101 but faster data transfer speed. On-chip cache 103 can be a general matrix main buffer (GMB).
[0071] Compared to the on-chip cache 103, the register 106 has a smaller capacity but a faster data exchange speed than the on-chip cache 103. The register may be a thread-local register (TLR) or other types of registers.
[0072] In addition to the above-mentioned structure, the processor chip 100 in this application may also include other structures, which are not specifically limited in this application.
[0073] The processor chip 100 may be a central processing unit (CPU), a graphics processing unit (GPU), a general-purpose computing on graphics processing units (GPGPU), a domain-specific architecture (DSA), etc.
[0074] In technical fields like machine vision and image processing, AI chips often need to rotate the entire input image before processing it. To achieve this rotation, the relevant rotation operator loads the input image onto the chip at a pixel granularity, performs the rotation operation at that pixel granularity, and writes the result to the video memory.
[0075] For example, Figure 2 The example shows four pixels included in the input image, namely pixel 00, pixel 01, pixel 10, and pixel 11. Pixel 00 includes three channels, namely (R00, G00, B00); pixel 01 includes three channels, namely (R01, G01, B01); pixel 10 includes three channels, namely (R10, G10, B10); and pixel 11 includes three channels, namely (R11, G11, B11). It should be noted that the input image is stored in the video memory, and the input image also includes other pixels not shown.
[0076] When the input image is rotated 90 degrees counterclockwise using the rotation operator, pixel 00 is first loaded from the video memory into the register. Then the rotation operation is performed on the original coordinates of pixel 00 to obtain the target coordinates of pixel 00. Pixel 00 is written to the video memory according to the target coordinates of pixel 00.
[0077] Then, in the same way, rotation operations are performed on pixel 01, pixel 10, and pixel 11 respectively, and the rotation results are written to the video memory.
[0078] During the execution of the above rotation operator, the input image is loaded from the video memory into the register at the pixel granularity. Therefore, the video memory needs to be accessed frequently, resulting in high memory access pressure and poor performance of the rotation operator.
[0079] In view of this, this application is based on Figure 1 The architecture diagram of the processor chip shown in FIG provides a flow chart of the execution method of the rotation operator, such as Figure 3As shown, the process of the method is executed by a processor chip and includes the following steps:
[0080] Step 301: Divide the image to be processed in the video memory into a plurality of original pixel blocks.
[0081] Specifically, the image to be processed may be split into multiple original pixel blocks according to different segmentation strategies, and the segmentation strategies may specifically be segmentation by row, segmentation by column, etc. Each original pixel block includes multiple pixel points.
[0082] In the present application, the image to be processed may be an image in different image formats, and the attribute information of each image format includes: the number of channels and the number of planars.
[0083] See Table 1 for attribute information of several image formats provided in the embodiments of this application.
[0084] Table 1.
[0085] Image Format Number of channels Number of planes GRAY 1 1 RGB / BGR 3 1 RGBP / BGRP 3 3 NV12 / NV21 3 2 YU12 / YV12 3 3 RGBA / BGRA 4 1
[0086] The following description uses the RGB format and NV12 format in Table 1 as examples, and other image formats are not described in detail.
[0087] For RGB format, see Figure 4A , the number of channels is 3, namely R channel, G channel and B channel. The number of planes is 1, and one plane (i.e. one pixel point) includes: R channel, G channel and B channel.
[0088] For NV12 format, see Figure 4B , the number of channels is 3, namely Y channel, U channel and V channel. The number of planes is 2, one plane (i.e. one pixel point) includes Y channel, and the other plane (i.e. one pixel point) includes U channel and V channel.
[0089] Step 302: For a plurality of original pixel blocks, perform the following steps 3021 to 3024 respectively:
[0090] Specifically, after the image to be processed in the video memory is divided into multiple original pixel blocks, the processor chip can process the multiple original pixel blocks in parallel through multiple programmable multiprocessors 102, that is, multiple programmable multiprocessors 102 execute steps 3021 to 3024 in parallel. Of course, the processor chip can also process multiple original pixel blocks sequentially through a single programmable multiprocessor 102, that is, a single programmable multiprocessor 102 executes steps 3021 to 3024 multiple times. This application does not impose any specific limitations on this.
[0091] Step 3021: Load an original pixel block from the video memory into the on-chip cache.
[0092] In practical applications, an input buffer is allocated for the rotation operator in the on-chip cache. When the rotation operator is executed, the original pixel block is loaded from the video memory into the input buffer in the on-chip cache.
[0093] Step 3022: read multiple pixel points included in an original pixel block from the on-chip cache, and perform rotation operations on the original coordinates of the multiple pixel points to obtain target coordinates of the multiple pixel points.
[0094] Specifically, a pixel point where a rotation operation is performed refers to a plane, and the number of channels contained in a plane is related to the image format. For example, see Figure 4A When the image to be processed is an RGB format image, a plane includes: R channel, G channel and B channel, that is, a pixel point includes: R channel, G channel and B channel.
[0095] Taking a pixel as a whole, a rotation operation is performed on the original coordinates of the pixel to obtain the target coordinates of the pixel after rotation. The rotation angles include but are not limited to: 90 degrees, 280 degrees, 270 degrees, and the rotation direction can be counterclockwise or clockwise.
[0096] In practical applications, corresponding coordinate calculation formulas are derived in advance for different rotation angles and directions. The original coordinates of the pixel point are substituted into the coordinate calculation formula to perform rotation operations and obtain the target coordinates of the pixel point after rotation.
[0097] Several coordinate calculation formulas are exemplified below.
[0098] Set the width and height of the image to be processed before rotation to W and H respectively, and the original coordinates of the pixel before rotation to (h, w).
[0099] For a 90-degree counterclockwise rotation, the corresponding coordinate calculation formula is shown in the following formula (1):
[0100] Rotate90(h,w)=(Ww-1,h)………(1)
[0101] Here, Rotate90 indicates a 90-degree counterclockwise rotation, and (Ww-1, h) indicates the target coordinates of the pixel after rotation.
[0102] For a counterclockwise rotation of 180 degrees, the corresponding coordinate calculation formula is shown in the following formula (2):
[0103] Rotate180(h,w)=(Hh-1,Ww-1)......(2)
[0104] Here, Rotate180 indicates a counterclockwise rotation of 180 degrees, and (Hh-1, Ww-1) indicates the target coordinates of the pixel after rotation.
[0105] For a counterclockwise rotation of 270 degrees, the corresponding coordinate calculation formula is shown in the following formula (3):
[0106] Rotate270(h,w)=(w,Hh-1)………(3)
[0107] Here, Rotate270 indicates a counterclockwise rotation of 270 degrees, and (w, Hh-1) indicates the target coordinates of the pixel after rotation.
[0108] It should be noted that for other rotation angles and rotation directions, corresponding coordinate calculation formulas can also be derived in advance, which will not be repeated here.
[0109] Step 3023 , writing the multiple pixel points to the on-chip cache respectively according to the target coordinates of the multiple pixel points to obtain the target pixel block.
[0110] Specifically, an output buffer is allocated for the rotation operator in the on-chip cache. When the rotation operator is executed, multiple pixel points are loaded into the output buffer in the on-chip cache according to their target coordinates to obtain a target pixel block.
[0111] The input cache for storing the original pixel block and the output cache for storing the target pixel block in the on-chip cache may be the same cache or different caches, and this application does not make any specific limitation on this.
[0112] Step 3024: Write the target pixel block from the on-chip cache to the video memory.
[0113] Specifically, the target pixel block refers to a pixel block obtained by performing a rotation operation on the original pixel block. When the rotation operation is completed on the entire original pixel block to obtain the target pixel block, the entire target pixel block is written out from the on-chip cache to the video memory.
[0114] In an embodiment of the present application, the image to be processed in the video memory is divided into multiple original pixel blocks, each original pixel block includes multiple pixel points, and then the entire original pixel block is loaded into the on-chip cache, thereby increasing the granularity of data transmission between on-chip and off-chip (i.e., between the video memory and the on-chip cache), reducing the frequency of accessing the video memory, and thus reducing memory access pressure.
[0115] Secondly, the pixels in the original pixel block are read from the on-chip cache to perform the rotation operation, and the pixels are written to the on-chip cache according to the rotated target coordinates. When the rotation operation is completed for the entire original pixel block, the rotated target pixel block is written from the on-chip cache to the video memory. In other words, this application uses the on-chip cache to buffer data and utilizes the high on-chip bandwidth to perform the image rotation operation, thereby improving the performance of the image rotation operator.
[0116] In some embodiments, Figure 2 During the execution of the rotation operator shown, the input image is loaded from the video memory into the register at the pixel granularity. Therefore, each time data is loaded from the video memory, data at consecutive locations is not loaded. This prevents the hardware from performing memory merging operations, which in turn leads to high memory access pressure.
[0117] In view of this, the present application divides the image to be processed in the video memory into multiple original pixel blocks, so that among the multiple pixels contained in an original pixel block, at least two pixels are located consecutively in the video memory. Then, a memory merge access method is used to load an original pixel block from the video memory into the on-chip cache.
[0118] Specifically, among the multiple pixel points included in the original pixel block, all the pixel points may be continuous; or some of the pixel points may be continuous and some of the pixel points may be discontinuous (ie, piecewise continuous).
[0119] Since the original pixel block has continuous pixels, a memory merge access method can be used to load an original pixel block from the video memory to the on-chip cache.
[0120] For example, see Figure 5A , split the image to be processed vertically into 3 original pixel blocks, namely: original pixel block 1, original pixel block 2, and original pixel block 3.
[0121] Original pixel block 1 includes pixel 00, pixel 01, pixel 10, pixel 11, ..., pixel 80, and pixel 81. Each pixel includes an R channel, a G channel, and a B channel. In original pixel block 1, pixel 00 and pixel 01 are continuous in the video memory, while pixel 01 and pixel 10 are discontinuous, ..., and pixel 80 and pixel 81 are continuous in the video memory. In other words, original pixel block 1 is piecewise continuous.
[0122] Similarly, the original pixel block 2 and the original pixel block 3 are also piecewise continuous.
[0123] SPC1 uses memory coalescing to load raw pixel block 1 from video memory into the on-chip cache. SPC2 uses memory coalescing to load raw pixel block 2 from video memory into the on-chip cache. SPC3 uses memory coalescing to load raw pixel block 3 from video memory into the on-chip cache.
[0124] For example, see Figure 5B , split the image to be processed horizontally into 3 original pixel blocks, namely: original pixel block 4, original pixel block 5, and original pixel block 6.
[0125] Original pixel block 4 includes pixel 00, pixel 01, ..., pixel 05, pixel 10, pixel 11, ..., pixel 15, pixel 20, pixel 21, ..., pixel 25. Each pixel includes an R channel, a G channel, and a B channel. In original pixel block 1, all pixels are located continuously in the video memory.
[0126] Similarly, the positions of all pixels in the original pixel block 2 in the video memory are continuous, and the positions of all pixels in the original pixel block 3 in the video memory are continuous.
[0127] SPC1 uses memory coalescing to load raw pixel block 4 from video memory into the on-chip cache. SPC2 uses memory coalescing to load raw pixel block 5 from video memory into the on-chip cache. SPC3 uses memory coalescing to load raw pixel block 6 from video memory into the on-chip cache.
[0128] In an embodiment of the present application, when the image to be processed is divided into multiple original pixel blocks, continuous pixel points on the video memory are present in the original pixel blocks, and then the advantages of hardware memory access merging are utilized to load the original pixel blocks from the video memory into the on-chip cache, thereby reducing the pressure on the cache and memory access.
[0129] In this application, when performing rotation operations using on-chip cache, there are at least the following implementations:
[0130] In some embodiments, for multiple pixel points included in an original pixel block, the following operations are performed respectively: loading a pixel point from an on-chip cache to a register; performing a rotation operation on the pixel point in the register to obtain the target coordinates of the pixel point.
[0131] According to the target coordinates of the multiple pixel points, the multiple pixel points are written out from the register to the on-chip cache respectively to obtain the target pixel block.
[0132] Specifically, each time a pixel is loaded from the on-chip cache into a thread's register, the thread reads the pixel from the register and calculates its coordinates using the coordinate calculation formula described above to obtain the target coordinates. The pixel is then written from the register to the on-chip cache based on the target coordinates.
[0133] In an embodiment of the present application, the entire pixel block is first loaded from the video memory to the on-chip cache, and then the individual pixel points in the pixel block are loaded from the on-chip cache to the register for calculation. Compared with the path of loading pixel points from the video memory to the register, the path of loading pixel points from the on-chip cache to the register is shorter, and therefore, memory access is more efficient.
[0134] In some embodiments, multiple pixels contained in an original pixel block are loaded from an on-chip cache into a register in a cache line-aligned manner, and then a rotation operation is performed on each pixel in the register to obtain target coordinates of the multiple pixels. Based on the target coordinates of the multiple pixels, the multiple pixels are written from the register to the on-chip cache to obtain a target pixel block.
[0135] Specifically, a cache line is the smallest unit of storage managed by the cache. When the cache line size doesn't match the memory size of a pixel, bandwidth is wasted. For example, if a cache line is 4 bytes and each pixel occupies 3 bytes of memory, loading a pixel through a cache line results in 1 byte of wasted bandwidth.
[0136] In view of this, the present application loads multiple pixels contained in the original pixel block from the on-chip cache into the register in a cache line aligned manner. In other words, the data structure during loading is redesigned so that the data structure can be completely accommodated in the cache line.
[0137] For example, if you set the load time to 4 pixels, that is, 12 bytes of data, then in each load process, 4 pixels can be loaded through 3 cache lines without wasting bandwidth.
[0138] Each time a pixel is loaded from the on-chip cache into the register, the pixel is read from the register and the coordinates are calculated to obtain the target coordinates of the pixel. Then, according to the target coordinates of the pixel, the pixel is written from the register to the on-chip cache.
[0139] In an embodiment of the present application, the rotation operation is performed using the on-chip cache, and data is moved on the on-chip cache in a cache line aligned manner, so that memory access within the on-chip cache is optimized, thereby improving memory access efficiency and bandwidth utilization.
[0140] In some embodiments, after performing a rotation operation on the original pixel block to obtain a target pixel block, at least two of the multiple pixels contained in the target pixel block have consecutive locations when written to the video memory. Therefore, a memory merge access method is used to write the target pixel block from the on-chip cache to the video memory.
[0141] Specifically, among the multiple pixel points contained in the target pixel block, the positions of all pixel points written to the video memory may be continuous; or the positions of some pixel points written to the video memory may be continuous, and the positions of some pixel points written to the video memory may be discontinuous (i.e., segmented continuous).
[0142] Since there are pixels in the target pixel block that are written to the video memory at consecutive locations, a memory merge access method can be used to write the target pixel block from the on-chip cache to the video memory.
[0143] For example, see Figure 6A , set to use memory merge access mode, and load an original pixel block from the video memory into the input buffer of the on-chip cache. The original pixel block includes: pixel point 00, pixel point 01, pixel point 10, pixel point 11, pixel point 20, pixel point 21, where the positions of pixel point 00 and pixel point 01 in the video memory are continuous, the positions of pixel point 01 and pixel point 10 in the video memory are discontinuous, and the positions of pixel point 10 and pixel point 11 in the video memory are continuous; the positions of pixel point 11 and pixel point 20 in the video memory are discontinuous, and the positions of pixel point 20 and pixel point 21 in the video memory are continuous.
[0144] Read pixel 00 from the input buffer into the register. Then, perform a 90-degree counterclockwise rotation of the original coordinates of pixel 00 to obtain the target coordinates of pixel 00. Pixel 00 is written from the register to the output buffer in the on-chip cache at the target coordinates. Perform the rotation operation on the remaining pixels in the same manner to obtain the target coordinates. These are then written to the output buffer at the target coordinates to obtain the target pixel block.
[0145] In the target pixel block, the positions of all pixels written to the video memory are continuous, that is, the positions of pixel 01, pixel 11, pixel 21, pixel 00, pixel 10 and pixel 20 written to the video memory are all continuous. Therefore, the memory merge access method is used to load the target pixel block from the on-chip cache to the video memory.
[0146] For example, see Figure 6B, set up memory coalescing access to load a raw pixel block from the video memory into the input buffer in the on-chip cache. The raw pixel block consists of: pixel 00, pixel 01, pixel 02, pixel 03, pixel 10, pixel 11, pixel 12, pixel 13. These six pixels are located consecutively in the video memory.
[0147] Read pixel 00 from the input buffer into the register. Then, perform a 90-degree counterclockwise rotation of the original coordinates of pixel 00 to obtain the target coordinates of pixel 00. Pixel 00 is written from the register to the output buffer in the on-chip cache at the target coordinates. Perform the rotation operation on the remaining pixels in the same manner to obtain the target coordinates. These are then written to the output buffer at the target coordinates to obtain the target pixel block.
[0148] In the process of writing the target pixel block from the on-chip cache to the video memory, the positions where pixel 03 and pixel 13 are written to the video memory are continuous; the positions where pixel 13 and pixel 02 are written to the video memory are discontinuous; the positions where pixel 02 and pixel 12 are written to the video memory are continuous; the positions where pixel 12 and pixel 01 are written to the video memory are discontinuous; the positions where pixel 01 and pixel 11 are written to the video memory are continuous; the positions where pixel 11 and pixel 00 are written to the video memory are discontinuous; and the positions where pixel 00 and pixel 10 are written to the video memory are continuous.
[0149] That is to say, the location where the target pixel block is written out to the video memory is segmented and continuous. Therefore, the memory merge access method is adopted to load the target pixel block 1 from the on-chip cache to the video memory.
[0150] In an embodiment of the present application, a rotation operation is performed on the original pixel block to obtain a target pixel block. The position of writing the target pixel block to the video memory is completely continuous or segmented continuous. Therefore, the target pixel block is written back to the video memory from the on-chip cache by using a hardware memory access merging method, thereby effectively improving the memory access efficiency and reducing the memory access pressure. Secondly, the present application not only optimizes the memory access between the video memory and the on-chip cache, but also optimizes the memory access within the on-chip cache, which greatly reduces the memory access pressure and improves the performance of the image rotation operator. In addition, the memory access optimization between the video memory and the on-chip cache in the present application is independent of the memory access optimization within the on-chip cache and does not interfere with each other, so it is more convenient for targeted optimization and programming.
[0151] Based on the same technical concept, the embodiment of the present application provides a structural diagram of an execution device of a rotation operator, such as Figure 7 As shown, the execution device 700 of the rotation operator includes:
[0152] The division module 701 is configured to divide the image to be processed in the video memory into a plurality of original pixel blocks, and perform the following operations on each of the plurality of original pixel blocks:
[0153] A loading module 702 is configured to load an original pixel block from the video memory into an on-chip cache;
[0154] The calculation module 703 is configured to read the plurality of pixels included in the original pixel block from the on-chip cache, perform rotation operations on the original coordinates of the plurality of pixels to obtain target coordinates of the plurality of pixels, and write the plurality of pixels to the on-chip cache according to the target coordinates of the plurality of pixels to obtain a target pixel block.
[0155] The writing module 704 is configured to write the target pixel block from the on-chip cache to the video memory.
[0156] Optionally, among the multiple pixel points included in the one original pixel block, there are at least two pixel points whose positions in the video memory are continuous;
[0157] The loading module 702 is specifically used for:
[0158] A memory merge access method is used to load an original pixel block from the video memory into an on-chip cache.
[0159] Optionally, among the multiple pixel points included in the target pixel block, there are at least two pixel points whose positions written to the video memory are continuous;
[0160] The writing module 704 is specifically used for:
[0161] The target pixel block is written out from the on-chip cache to the video memory using a memory merge access method.
[0162] Optionally, the calculation module 703 is specifically configured to:
[0163] Loading the plurality of pixels included in the original pixel block from the on-chip cache into a register in a cache line aligned manner;
[0164] A rotation operation is performed on each pixel point in the register to obtain target coordinates of the multiple pixel points.
[0165] Optionally, the calculation module 703 is specifically configured to:
[0166] For the multiple pixel points included in the original pixel block, perform the following operations respectively:
[0167] Loading a pixel from the on-chip cache into a register;
[0168] A rotation operation is performed on the pixel point in the register to obtain a target coordinate of the pixel point.
[0169] Optionally, the calculation module 703 is specifically configured to:
[0170] According to the target coordinates of the multiple pixel points, the multiple pixel points are respectively written out from the register to the on-chip cache to obtain a target pixel block.
[0171] Optionally, the on-chip cache includes: an input cache for storing the one original pixel block, and an output cache for storing the target pixel block.
[0172] In an embodiment of the present application, the image to be processed in the video memory is divided into multiple original pixel blocks, each original pixel block includes multiple pixel points, and then the entire original pixel block is loaded into the on-chip cache, thereby increasing the granularity of data transmission between on-chip and off-chip (i.e., between the video memory and the on-chip cache), reducing the frequency of accessing the video memory, and thus reducing memory access pressure.
[0173] Secondly, the pixels in the original pixel block are read from the on-chip cache to perform the rotation operation, and the pixels are written to the on-chip cache according to the rotated target coordinates. When the rotation operation is completed for the entire original pixel block, the rotated target pixel block is written from the on-chip cache to the video memory. In other words, this application uses the on-chip cache to buffer data and utilizes the high on-chip bandwidth to perform the image rotation operation, thereby improving the performance of the image rotation operator.
[0174] Based on the same technical concept, the embodiment of the present application provides a computer device, such as Figure 8 As shown, it includes at least one processor chip 100 and a memory 801 connected to the at least one processor chip 100. The specific connection medium between the processor chip 100 and the memory 801 is not limited in the embodiment of the present application. Figure 8 For example, the processor chip 100 and the memory 801 are connected via a bus. The bus can be divided into an address bus, a data bus, a control bus, and the like.
[0175] In an embodiment of the present application, the memory 801 stores instructions that can be executed by at least one processor chip 100. By executing the instructions stored in the memory 801, the at least one processor chip 100 can perform the steps of the above-mentioned rotation operator execution method.
[0176] Among them, the processor chip 100 is the control center of the computer device, which can use various interfaces and lines to connect various parts of the computer device, and realize the execution of the rotation operator by running or executing instructions stored in the memory 801 and calling data stored in the memory 801. Optionally, the processor chip 100 may include one or more processing units. The processor chip 100 may integrate an application processor and a modem processor, wherein the application processor mainly processes the operating system, user interface, and application programs, etc., and the modem processor mainly processes wireless communications. It is understandable that the above-mentioned modem processor may not be integrated into the processor chip 100. In some embodiments, the processor chip 100 and the memory 801 can be implemented on the same chip. In some embodiments, they can also be implemented separately on independent chips.
[0177] The processor chip 100 can be a general-purpose processor, such as a central processing unit (CPU), a digital signal processor, an application-specific integrated circuit (ASIC), a field programmable gate array or other programmable logic device, a discrete gate or transistor logic device, or a discrete hardware component, and can implement or execute the various methods, steps, and logic block diagrams disclosed in the embodiments of the present application. A general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in the embodiments of the present application can be directly embodied as being executed by a hardware processor, or can be executed by a combination of hardware and software modules in the processor.
[0178] Memory 801 is a kind of non-volatile computer readable storage medium, which can be used for storing non-volatile software programs, non-volatile computer executable programs and modules.Memory 801 may include at least one type of storage medium, such as a flash memory, a hard disk, a multimedia card, a card-type memory, a random access memory (Random Access Memory, RAM), a static random access memory (Static Random Access Memory, SRAM), a programmable read-only memory (Programmable Read Only Memory, PROM), a read-only memory (Read Only Memory, ROM), an electrically erasable programmable read-only memory (Electrically Erasable Programmable Read-Only Memory, EEPROM), a magnetic storage device, a disk, an optical disk, etc. Memory 801 is any other medium that can be used to carry or store a desired program code in the form of an instruction or data structure and can be accessed by a computer device, but is not limited thereto. The memory 801 in the embodiment of the present application can also be a circuit or other arbitrarily capable of realizing a storage function, for storing program instructions and / or data.
[0179] Based on the same inventive concept, an embodiment of the present application provides a computer-readable storage medium, which stores a computer program that can be executed by a computer device. When the computer program runs on the computer device, the computer device executes the steps of the execution method of the above-mentioned rotation operator.
[0180] Based on the same inventive concept, an embodiment of the present application provides a computer program product, which includes a computer program stored on a computer-readable storage medium, and the computer program includes program instructions. When the program instructions are executed by a computer device, the computer device executes the steps of the execution method of the above-mentioned rotation operator.
[0181] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods or computer program products. Thus, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.
[0182] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as a combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer device or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.
[0183] These computer program instructions may also be stored in a computer readable memory that can direct a computer device or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.
[0184] These computer program instructions can also be loaded onto a computer device or other programmable data processing device so that a series of operating steps are executed on the computer device or other programmable device to produce a process implemented by the computer device, thereby providing instructions for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.
[0185] Although the preferred embodiments of the present invention have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present invention.
[0186] Obviously, those skilled in the art may make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if such changes and modifications fall within the scope of the claims and their equivalents, the present invention is intended to include such changes and modifications.
Claims
1. A method for executing a rotation operator, characterized in that: Applied to GPU, including: The image to be processed in the video memory is divided into a plurality of original pixel blocks, and the following operations are performed on the plurality of original pixel blocks: Allocating an input buffer for the rotation operator in an on-chip cache; loading an original pixel block from the video memory into the input buffer using a memory merge access method, wherein at least two of the multiple pixel points included in the original pixel block are located consecutively in the video memory; Reading a plurality of pixel points included in the original pixel block from the input buffer, and performing rotation operations on the original coordinates of the plurality of pixel points respectively to obtain target coordinates of the plurality of pixel points; Allocating an output buffer for the rotation operator in the on-chip cache; writing the plurality of pixel points to the output buffer respectively according to target coordinates of the plurality of pixel points to obtain a target pixel block; wherein positions of at least two pixel points written to the video memory are consecutive among the plurality of pixel points included in the target pixel block; The target pixel block is written out from the output buffer to the video memory by using a memory merge access method.
2. The method according to claim 1, wherein The step of reading the plurality of pixels included in the original pixel block from the input buffer and performing rotation operations on the original coordinates of the plurality of pixels to obtain target coordinates of the plurality of pixels includes: Loading the plurality of pixels included in the original pixel block from the input cache into a register in a cache line aligned manner; A rotation operation is performed on each pixel point in the register to obtain target coordinates of the multiple pixel points.
3. The method according to claim 1, wherein The step of reading the plurality of pixels included in the original pixel block from the input buffer and performing rotation operations on the original coordinates of the plurality of pixels to obtain target coordinates of the plurality of pixels includes: For the multiple pixel points included in the original pixel block, perform the following operations respectively: Loading a pixel from the input buffer into a register; A rotation operation is performed on the pixel point in the register to obtain a target coordinate of the pixel point.
4. The method according to claim 2 or 3, wherein: Writing the plurality of pixel points to the input buffer respectively according to the target coordinates of the plurality of pixel points to obtain a target pixel block includes: According to the target coordinates of the multiple pixel points, the multiple pixel points are respectively written out from the register to the input buffer to obtain a target pixel block.
5. A computer device comprising a memory, a processor chip, and a computer program stored in the memory and executable on the processor chip, wherein: When the processor chip executes the computer program, the steps of the method according to any one of claims 1 to 4 are implemented.
6. A computer-readable storage medium, characterized in that It stores a computer program that can be executed by a computer device. When the computer program is run on the computer device, the computer device executes the steps of any one of the methods according to claims 1 to 4.
7. A computer program product, characterized in that The computer program product comprises a computer program stored on a computer-readable storage medium, wherein the computer program comprises program instructions. When the program instructions are executed by a computer device, the computer device is caused to perform the steps of the method according to any one of claims 1 to 4.
Citation Information
Patent Citations
Image partitioned processing method and device and computer readable storage medium
CN107610047A
Data processing method and device based on hardware characteristics
CN116126750A
Data access method and device, storage medium and program product
CN117971501A