A pixel shuffle implementation method based on simd instruction
By implementing the PixelShuffle algorithm using the SIMD instruction set on the Beijing Junzheng chip, the problem of not being able to run Python algorithms on the chip was solved, achieving efficient feature map upsampling and improving the calculation speed.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- INGENIC SEMICON CO LTD
- Filing Date
- 2022-03-28
- Publication Date
- 2026-08-04
AI Technical Summary
The PixelShuffle algorithm in existing deep learning frameworks cannot run on the chips of Beijing Junzheng Integrated Circuit Co., Ltd., and the algorithm implemented in Python is slow on resource-limited platforms, resulting in low computational efficiency.
The PixelShuffle algorithm is implemented using SIMD instruction sets, especially the Ingenic_simd512 instruction set, through data loading, sorting, and storage instructions to reassemble low-resolution feature maps to obtain high-resolution feature maps.
The same PixelShuffle algorithm results as the Python implementation were achieved on the Beijing Junzheng chip, and the calculation speed was significantly improved, with a speedup of 5 times compared to the C implementation of the PixelShuffle algorithm.
Smart Images
Figure CN116860330B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of neural network technology, and in particular to a PixelShuffle implementation method based on SIMD instructions. Background Technology
[0002] In deep learning neural networks, multiple convolutional operations reduce the resolution of feature maps, resulting in less extracted feature information. To retain more feature information, feature fusion is performed between the low-resolution convolutional feature map and the original feature map. This requires restoring the low-resolution convolutional feature map to its original size; this operation is called upsampling. PixelShuffle (also known as Sub-Pixel Convolutional Neural Network) is an upsampling method that effectively enlarges scaled-down feature maps. Its main function is to reorganize low-resolution feature maps into high-resolution feature maps through channel pixel reassembly, making it an effective upsampling technique. The PixelShuffle algorithm takes a Tensor-type feature map as input, where the Tensor's shape is... H is the height of the feature map, and W is the width of the feature map. The number of channels in the feature map is used to reorder the data and output a shape of... Feature map, For the height of the new feature map, Let C be the width of the new feature map, C be the number of channels in the new feature map, and r be the upsampling factor, which is the image magnification. This is because it is necessary to upsample the low-resolution feature map... Zoom in PixelShuffle rearranges the Tensor data, changing the original feature map... Each pixel in a direction is considered a grid, and each grid is expanded to... Each grid represents a pixel in the original feature map along its channel direction. The data are arranged in order here. In each cell, the width and height are increased by r times, and the passage becomes the original... This allows for the upsampling operation to be performed by obtaining a high-resolution feature map.
[0003] Currently, deep learning frameworks like PyTorch, MXNet, and TensorFlow all use corresponding implementations of PixelShuffle. Taking PyTorch as an example, the PixelShuffle algorithm is primarily implemented using the `nn.PixelShuffle` function. The following is an example from the official documentation for reference:
[0004] ps = nn.PixelShuffle(3)
[0005] input = autograd.Variable(torch.Tensor(1, 9, 4, 4))
[0006] output = ps(input)
[0007] print(output.size())
[0008] torch.Size([1, 1, 12, 12])
[0009] Call the nn.PixelShuffle function and pass in an upsampling factor of 3 to get a ps instance. Define a Tensor of type (1,9,4,4,) which is a feature map input with a height and width of 4 and a channel of 9. Pass the input to ps to get a new feature map output. Print the size of the new feature map, which has a height and width of 12, both of which are 3 times larger, and a channel of 1, which is 1 / 9 of the original.
[0010] However, existing technologies have the following drawbacks:
[0011] 1. The PixelShuffle algorithm in existing deep learning frameworks is implemented in Python and can run on PCs. However, the Python-implemented algorithm cannot run on some chips used by chip manufacturers to run deep learning neural networks, such as chips from Beijing Junzheng Integrated Circuit Co., Ltd. (hereinafter referred to as Beijing Junzheng).
[0012] 2. Python's underlying language is C. Running C on resource-constrained platforms or chips will result in a lot of cache waste, leading to slow processing speed. For some applications that require high processing speed, the algorithms implemented in Python are not the best choice.
[0013] In addition, the following are commonly used terms in the prior art:
[0014] SIMD stands for Single Instruction Multiple Data, which is a set of instructions that can process multiple operands at once and pack them into a large register. Summary of the Invention
[0015] To address the problems in the prior art, the purpose of this application is to propose an implementation method for the PixelShuffle algorithm based on the SIMD instruction set. This method achieves the same functionality as the PixelShuffle algorithm implemented in Python by combining SIMD instructions. It can efficiently run the PixelShuffle algorithm on specific chips, such as the chips used by Beijing Junzheng for running deep learning neural networks, to upsample the low-resolution feature maps after convolution to obtain high-resolution feature maps.
[0016] Specifically, this invention provides a PixelShuffle implementation method based on SIMD instructions, wherein the method uses a PixelShuffle of size [missing information]. The low-resolution feature map is obtained by reordering the data in the register according to certain rules using two data sorting instructions in the SIMD set, namely Ingenic_simd512_shuffle and Ingenic_simd512_gather_byte, and then recombining them to obtain a size of The high-resolution feature map, where H is the height of the feature map and W is the width of the feature map. denoted as the number of channels in the feature map, C is the number of channels in the new feature map, and r is the upsampling factor, which is the image magnification factor. The PixelShuffle algorithm is run on a chip using the SIMD instruction set.
[0017] The method further includes the following steps:
[0018] S1, Initialization Settings:
[0019] Let w and h represent looping in the width and height directions of the input feature map, respectively;
[0020] The input feature map has a height of in_height, a width of in_width, a data pointer of input_raddr, row data pointers (height direction) of input_raddr_rows, and column data pointers (width direction) of input_raddr_cols. Figure 1 The row data size is the width multiplied by the number of channels, denoted as one_in_line_size. Adding one_in_line_size to a data pointer of the input feature map indicates jumping to the corresponding data pointer of the next row.
[0021] The output feature map width is `out_width`, the row data (i.e., pointers in the height direction) is `output_raddr_rows`, and the column data (i.e., pointers in the width direction) is `output_raddr_cols`. The output feature map... Figure 1The row data size is the width multiplied by the number of channels, denoted as one_out_line_size. Adding one_out_line_size to a data pointer in the output feature map indicates jumping to the corresponding data pointer in the next row.
[0022] S2, enter the first loop, looping along the height direction of the input feature map:
[0023] Before the loop, h=0, indicating that the loop starts from a height of 1 and continues until the height is in_height. After each loop, h is incremented by one, indicating that each loop iteration is based on one height data point, and the loop repeats a total of in_height times.
[0024] In a loop, w=0 means that a new loop should start every time the loop width is entered.
[0025] The loop condition is: check if h < in_height is true. If not, exit the first loop; if true, execute the first loop once.
[0026] When entering the loop, the pointer to the input feature map row is assigned to the pointer to the first column of data in that row, input_raddr_cols, by setting input_raddr_cols = input_raddr_rows.
[0027] The pointer to the first column of the output feature map is obtained by setting output_waddr_cols = output_waddr_rows;
[0028] Before the end of each loop, the input feature map row pointer is moved to the next row by adding `input_raddr_rows += one_in_line_size`.
[0029] output_waddr_rows += sampling factor The output feature map row pointer jumps to the sampling factor value row, and in this way, one row of input feature map data is processed in one loop to obtain the data of the output feature map in the sampling factor value row;
[0030] S3, execute the second loop, looping along the width of the input feature map:
[0031] Before the loop, set w=0, indicating that the loop starts from a width of 1 and continues until the width is in_width. After each loop, increment w by one, indicating that each loop iteration is for one width value, and the loop repeats for a total of in_width times.
[0032] The condition for the second loop is: check if w < in_width is true. If not, exit the second loop; if true, execute:
[0033] The second loop needs to process the row data from the first loop. The second loop loads one column of data at a time. Column data composed of channel data;
[0034] Executing data loading instructions in the SIMD set is denoted as:
[0035] Register1 = Ingenic_simd512_load(input_raddr_cols), where
[0036] Ingenic_simd512_load is a data loading instruction. Its function is to load data into a register. A register can be loaded with 512 bits of data at a time.
[0037] Since the input feature map data type is unsigned 8-bit, a maximum of 64 data can be loaded at a time; the parameter input_raddr_cols represents column data containing 32 channels. This instruction loads these 32 data into positions 1-32 of register Register1, while positions 33-64 of Register1 are empty.
[0038] Executing data sorting instructions in a SIMD set is denoted as:
[0039] Register2 = Ingenic_simd512_shuffle_4_group(Register1), where
[0040] Ingenic_simd512_shuffle_4_group is a data sorting instruction, which reorders the data in the register according to certain rules;
[0041] This instruction operates on word-type data, i.e., four 8-bit data units. The 64 8-bit data in Register1 are transformed into 16 words. These 16 words are divided into four groups. The first word of each group is taken to form a new group and placed in positions 1-4 of Register2. This process is repeated, with the second word of each group being taken to form a new group and placed in positions 5-8 of Register2, the third word in positions 9-12, and the fourth word in positions 13-16. After the operation, the data in Register2 is treated as 8-bit data units. At this point, positions 1-8 in Register2 correspond to positions 1-4 and 17-20 in Register1, positions 17-24 in Register2 correspond to positions 5-8 and 21-24 in Register1, positions 33-40 in Register2 correspond to positions 9-12 and 25-28 in Register1, and positions 49-56 in Register2 correspond to positions 13-16 and 29-32 in Register1.
[0042] Execute another data sorting instruction in the SIMD set, denoted as:
[0043] Register2 = Ingenic_simd512_gather_byte(Register2), where
[0044] Ingenic_simd512_gather_byte is also a data sorting instruction. This instruction treats every two words of data as a group, placing the 8 bits of data at even positions in the first word and the 8 bits of data at odd positions in the second word.
[0045] After performing this operation on the 8 pairs of words in Register2, the positions 1-8 in Register2 are now 1, 3, 17, 19, 2, 4, 18, 20 in Register1; the positions 17-24 in Register2 are now 5, 7, 21, 23, 6, 8, 22, 24 in Register1; the positions 33-40 in Register2 are now 9, 11, 25, 27, 10, 12, 26, 28 in Register1; and the positions 49-56 in Register2 are now 13, 15, 29, 31, 14, 16, 30, 32 in Register1.
[0046] After performing another Ingenic_simd512_gather_byte operation, positions 1-8 in Register2 are 1, 17, 2, 18, 3, 19, 4, 20 in Register1; positions 17-24 in Register2 are 5, 21, 6, 22, 7, 23, 8, 24 in Register1; positions 33-40 in Register2 are 9, 25, 10, 26, 11, 27, 12, 28 in Register1; and positions 49-56 in Register2 are 13, 29, 14, 30, 15, 31, 16, 32 in Register1.
[0047] Executing a shift instruction in SIMD is denoted as:
[0048] Register3 = Ingenic_simd512_logical_shift_right_16_bit(Register2), where Ingenic_simd512_logical_shift_right_16_bit is a shift instruction that logically shifts each word of data in Register2 to the right by 16 bits. The resulting data is stored in Register3. At this point, positions 1-2 in Register3 are 2 and 18 from Register1, positions 5-6 in Register3 are 4 and 20 from Register1, positions 17-18 in Register3 are 6 and 22 from Register1, positions 21-22 in Register3 are 8 and 24 from Register1, positions 33-34 in Register3 are 10 and 26 from Register1, positions 37-38 in Register3 are 12 and 28 from Register1, positions 49-50 in Register3 are 14 and 30 from Register1, and positions 53-54 in Register3 are 16 and 32 from Register1.
[0049] Executing data storage instructions in SIMD is denoted as:
[0050] output_waddr_cols=Ingenic_simd512_store(Register2,Register3), where Ingenic_simd512_store is a data storage instruction. Its function is to store the data in registers Register2 and Register3 in the four columns of the corresponding row pointer of the output feature map. Each column contains two channel data and the four columns of the second, third and fourth rows of the current row.
[0051] Before the second loop ends, the input feature map column pointer is incremented by the number of channels using `input_raddr_cols += number of channels`, which means it points to the next column of data.
[0052] By += sampling factor of output_waddr_cols The number of channels will be the output feature map column pointer plus a sampling factor. The channel number is indicated by a pointer to the column data that needs to be stored in the next loop.
[0053] In the method described, the upsampling factor is set to 4, the width and height of the input feature map are both 10, and the number of channels is 32. The column data consists of 32 channels, and the pixel data type is unsigned 8-bit, i.e., uint8_t. Through the sorting instructions Ingenic_simd512_shuffle and Ingenic_simd512_gather_byte, an output feature map with a width and height of 40 and 2 channels is obtained.
[0054] The method described above applies to feature map data storage of the Tensor type, and its storage method is contiguous storage. Each channel's data forms a column of data, i.e., data in the width direction, and is then stored consecutively. Each data point constitutes a row of data, representing the data along the height direction, and is then stored contiguously. The size of the entire feature map is determined by these data points.
[0055] The method uses a SIMD instruction set that can load 512 bits of data into a register at once, and can perform the same operation on 512 bits of data at once during computation. This method uses the data loading, data storage, shifting, and data sorting instructions in the SIMD instruction set.
[0056] Therefore, the advantage of this application lies in the following: The PixelShuffle upsampling algorithm is implemented based on the SIMD instruction set using the method described in this application. On chips that cannot run Python, such as the Beijing Junzheng chip, the PixelShuffle algorithm achieves the same results as the Python-implemented PixelShuffle algorithm. Here, the SIMD instruction can load 512 bits of data at a time, and a single instruction can perform calculations on 512 bits of data simultaneously, greatly improving the calculation speed. It can run efficiently on chips with limited hardware resources and high computational speed requirements. In comparison, under the same chip conditions, the SIMD-implemented PixelShuffle algorithm is 5 times faster than the C-implemented PixelShuffle algorithm. Attached Figure Description
[0057] The accompanying drawings, which are provided to further illustrate the invention and form part of this application, are not intended to limit the scope of the invention.
[0058] Figure 1 This is a flowchart of the method of the present invention.
[0059] Figure 2 This is a schematic diagram of the core code involved in the embodiments of the present invention. Detailed Implementation
[0060] To better understand the technical content and advantages of the present invention, the present invention will now be described in further detail with reference to the accompanying drawings.
[0061] This invention relates to a PixelShuffle implementation method based on SIMD instructions. The SIMD instruction set used in this embodiment is the Ingenic_simd512 instruction set from Beijing Junzheng Integrated Circuit Co., Ltd. (hereinafter referred to as Beijing Junzheng). This instruction set can load 512 bits of data into a register at once, and perform the same operation on 512 bits of data simultaneously during computation. This instruction set includes data loading and data storage instructions; arithmetic operations such as addition, subtraction, and multiplication; logical operations such as AND, OR, NOT, and XOR; shift operations such as logical left shift, logical right shift, and arithmetic right shift; and permutation operations such as SHUF, GATHER, and move. This method utilizes data loading, data storage, shifting, and data sorting instructions.
[0062] Tensor-type feature map data is stored contiguously. Each channel's data forms a column of data, i.e., data in the width direction, and is then stored consecutively. Each data point constitutes a row of data, representing the data along the height direction, and is then stored contiguously. The size of the entire feature map is determined by these data points.
[0063] In this embodiment, the upsampling factor is 4, the input feature map has a width and height of 10, 32 channels, and the pixel data type is unsigned 8-bit (uint8_t). This method uses the sorting instructions Ingenic_simd512_shuffle and Ingenic_simd512_gather_byte to obtain an output feature map with a width and height of 40 and 2 channels.
[0064] like Figure 1 , 2 As shown, the method further includes the following steps:
[0065] S1, Initialization Settings:
[0066] Let w and h represent looping in the width and height directions of the input feature map, respectively;
[0067] The input feature map has a height of in_height, a width of in_width, a data pointer of input_raddr, row data pointers (height direction) of input_raddr_rows, and column data pointers (width direction) of input_raddr_cols. Figure 1 The row data size is the width multiplied by the number of channels, one_in_line_size. Adding one_in_line_size to a data pointer in the input feature map indicates jumping to the corresponding data pointer in the next row.
[0068] The output feature map width is `out_width`, the row data (i.e., pointers in the height direction) is `output_raddr_rows`, and the column data (i.e., pointers in the width direction) is `output_raddr_cols`. The output feature map... Figure 1 The row data size is the width multiplied by the number of channels, one_out_line_size. Adding one_out_line_size to a data pointer in the output feature map indicates jumping to the corresponding data pointer in the next row.
[0069] S2, the first loop iterates along the height direction of the input feature map. Before the loop, h=0, indicating that the loop starts from height 1 and continues until the height is in_height. After each loop, h is incremented by one, indicating that each iteration iterates through one height data point, for a total of in_height iterations. During the loop, w=0, indicating that a new loop begins each time the loop is entered along the width direction. Within each loop, upon entering the loop, the input feature map row pointer is assigned to the pointer to the first column of the current row, input_raddr_cols, using input_raddr_cols = input_raddr_rows. Similarly, the output feature map row pointer is obtained and assigned to the pointer to the first column of the current row, output_waddr_cols, using output_waddr_cols = output_waddr_rows. Before the end of each loop, the input feature map row pointer is moved to the next row using input_raddr_rows += one_in_line_size. Jump the output feature map row pointer four rows, and in this way, process one row of input feature map data in one loop to obtain four rows of output feature map data.
[0070] S3, the second loop iterates along the width of the input feature map. The loop starts with a width of 1 and continues until the width is `in_width`. After each loop, `w` is incremented by one, indicating that one width of data is processed each time. The loop iterates a total of `in_width` times. The second loop needs to process the row data from the first loop; it loads one column of data at a time. The column data consists of 32 channels in this application.
[0071] The command `Register1 = Ingenic_simd512_load( input_raddr_cols )` is executed, where...
[0072] `Ingenic_simd512_load` is a data loading instruction. Its function is to load data into a register. A register can load 512 bits of data at a time. Since the input feature map data type is unsigned 8 bits, a maximum of 64 data items can be loaded at once. `input_raddr_cols` represents 32 columns of data. This instruction loads these 32 data items into positions 1-32 of register `Register1`, while positions 33-64 of `Register1` remain empty.
[0073] Execute `Register2=Ingenic_simd512_shuffle_4_group(Register1)`, where...
[0074] Ingenic_simd512_shuffle_4_group is a data sorting instruction that reorders the data in a register according to certain rules. This instruction operates on word-type data, i.e., four 8-bit data units. The 64 8-bit data in Register1 are transformed into 16 words. These 16 words are divided into four groups. The first word of each group is taken to form a new group and placed in positions 1-4 of Register2. This process is repeated, with the second word of each group placed in positions 5-8, the third word in positions 9-12, and the fourth word in positions 13-16. After the operation, the data in Register2 is treated as 8-bit data units. At this point, positions 1-8 in Register2 correspond to positions 1-4 and 17-20 in Register1, positions 17-24 in Register2 correspond to positions 5-8 and 21-24 in Register1, positions 33-40 in Register2 correspond to positions 9-12 and 25-28 in Register1, and positions 49-56 in Register2 correspond to positions 13-16 and 29-32 in Register1.
[0075] Execute, Register2 = Ingenic_simd512_gather_byte(Register2), where
[0076] Ingenic_simd512_gather_byte is also a data sorting instruction. This instruction treats every two words of data as a group, placing the 8 bits of data at even-numbered positions in the first word and the 8 bits of data at odd-numbered positions in the second word. After performing this operation on the 8 pairs of words in Register2, the positions 1-8 in Register2 are now 1, 3, 17, 19, 2, 4, 18, 20 in Register1; the positions 17-24 in Register2 are now 5, 7, 21, 23, 6, 8, 22, 24 in Register1; the positions 33-40 in Register2 are now 9, 11, 25, 27, 10, 12, 26, 28 in Register1; and the positions 49-56 in Register2 are now 13, 15, 29, 31, 14, 16, 30, 32 in Register1. After performing another Ingenic_simd512_gather_byte operation, positions 1-8 in Register2 are 1, 17, 2, 18, 3, 19, 4, 20 in Register1; positions 17-24 in Register2 are 5, 21, 6, 22, 7, 23, 8, 24 in Register1; positions 33-40 in Register2 are 9, 25, 10, 26, 11, 27, 12, 28 in Register1; and positions 49-56 in Register2 are 13, 29, 14, 30, 15, 31, 16, 32 in Register1.
[0077] The command `Register3 = Ingenic_simd512_logical_shift_right_16_bit(Register2)` is executed. `Ingenic_simd512_logical_shift_right_16_bit` is a shift instruction that logically shifts each word in register `Register2` 16 bits to the right. The resulting data is stored in register `Register3`. At this point, positions 1-2 in `Register3` are positions 2 and 18 of `Register1`, positions 5-6 in `Register3` are positions 4 and 20 of `Register1`, positions 17-18 in `Register3` are positions 6 and 22 of `Register1`, positions 21-22 in `Register3` are positions 8 and 24 of `Register1`, positions 33-34 in `Register3` are positions 10 and 26 of `Register1`, positions 37-38 in `Register3` are positions 12 and 28 of `Register1`, positions 49-50 in `Register3` are positions 14 and 30 of `Register1`, and positions 53-54 in `Register3` are positions 16 and 32 of `Register1`.
[0078] The command `output_waddr_cols=Ingenic_simd512_store(Register2,Register3)` is executed. `Ingenic_simd512_store` is a data storage instruction. This instruction stores the data in registers `Register2` and `Register3` in the four columns of the corresponding row pointer of the output feature map. Each column contains two channel data and the four columns of the second, third, and fourth rows of the current row.
[0079] Before the second loop ends, the input feature map column pointer is incremented by 32 by `input_raddr_cols += 32`, which means it points to the next column of data. Add the output feature map column pointer The pointer points to the column data that needs to be stored in the next iteration of the loop.
[0080] In summary, the key points of this application are:
[0081] 1. Using the SIMD instruction set, specifically the Ingenic_simd512 instruction set from the chip developed by Beijing Junzheng in this embodiment, the PixelShuffle upsampling algorithm was implemented, reducing the size to [missing information]. The low-resolution feature map is obtained by recombining the pixels on the channel to obtain a size of The high-resolution feature maps can be used to run the PixelShuffle algorithm on the Beijing Junzheng chip using the Ingenic_simd512 instruction set, and the results are consistent with the PixelShuffle algorithm implemented in Python.
[0082] 2. The Ingenic_simd512 instruction set can load 512 bits of data at a time, and a single instruction can perform calculations on 512 bits of data simultaneously, greatly improving the calculation speed. It can run efficiently on chips with limited hardware resources and high calculation speed requirements. In comparison, under the same chip conditions, the SIMD-implemented PixelShuffle algorithm is 5 times faster than the C-implemented PixelShuffle algorithm.
[0083] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, various modifications and variations can be made to the embodiments of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.
Claims
1. A PixelShuffle implementation method based on SIMD instructions, characterized in that, The method will have a size of The low-resolution feature map is obtained by reordering the data in the register according to certain rules using two data sorting instructions in the SIMD set, namely Ingenic_simd512_shuffle and Ingenic_simd512_gather_byte, and then recombining them to obtain a size of The high-resolution feature map, where H is the height of the feature map and W is the width of the feature map. denoted as the number of channels in the feature map, C is the number of channels in the new feature map, where r is the upsampling factor, which is the image magnification factor, and the PixelShuffle algorithm is run on a chip using the SIMD instruction set. Executing data sorting instructions in a SIMD set is denoted as: Register2 = Ingenic_simd512_shuffle_4_group(Register1), where Ingenic_simd512_shuffle_4_group is a data sorting instruction, which reorders the data in the register according to certain rules; This instruction operates on word-type data, which is 4 8-bit data units. The 64 8-bit data in Register1 are transformed into 16 words. The 16 words are divided into 4 groups. The first word of each group is taken to form a new group and placed in positions 1-4 of Register2. Similarly, the second word of each group is taken to form a new group and placed in positions 5-8 of Register2. The third word of each group is placed in positions 9-12, and the fourth word of each group is placed in positions 13-16. Execute another data sorting instruction in the SIMD set, denoted as: Register2 = Ingenic_simd512_gather_byte(Register2), where Ingenic_simd512_gather_byte is also a data sorting instruction. This instruction treats every two words of data as a group, placing the 8 bits of data at even positions in the first word and the 8 bits of data at odd positions in the second word. After performing another Ingenic_simd512_gather_byte operation, the shift instruction in SIMD is executed, denoted as: Register3 = Ingenic_simd512_logical_shift_right_16_bit(Register2), where Ingenic_simd512_logical_shift_right_16_bit is a shift instruction that logically shifts each word of data in Register2 to the right by 16 bits, and the resulting data is stored in Register3.
2. The PixelShuffle implementation method based on SIMD instructions according to claim 1, characterized in that, The method further Includes the following steps: S1, Initialization Settings: Let w and h represent looping in the width and height directions of the input feature map, respectively; The height of the input feature map is in_height, the width is in_width, the data pointer is input_raddr, the row data (i.e., the pointer in the height direction) is input_raddr_rows, and the column data (i.e., the pointer in the width direction) is input_raddr_cols. The size of a row of data in the input feature map is the width multiplied by the number of channels, denoted as one_in_line_size. Adding one_in_line_size to a data pointer of the input feature map indicates jumping to the corresponding data pointer of the next row. The width of the output feature map is out_width, the pointers in the row data (height direction) are output_raddr_rows, and the pointers in the column data (width direction) are output_raddr_cols. The size of a row of data in the output feature map is the width multiplied by the number of channels, denoted as one_out_line_size. Adding one_out_line_size to a data pointer in the output feature map indicates jumping to the corresponding data pointer in the next row. S2, enter the first loop, looping along the height direction of the input feature map: Before the loop, h=0, indicating that the loop starts from a height of 1 and continues until the height is in_height. After each loop, h is incremented by one, indicating that each loop iteration is based on one height data point, and the loop repeats a total of in_height times. In a loop, w=0 means that a new loop should start every time the loop width is entered. The loop condition is: check if h < in_height is true. If not, exit the first loop; if true, execute the first loop once. When entering the loop, the pointer to the input feature map row is assigned to the pointer to the first column of data in that row, input_raddr_cols, by setting input_raddr_cols = input_raddr_rows. The pointer to the first column of the output feature map is obtained by setting output_waddr_cols = output_waddr_rows; Before the end of each loop, the input feature map row pointer is moved to the next row by adding `input_raddr_rows += one_in_line_size`. output_waddr_rows += sampling factor The output feature map row pointer jumps to the sampling factor value row, and in this way, one row of input feature map data is processed in one loop to obtain the data of the output feature map in the sampling factor value row; S3, execute the second loop, looping along the width of the input feature map: Before the loop, set w=0, indicating that the loop starts from a width of 1 and continues until the width is in_width. After each loop, increment w by one, indicating that each loop iteration is for one width value, and the loop repeats for a total of in_width times. The condition for the second loop is: check if w < in_width is true. If not, exit the second loop; if true, execute: The second loop needs to process the row data from the first loop. The second loop loads one column of data at a time. Column data consisting of data from each channel: Executing data loading instructions in the SIMD set is denoted as: Register1 = Ingenic_simd512_load(input_raddr_cols), where Ingenic_simd512_load is a data loading instruction. Its function is to load data into a register. A register can be loaded with 512 bits of data at a time. Since the input feature map data type is unsigned 8-bit, a maximum of 64 data can be loaded at a time; the parameter input_raddr_cols represents column data containing 32 channels. This instruction loads these 32 data into positions 1-32 of register Register1, while positions 33-64 of Register1 are empty. Executing data sorting instructions in a SIMD set is denoted as: Register2 = Ingenic_simd512_shuffle_4_group( Register1 ) After the operation is completed, the data in Register2 is treated as 8-bit data units. At this time, positions 1-8 in Register2 are positions 1-4 and 17-20 in Register1, positions 17-24 in Register2 are positions 5-8 and 21-24 in Register1, positions 33-40 in Register2 are positions 9-12 and 25-28 in Register1, and positions 49-56 in Register2 are positions 13-16 and 29-32 in Register1. Execute another data sorting instruction in the SIMD set. After operating on the 8 pairs of words in Register2, the positions 1-8 in Register2 are now 1, 3, 17, 19, 2, 4, 18, 20 in Register1; the positions 17-24 in Register2 are now 5, 7, 21, 23, 6, 8, 22, 24 in Register1; the positions 33-40 in Register2 are now 9, 11, 25, 27, 10, 12, 26, 28 in Register1; and the positions 49-56 in Register2 are now 13, 15, 29, 31, 14, 16, 30, 32 in Register1. After performing another Ingenic_simd512_gather_byte operation, positions 1-8 in Register2 are 1, 17, 2, 18, 3, 19, 4, 20 in Register1; positions 17-24 in Register2 are 5, 21, 6, 22, 7, 23, 8, 24 in Register1; positions 33-40 in Register2 are 9, 25, 10, 26, 11, 27, 12, 28 in Register1; and positions 49-56 in Register2 are 13, 29, 14, 30, 15, 31, 16, 32 in Register1. Executing a shift instruction in SIMD is denoted as: Register3 = Ingenic_simd512_logical_shift_right_16_bit(Register2). At this time, positions 1-2 in Register3 are 2 and 18 in Register1, positions 5-6 in Register3 are 4 and 20 in Register1, positions 17-18 in Register3 are 6 and 22 in Register1, positions 21-22 in Register3 are 8 and 24 in Register1, positions 33-34 in Register3 are 10 and 26 in Register1, positions 37-38 in Register3 are 12 and 28 in Register1, positions 49-50 in Register3 are 14 and 30 in Register1, and positions 53-54 in Register3 are 16 and 32 in Register1. Executing data storage instructions in SIMD is denoted as: output_waddr_cols=Ingenic_simd512_store(Register2,Register3), where Ingenic_simd512_store is a data storage instruction. Its function is to store the data in registers Register2 and Register3 in the four columns of the corresponding row pointer of the output feature map. Each column contains two channel data and the four columns of the second, third and fourth rows of the current row. Before the second loop ends, the input feature map column pointer is incremented by the number of channels using `input_raddr_cols += number of channels`, which means it points to the next column of data. By += sampling factor of output_waddr_cols The number of channels will be the output feature map column pointer plus a sampling factor. The channel number is indicated by a pointer to the column data that needs to be stored in the next loop.
3. The PixelShuffle implementation method based on SIMD instructions according to claim 2, characterized in that, In the method described, the upsampling factor is set to 4, the width and height of the input feature map are both 10, and the number of channels is 32. The column data consists of 32 channels, and the pixel data type is unsigned 8-bit, i.e., uint8_t. Through the sorting instructions Ingenic_simd512_shuffle and Ingenic_simd512_gather_byte, an output feature map with a width and height of 40 and 2 channels is obtained.
4. The PixelShuffle implementation method based on SIMD instructions according to claim 1, characterized in that, The method described above applies to feature map data storage of the Tensor type, and its storage method is contiguous storage. Each channel's data forms a column of data, i.e., data in the width direction, and is then stored consecutively. Each data point constitutes a row of data, representing the data along the height direction, and is then stored contiguously. The size of the entire feature map is determined by these data points.
5. The PixelShuffle implementation method based on SIMD instructions according to claim 1, characterized in that, The method uses a SIMD instruction set that can load 512 bits of data into a register at once, and can perform the same operation on 512 bits of data at once during computation. This method uses the data loading, data storage, shifting, and data sorting instructions in the SIMD instruction set.