Parallel data preprocessing method using high-resolution information

Abstract

The invention discloses a parallel data preprocessing method utilizing high-resolution information, which comprises the following steps of: for three input RGB (Red, Green, Blue) channel images, converting the input images into data of which the length and the width are 1 / 2 of the original length and the original width and the number of channels is 16 through three different image processing methods, namely scaling, converting a grey-scale map and Pixelunshuffle, and finally adding position codes. According to the method disclosed by the invention, the parallel capability between the channels of the neural network computing equipment can be better utilized, so that the overall reasoning time delay change of the model is small, and the speed on equipment with high parallelism, such as a GPU, is basically consistent with the speed of the model input by three channels.

Description

Technical Field

[0001] The present invention belongs to the technical field of neural networks, and specifically relates to a parallel data preprocessing method using high-resolution information. Background Art

[0002] Computer vision has a wide range of applications in real life, including but not limited to industrial defect detection, face recognition, autonomous driving, target detection in the field of remote sensing, etc. Currently, deep learning methods occupy a dominant position in computer vision due to their end-to-end characteristics and high accuracy. Common methods include Yolo, Mask-RCNN, etc.

[0003] In a visual model constructed by a neural network, increasing the resolution of the input image can often simply and crudely improve the accuracy of the model. This is because the model can capture more detailed information in the image through higher resolution. However, there are also problems, that is, both the amount of calculation and the inference latency will increase significantly, so that it cannot run properly on devices with limited computing resources at the edge side.

[0004] In addition, currently existing devices supporting neural network operations often have strong parallelism. Some mobile phones contain GPUs and can perform parallel computing by calling the OpenCL interface. However, in the current process of neural network design, attention is often only paid to reducing the amount of its own calculation (FLOPs), while ignoring the full utilization of the parallel computing power of the specified hardware, so there is a large room for optimization here. Summary of the Invention

[0005] In order to overcome the deficiencies of the prior art, the present invention provides a parallel data preprocessing method using high-resolution information. For a 3-input RGB channel image, through three different image processing methods, namely scaling, converting to grayscale, and Pixelunshuffle, the input image is converted into data with the length and width becoming 1 / 2 of the original and the number of channels being 16, and finally position encoding is added. The method of the present invention can better utilize the parallel ability between channels of a neural network computing device, making the overall inference latency of the model change little, and the speed on devices with strong parallelism such as GPUs is basically the same as that of the model with 3-channel input.

[0006] The technical solution adopted by the present invention to solve its technical problems is as follows:

[0007] Step 1: Scale the input original image;

[0008] Using the bilinear interpolation algorithm, scale the input 3-channel RGB image so that both its length and width become 1 / 2 of the original, and the number of channels remains unchanged;

[0009] Step 2: Convert the scaled image to grayscale;

[0010] Convert the scaled 3-channel RGB image into a single-channel grayscale image, with the number of channels becoming 1;

[0011] Step 3: Perform Pixelunshuffle on the input 3-channel RGB image;

[0012] Using the Pixelunshuffle method, convert each 1×2×2 pixel block in the 3-channel RGB image into a 4×1×1 pixel block, obtaining an image with 12 channels and the length and width becoming 1 / 2 of the original;

[0013] Step 4: Stack the images obtained in Step 1 to Step 3 and add positional encoding;

[0014] Stack the three types of images obtained in Step 1 to Step 3 together to obtain a new image with 16 channels and the length and width becoming 1 / 2 of the original; then add absolute positional encoding to the new image to obtain the final image; this image can replace the original 3-channel RGB image and be fed into the neural network for training.

[0015] Preferably, the Pixelunshuffle method is to split each 2×2 pixel block into 4 different channels, so that while the length and width are reduced to 1 / 2 of the original, the number of channels is expanded 4 times.

[0016] A computer program that causes a computer to execute the above parallel data preprocessing method.

[0017] An electronic device, comprising: a processor and a memory; the memory is used to store a computer program, and the processor is used to execute the computer program stored in the memory so that the electronic device executes the above parallel data preprocessing method.

[0018] A computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the above parallel data preprocessing method is implemented.

[0019] A chip, comprising: a processor for calling and running a computer program from a memory, so that a device installed with the chip executes the above parallel data preprocessing method.

[0020] A computer program product, the computer program product includes a computer storage medium, the computer storage medium stores a computer program, the computer program includes instructions that can be executed by at least one processor, and when the instructions are executed by the at least one processor, the above parallel data preprocessing method is implemented.

[0021] The beneficial effects of the present invention are as follows:

[0022] The present invention proposes a plug-and-play data preprocessing module, which can be applied to various deep learning-based vision tasks, including but not limited to image classification, object detection, instance segmentation, and semantic segmentation. This module utilizes high-resolution input information, which can improve the training accuracy of the model. At the same time, compared with the original 3-channel input data, the 16-channel input can better utilize the parallel computing ability between channels of the neural network computing device, resulting in little change in the overall inference latency of the model, and the speed on devices with strong parallelism such as GPUs is basically the same as that of the model with 3-channel input. Brief Description of the Drawings

[0023] Figure 1 It is a schematic diagram of the Pixelunshuffle method. Detailed Embodiment

[0024] The present invention will be further described below in conjunction with the drawings and embodiments.

[0025] The present invention provides a parallel data preprocessing method using high-resolution information, which can make more full use of the parallel computing ability of the neural network computing device, thereby improving the overall accuracy performance of the network.

[0026] Step 1: Scale the input original image;

[0027] Using the bilinear interpolation algorithm, scale the input 3-channel RGB image so that both its length and width become 1 / 2 of the original, and the number of channels remains unchanged;

[0028] Step 2: Convert the scaled image to a grayscale image;

[0029] Convert the scaled 3-channel RGB image to a single-channel grayscale image, and the number of channels becomes 1;

[0030] Step 3: Perform Pixelunshuffle on the input 3-channel RGB image;

[0031] Using the Pixelunshuffle method, convert each 1×2×2 pixel block in the 3-channel RGB image into a 4×1×1 pixel block, obtaining an image with 12 channels and the length and width becoming 1 / 2 of the original;

[0032] Step 4: Stack the images obtained in Steps 1 to 3 and add positional encoding;

[0033] Stack the three images obtained in Steps 1 to 3 together to obtain a new image with 16 channels and the length and width becoming 1 / 2 of the original; then add absolute positional encoding to the new image to obtain the final image; this image can replace the original 3-channel RGB image and be fed into the neural network for training.

[0034] The pixelunshuffle method splits each 2×2 pixel block into 4 different channels, reducing the length and width to 1 / 2 of the original while increasing the number of channels by 4 times.

[0035] Example:

[0036] Taking image classification as an example, for a 3×448×448 image, three different image processing methods, namely scaling, grayscale conversion, and pixelunshuffle, are used to convert it into data with a length and width of 1 / 2 of the original and a channel number of 16, that is, a 16×224×224 image. Finally, position encoding is added.

[0037] Step 1: Scale the original input image;

[0038] Using the bilinear interpolation algorithm, the input 3-channel RGB image is scaled ( torchvision.transforms.Resize ) so that its length and width are both reduced to 1 / 2 of the original, while the number of channels remains unchanged, resulting in a 3×224×224 image.

[0039] Step 2: Convert the input image to grayscale;

[0040] For the scaled RGB three-channel image, convert it to a single-channel grayscale image ( torchvision.transforms.Grayscale ). At this time, the length and width of the original image are reduced to 1 / 2 of the original image, and the channel is 1 data, that is, 1×224×224.

[0041] Step 3: Pixelunshuffle the input image;

[0042] For the original 3-channel RGB input image, Pixelunshuffle is used to convert each 1×2×2 pixel block in the original image into a 4×1×1 pixel block, thus obtaining a 12-channel image with a length and width reduced to half of the original image, that is, 12×224×224.

[0043] Step 4: Stack the processed images and add position encoding;

[0044] The 3-channel scaled image, 1-channel grayscale image, and 12-channel pixelunshuffle image generated in the previous three steps are stacked together to create a 16-channel image with half the length and width of the original image, that is, 16×224×224. Absolute position encoding is then added. This image can replace the original 3-channel RGB image and be fed into the neural network for training.

Claims

1. A parallel data preprocessing method using high-resolution information, characterized in that It includes the following steps: Step 1: Scale the input original image; Using the bilinear interpolation algorithm, scale the input 3-channel RGB image so that both its length and width become 1 / 2 of the original, and the number of channels remains unchanged; Step 2: Convert the scaled image to a grayscale image; Convert the scaled 3-channel RGB image to a single-channel grayscale image, and the number of channels becomes 1; Step 3: Perform Pixelunshuffle on the input 3-channel RGB image; Using the Pixelunshuffle method, convert each 1×2×2 pixel block in the 3-channel RGB image into a 4×1×1 pixel block, obtaining an image with 12 channels and the length and width becoming 1 / 2 of the original; Step 4: Stack the images obtained in Steps 1 to 3 and add positional encoding; Stack the three images obtained in Steps 1 to 3 together to obtain a new image with 16 channels and the length and width becoming 1 / 2 of the original; Then add absolute positional encoding to the new image to obtain the final image; this image can replace the original 3-channel RGB image and be sent to the neural network for training.

2. The parallel data preprocessing method using high-resolution information according to claim 1, wherein The Pixelunshuffle method is to split each 2×2 pixel block into 4 different channels, so that while the length and width are reduced to 1 / 2 of the original, the number of channels is expanded 4 times.

3. A computer program, characterized in that, The computer program causes the computer to execute the method described in any one of claims 1 to 2.

4. An electronic device, characterized in that, It includes: A processor and a memory; The memory is used to store the computer program, and the processor is used to execute the computer program stored in the memory, so that the electronic device executes the method described in any one of claims 1 to 2.

5. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the method described in any one of claims 1 to 2.

6. A chip, characterized in that, It includes: A processor, which is used to call and run the computer program from the memory, so that the device installed with the chip executes the method described in any one of claims 1 to 2.

7. A computer program product, characterized in that, The computer program product includes a computer storage medium, the computer storage medium stores a computer program, and the computer program includes instructions that can be executed by at least one processor. When the instructions are executed by the at least one processor, the method described in any one of claims 1 to 2 is implemented.

Images