A Remote Sensing Image Slicing System and Method Based on FPGA

Abstract

The present invention provides a remote sensing image slicing system and method based on FPGA. The system includes a CPU module and an FPGA module. The CPU module is used to decode the input original image and transmit the decoded image data and corresponding first configuration parameters to the FPGA module. The FPGA module includes an image slicing module and a memory module. The image slicing module is based on the first configuration parameters and is used to slice the original image into multiple sliced images, and the memory module is used to store the original image and the generated sliced images. By adopting the combination of the CPU module and the FPGA module and utilizing the parallel computing ability of the FPGA, multiple image slices can be processed simultaneously, improving the processing speed and efficiency; and having a lower processing delay and flexibility.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and particularly to a remote sensing image slicing system and method based on FPGA. Background Art

[0002] Remote sensing images usually have high resolution and large data volume. A single image file may be very large, and processing the image may result in excessive memory occupation and low processing efficiency. For the processing of large-scale remote sensing images, traditional computing resources may not meet the requirements, and technologies such as parallel computing and distributed processing are usually required to improve the processing efficiency.

[0003] Based on the requirements and challenges of large-scale remote sensing image processing, the remote sensing image slicing processing technology has emerged. Remote sensing images usually need to be preprocessed, such as denoising, enhancement, correction, feature extraction, etc. Directly preprocessing the entire image requires a large amount of computing resources and time, and different regions or targets of remote sensing images may require different processing methods. After slicing the image, personalized preprocessing operations can be performed on each small block to improve the processing accuracy and efficiency; in addition, some remote sensing applications require real-time processing of remote sensing image data. After dividing the image into small blocks, parallel processing can be achieved to improve the real-time performance and response speed; when transmitting remote sensing image data over the network, dividing the image into small blocks can reduce the size of each data packet and improve the efficiency and stability of data transmission; for deep learning model training, slicing large-scale remote sensing images into small blocks helps to improve the training efficiency and reduce memory occupation, and at the same time, distributed computing resources can be used for training.

[0004] Remote sensing image slicing can combine technologies such as parallel computing, personalized processing, and real-time requirements, and can effectively process large-scale remote sensing image data, improve the processing efficiency and accuracy, and provide support for remote sensing image analysis and applications. However, existing remote sensing image slicing systems usually use CPUs, resulting in low processing speed, large full-link latency, high memory pressure, and high cost investment.

[0005] It should be noted that the above introduction to the technical background is only for the convenience of clearly and completely explaining the technical solutions of this application and facilitating the understanding of those skilled in the art. It cannot be considered that the above technical solutions are well-known to those skilled in the art just because these solutions are described in the background art part of this application. Summary of the Invention

[0006] The purpose of the present invention is to provide a remote sensing image slicing system and method based on FPGA to obtain higher processing efficiency, lower latency, and better flexibility.

[0007] To solve the above problems, in a first aspect, the following provides a remote sensing image slicing system based on FPGA, including a CPU module and an FPGA module;

[0008] The CPU module is used to decode the input original image and transmit the decoded image data and corresponding first configuration parameters to the FPGA module;

[0009] The FPGA module includes an image slicing module and a memory module;

[0010] Based on the first configuration parameters, the image slicing module is used to slice the original image into multiple sliced images, and the memory module is used to store the original image and the generated sliced images.

[0011] By adopting the combination of the CPU module and the FPGA module and utilizing the parallel computing ability of the FPGA, multiple image slices can be processed simultaneously, improving the processing speed and efficiency; and due to the hardware implementation method of the FPGA, it has a lower processing delay and is more suitable for the field of remote sensing image processing with high real-time requirements; at the same time, using the FPGA implementation can reduce the load pressure on the CPU, and can also make the CPU embedded and made into an independent device, reducing costs; the FPGA can also be customized according to actual needs, avoiding waste of computing resources, and having greater flexibility in improving resource utilization.

[0012] The image slicing module includes a coordinate calculation module, a burst read information generation module, a slice read module, and a slice write module connected in sequence; the coordinate calculation module is used to receive the first configuration parameters and calculate the address information corresponding to the to-be-formed sliced image; the burst read information generation module is used to receive the address information and generate corresponding burst information; the slice read module is used to receive the burst information and read the corresponding image data from the memory module; the slice write module is used to receive the image data and write it into the memory module to generate the sliced image.

[0013] An original image frame buffer queue and a slice buffer queue are set in the memory module; the original image is stored in the original image frame buffer queue, and the sliced image is stored in the slice buffer queue.

[0014] The FPGA module also includes a first PCIe core and a first register; the CPU module transmits the decoded image data to the original image frame buffer queue through the first PCIe core; the CPU module transmits the first configuration parameters to the first register through the first PCIe core.

[0015] The FPGA module includes a second PCIe core and a second register; the sliced image is transmitted to a subsequent module through the second PCIe core, and the subsequent module includes a GPU module; the second register is used to store second configuration parameters related to the GPU module, and the second register is connected to the second PCIe core and the image slicing module.

[0016] On the other hand, the present application also provides a remote sensing image slicing method based on an FPGA, which is carried out based on the remote sensing image slicing system according to any one of the first aspect, and includes:

[0017] Input the original image into the CPU module for decoding to obtain multi-band image cube data, and set the corresponding first configuration parameters in the CPU module;

[0018] Sequentially transmit the image data of one spectral band in the image cube data and the corresponding first configuration parameters to the FPGA module;

[0019] Based on the first configuration parameters, the original image is sliced into a plurality of sliced images through the image slicing module.

[0020] The FPGA module includes an image slicing module and a memory module. The image slicing module includes a coordinate calculation module, a burst read information generation module, a slice read module, and a slice write module connected in sequence. The step of slicing the original image into a plurality of sliced images through the image slicing module includes: generating a corresponding two-dimensional coordinate system according to the size of the original image, calculating the coordinate information of each sliced image to be formed through the coordinate calculation module based on the configuration parameters, and converting the coordinate information into the corresponding address information of the sliced image in the memory module; transmitting the address information to the burst read information generation module, generating corresponding burst information through the burst read information generation module, where the burst information includes a burst start address and a burst length; transmitting the burst information to the slice read module, reading the corresponding image data from the memory module through the slice read module; and writing the read image data into the memory module through the slice write module to generate the sliced image.

[0021] The configuration parameters include the size of the original image, the size of the sliced image to be formed, and the overlapping area of adjacent sliced images; the overlapping area is determined based on the size of the target to be recognized, and the overlapping area at the boundary is adaptively adjusted according to the remaining pixels.

[0022] The burst start address is the address corresponding to the first pixel of each row of the slice image to be formed in the memory module, and the burst length is the length of each row of the slice image to be formed.

[0023] The slicing method is a pipelined process.

[0024] Compared with the prior art, the beneficial effects of the present invention mainly include the following: 1) Improve processing efficiency: Utilize the parallel computing ability of the FPGA to process multiple image slices simultaneously, improving the processing speed and efficiency; 2) Reduce latency: Due to the hardware implementation method of the FPGA, it has a lower processing latency and is suitable for remote sensing image processing with high real-time requirements; 3) Reduce the computational pressure on the CPU: Implementing with the FPGA can reduce the load pressure on the CPU, enabling the CPU to be embedded and made into an independent device, reducing costs; 4) Save resources: The FPGA can be customized according to actual needs, avoiding waste of computing resources and improving resource utilization; 5) Flexibility: The FPGA can be customized according to different processing requirements and is suitable for various remote sensing image processing tasks. Brief Description of the Drawings

[0025] In order to more clearly illustrate the technical solutions in the specific embodiments of the present invention, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the following described drawings are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can be obtained based on these drawings.

[0026] Figure 1 It is the working principle diagram of the remote sensing image slicing system provided by the present invention.

[0027] Figure 2 It is the principle block diagram of the remote sensing image slicing system provided by the present invention.

[0028] Figure 3 It is the original image download flow chart provided by the present invention.

[0029] Figure 4 It is the schematic diagram of the slicing position and coordinates provided by the present invention.

[0030] Figure 5 It is the schematic diagram of the sliced image data reading process provided by the present invention.

[0031] Figure 6 It is the sliced image data reading flow chart provided by the present invention.

[0032] Figure 7 It is the schematic diagram of the sliced image data writing process provided by the present invention. Detailed Embodiments

[0033] The foregoing and other technical contents, features and effects of the present invention will be clearly presented in the following detailed description of a preferred embodiment with reference to the accompanying drawings. The directional terms mentioned in the following embodiments, such as: up, down, left, right, front or rear, etc., are only with reference to the directions in the attached drawings. Therefore, the directional terms used are for illustration and not for limiting the present invention.

[0034] The embodiments of the present application will be elaborated in detail below with reference to the accompanying drawings. However, those of ordinary skill in the art can understand that in the embodiments of the present application, many technical details are presented to help readers better understand the present application. However, even without these technical details and various changes and modifications based on the following embodiments, the technical solutions claimed in the present application can still be implemented.

[0035] The steps in the following embodiments do not correspond one by one to the content of the invention.

[0036] Embodiment 1

[0037] Remote sensing images have high resolution and large data volume, and a single image file is very large, making it inconvenient to directly perform image processing. To meet various requirements such as parallel computing, personalized processing, and real-time performance, the remote sensing image can be sliced first, and a large-scale remote sensing image is segmented into multiple sliced images that are easy for subsequent processing (for example, preprocessing operations such as denoising, enhancement, correction, feature extraction, etc.).

[0038] This embodiment provides a remote sensing image slicing system and method based on FPGA. Refer to Figure 1 and Figure 2 , the remote sensing image slicing system in this embodiment includes a CPU module and an FPGA module; among them, the CPU module is used to receive the original image input from the outside and then transmit it to the FPGA module, and the FPGA module slices the received image data to obtain sliced images and transmits them to the subsequent module (usually a GPU module) for subsequent processing.

[0039] Specifically, the remote sensing image slicing system in this embodiment includes an image decoding module, a memory module, and an image slicing module.

[0040] The image decoding module (i.e., the CPU module) is implemented in the main control CPU and is mainly used to implement the image decoding of mainstream image file formats such as Tiff, Img, JPG, and BMP. The image decoding module will parse the input original image into multi-band image cube data and then transfer it to the FPGA module for caching. Specifically, a first PCIe core and a memory module are provided in the FPGA module. In the memory module (in this embodiment, the memory module is DDR4), there is a raw image frame buffer queue for storing raw image data. The image decoding module transfers the parsed image data to the raw image frame buffer queue through the first PCIe core. The image decoding module, the first PCIe core, and the memory module together constitute a raw image distribution module, and the specific implementation process of the raw image distribution module is as follows Figure 3 shown. Refer to Figure 3 , first, the image decoding module sends an instruction (i.e., enabling the slicing function) to the FPGA module to confirm whether image data can be transmitted at this time. When it is determined that transmission can be performed, data transmission starts until the distribution of this frame of image is completed.

[0041] The image slicing module specifically includes a coordinate calculation module, a burst read information generation module, a slice reading module, and a slice writing module. Among them, the coordinate calculation module is used to calculate the address of each slice in the memory module and transmit the address information to the burst read information generation module; the burst read information generation module then confirms the burst start address and burst length according to the received address information and transmits them to the slice reading module; the slice reading module reads the corresponding image data from the memory module according to the received burst information and transmits it to the slice writing module; the slice writing module writes the received image data into the slice buffer queue in the memory module for storage and waits to be transmitted to the GPU module for processing.

[0042] Specifically, taking the slicing process of a frame of image as an example, for an input image with a size (resolution) of 4096*4096 (unit: pixel, the same below) (i.e., the image data of a certain spectral band in the image cube data or a part of it), this image is a two-dimensional structure (i.e., composed of a pixel array of 4096 rows * 4096 columns). When it is stored in the raw image frame buffer queue, it exists in the form of a one-dimensional structure, that is, the data corresponding to the 4096 rows of the raw image (each row has 4096 pixels) are arranged in sequence with the head and tail connected in the same row; during the process of the image decoding module transmitting the raw image data, the relevant configuration parameters (i.e., the first configuration parameters) of this frame of image data will also be transmitted simultaneously. The first configuration parameters include preset information such as the size of the image data, the size of the desired sliced image, and the overlapping size of adjacent sliced images. Generally, the first configuration parameters will be transmitted to the first register through the first PCIe core.

[0043] The coordinate calculation module interacts with the first register and calculates the coordinate information and address information of the sliced image according to the received first configuration parameter. The position and coordinates of the sliced image are as follows Figure 4 shown. The original image is a two-dimensional structure (generating a two-dimensional coordinate system accordingly). It is expected to divide the original image into m*n small sliced images, and the size of each sliced image is the same. To prevent information loss, an overlapping area (overlapping area x) is set between adjacent slices. When reaching the right and lower boundaries, the overlapping area needs to be adaptively adjusted according to the remaining pixels (overlapping areas x' and y). The values of the overlapping areas (including overlapping areas x, x', and y) are configured by the CPU module and recorded in the first configuration parameter. The setting of the overlapping area mainly avoids the problem when the target to be recognized is located at the boundary of different slices. When the target to be recognized is on the boundary, a non-overlapping slicing method may split the target into two different slices, resulting in failed recognition. By setting an overlapping method, this problem can be avoided. The size of the overlapping area can be set according to different application scenarios and is usually related to the size of the target to be recognized. For example, if the size of the subsequent target to be recognized is 32*32, an overlapping area with a width of at least 32 pixels can be set to avoid target loss.

[0044] For ease of understanding, the following is an example. When the size of the input original image is 4096*4096, the size of the required sliced image is 512*512, and the pre-set overlapping area x is 10, the original image will be divided into 9*9 small sliced images, and the corresponding overlapping areas x' and y are 442. Regarding each pixel as a unit length, the coordinate values of each sliced image in the two-dimensional coordinate system can be easily obtained. For example, the coordinates of the four vertices of the first slice (starting from the upper left corner and calculated in the clockwise direction) are (0,0), (0,512), (512,0), and (512,512) respectively. The coordinates of the four vertices of the second slice are (0,510), (510,1022), (512,510), and (512,1022) respectively. And so on, the coordinate information of each sliced image can be easily obtained. Through this method, the correspondence between any sliced image to be formed and the original image can be represented.

[0045] After the coordinate calculation module calculates the coordinate information of the sliced image, since the subsequent reading still needs to be performed from the memory module, and the original image exists in the memory module in the form of a one-dimensional structure, it is necessary to convert the coordinate information of the sliced image into its actual address information in the memory module. Specifically, according to the calculated coordinate information, for example, the coordinates of the four vertices of the first slice are (0, 0), (0, 512), (512, 0), and (512, 512), that is, it is a pixel array of 512 rows * 512 columns. The image data corresponding to this pixel array is stored in the memory module as 512 data segments spaced apart from each other, and the length of each data segment is 512, that is, 512 pixels in each row are a segment, and there is a gap of 4096 lengths (i.e., the length of the original image) between the data of adjacent rows. Then, the coordinate information can be converted into the actual storage address according to this relationship.

[0046] To improve the data reading speed, the present application adopts the burst transmission method. Therefore, it is necessary for the burst read information generation module to confirm the burst start address and burst length according to the address information. Refer to Figure 5 As described, for each slice, the burst start address is the address corresponding to the first pixel point of each row of the sliced image, and the burst length is the length of each row of the sliced image. For this embodiment, 512 burst start addresses need to be provided for each slice, and each burst length is 512, and the data is read 512 times in sequence.

[0047] The slice reading module reads the corresponding image data from the memory module according to the received burst information (including the burst start address and burst length). Specifically, as Figure 5 and Figure 6 shown, whenever the slice reading module receives a burst start address and a burst length, it reads the data at the corresponding address from the memory module according to the axi bus protocol, and then converts it into an AXI-Stream stream through the AXI2AXIS interface and transmits it to the slice writing module. After 512 times of data reading, a complete slice is obtained. It can be understood that the present system adopts a pipelined operation. The slice reading module does not need to wait for the original image to be completely cached in the memory module before reading different slices. It only needs to start reading when all the data of the current slice is cached in the memory module. For example, for the first slice, when the first 512 rows of the original image are cached, the data corresponding to the first slice is already sufficient, and at this time, the reading operation can be started to form the first slice image, and the same is true for subsequent slice images.

[0048] The slice data writing module writes the received data stream (AXI-Stream stream) into the slice cache queue in address order. Thus, the generation of a slice image is completed. For the generated slice images, they can be directly transmitted to the subsequent module (such as the GPU module), or they can be transmitted together after a certain number of slice images are generated. It can be understood that this transmission step is also a pipelined operation. The GPU module does not need to wait until all slices of the original image are cached in the memory module before transmission, and can be reasonably matched according to the generation speed of the slice images and the processing speed of the GPU module.

[0049] The transmission of the slice images from the memory module to the subsequent module can be constructed with reference to the original image distribution module. Specifically, a second PCIe core is set in the FPGA module, and a slice cache queue for storing slice images is provided in the memory module. The memory module, the second PCIe core, and the GPU module together constitute a slice image upload module. The specific implementation process of this module is as Figure 7 shown. First, the GPU module sends an idle flag to the FPGA module to indicate that data transmission can be performed. When the FPGA module has generated a specified number of slices, the image slicing module in the FPGA module generates an interrupt signal and sends it to the GPU module through the second PCIe core. At this time, the GPU module is set to the busy state and reads a specified number of slice images from the slice cache queue through the second PCIe core and performs corresponding processing. After processing the slice images read this time, the above process is repeated until all slice images are processed completely.

[0050] Similarly, a second register is set in the FPGA module. The second register stores configuration parameters related to the GPU module, and the second register can interact with the second PCIe core and the image slicing module.

[0051] Some common English nouns or letters used in this invention for the convenience of clear description are only for exemplary reference rather than restrictive interpretation or specific usage, and the protection scope of this invention should not be limited by their possible Chinese translations or specific letters.

[0052] It should also be noted that in this article, relational terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations.

Claims

1. A remote sensing image slicing system based on FPGA, characterized in that, It includes a CPU module and an FPGA module; The CPU module is used to decode the input original image and transmit the decoded image data and corresponding first configuration parameters to the FPGA module; The FPGA module includes an image slicing module and a memory module; Based on the first configuration parameters, the image slicing module is used to slice the original image into multiple sliced images, and the memory module is used to store the original image and the generated sliced images; The image slicing module includes a coordinate calculation module, a burst read information generation module, a slice read module, and a slice write module connected in sequence; The coordinate calculation module is used to receive the first configuration parameters and calculate the address information corresponding to the to-be-formed sliced image. The first configuration parameters include the size of the original image, the size of the to-be-formed sliced image, and the overlapping area of adjacent sliced images. The coordinate calculation module generates a corresponding two-dimensional coordinate system according to the size of the original image, calculates the coordinate information of each to-be-formed sliced image based on the first configuration parameters, and converts the coordinate information into the corresponding address information of the sliced image in the memory module; The burst read information generation module is used to receive the address information and generate corresponding burst information; The slice read module is used to receive the burst information and read the corresponding image data from the memory module; The slice write module is used to receive the image data and write it into the memory module to generate the sliced image.

2. The remote sensing image slicing system based on FPGA according to claim 1, wherein, An original image frame buffer queue and a slice buffer queue are set in the memory module; The original image is stored in the original image frame buffer queue, and the sliced image is stored in the slice buffer queue.

3. The remote sensing image slicing system based on FPGA according to claim 2, wherein The FPGA module also includes a first PCIe core and a first register; The CPU module transmits the decoded image data to the original image frame buffer queue through the first PCIe core; The CPU module transmits the first configuration parameters to the first register through the first PCIe core.

4. The remote sensing image slicing system based on FPGA according to claim 2, wherein The FPGA module includes a second PCIe core and a second register; The sliced image is transmitted to the subsequent module through the second PCIe core, and the subsequent module includes a GPU module; The second register is used to store second configuration parameters related to the GPU module, and the second register is connected to the second PCIe core and the image slicing module.

5. A method for slicing remote sensing images based on FPGA, characterized in that, Based on the remote sensing image slicing system according to any one of claims 1-4, it includes: Input the original image into the CPU module for decoding to obtain multi-band image cube data, and set the corresponding first configuration parameters in the CPU module; Sequentially transmit the image data of one spectral band in the image cube data and the corresponding first configuration parameters to the FPGA module; Based on the first configuration parameters, the original image is sliced into multiple sliced images by the image slicing module.

6. A method for slicing remote sensing images based on FPGA according to claim 5, characterized in that, The FPGA module includes an image slicing module and a memory module. The image slicing module includes a coordinate calculation module, a burst read information generation module, a slice reading module, and a slice writing module connected in sequence; The step of slicing the original image into a plurality of the sliced images by the image slicing module includes: Generating a corresponding two-dimensional coordinate system according to the size of the original image, and based on the first configuration parameter, calculating the coordinate information of each to-be-formed sliced image through the coordinate calculation module, and converting the coordinate information into the corresponding address information of the sliced image in the memory module; Transmitting the address information to the burst read information generation module, and generating corresponding burst information through the burst read information generation module, where the burst information includes a burst start address and a burst length; Transmitting the burst information to the slice reading module, and reading corresponding image data from the memory module through the slice reading module; Writing the read image data into the memory module through the slice writing module to generate the sliced image.

7. A method for slicing remote sensing images based on FPGA according to claim 6, characterized in that, The first configuration parameter includes the size of the original image, the size of the to-be-formed sliced image, and the overlapping area of adjacent sliced images; the overlapping area is determined based on the size of the target to be recognized, and the overlapping area at the boundary is adaptively adjusted according to the remaining pixels.

8. A method for slicing remote sensing images based on FPGA according to claim 7, characterized in that The burst start address is the address corresponding to the first pixel point of each row of the to-be-formed sliced image in the memory module, and the burst length is the length of each row of the to-be-formed sliced image.

9. A method for slicing remote sensing images based on FPGA according to claim 6, characterized in that, The slicing method is a pipelined process.

Images