An IP core for remote sensing image preprocessing

Abstract

This specification discloses an IP core for remote sensing image preprocessing, which includes a parameter configuration module and a processing module. The parameter configuration module reads the initialization parameters configured by the user. The initialization parameters include read-related information for data reading of the original remote sensing image, write-related information for storing the preprocessed sliced images, and processing-related information indicating the preprocessing operations. The processing-related information includes the downsampling factor. The processing module reads the pixel data of the original remote sensing image in memory according to the read-related information to obtain the data to be processed. The data to be processed is downsampled according to the downsampling factor to obtain the pixel data of the sliced image, and the pixel data of the sliced image is written back to memory according to the write-related information. The IP core can be flexibly adapted to remote sensing images of different sizes through user configuration, combines the downsampling operation and the slicing operation in the preprocessing operation, reads the original remote sensing image only once, and obtains the sliced image, thus improving the preprocessing efficiency.

Description

Technical Field

[0001] This specification relates to the field of computer technology, and in particular, to an IP core for remote sensing image preprocessing. Background Art

[0002] With the development of artificial intelligence technology, artificial intelligence algorithms have been deployed on satellites to perform data processing tasks such as target detection and image segmentation of remote sensing images in orbit through artificial intelligence algorithms. However, the resolution, accuracy, and coverage of remote sensing images are usually relatively large, and the corresponding data volume of remote sensing images is also large. Artificial intelligence algorithms cannot receive remote sensing images as input all at once. Usually, after preprocessing operations such as downsampling and slicing of remote sensing images, the preprocessed thumbnail images, sliced images, etc. are input into the artificial intelligence algorithm in batches for processing.

[0003] Due to the programmable characteristics of FPGA chips, the image processing chips for on-board remote sensing images can be implemented based on FPGA chips. Currently, for the preprocessing operations of remote sensing images, developers need to write corresponding preprocessing operation programs and deploy them on satellites. Since the sizes of thumbnail images and sliced images required by different artificial intelligence algorithms are different, it is necessary to repeatedly write preprocessing programs for different artificial intelligence algorithms, resulting in low efficiency.

[0004] In the field of chip design, an Intellectual Property (IP) core refers to an integrated circuit module with specific functions that can be reused and contains specific core elements (instruction sets, function descriptions, codes, etc.). It is an intermediate component in chip design. Using an IP core in chip design can improve chip design efficiency. Therefore, this specification provides an IP core for remote sensing image preprocessing. Summary of the Invention

[0005] This specification provides an IP core for remote sensing image preprocessing to at least partially solve the problem of low software programming efficiency existing in the prior art.

[0006] This specification adopts the following technical solutions:

[0007] This specification provides an IP core for remote sensing image preprocessing. The IP core is connected to a memory through a bus. The IP core includes a parameter configuration module and a processing module, where:

[0008] The parameter configuration module reads the initialization parameters configured by the user. The initialization parameters include read-related information for data reading of the original remote sensing image, write-related information for storing the sliced images obtained after preprocessing, and processing-related information indicating preprocessing operations. The processing-related information includes the downsampling ratio;

[0009] The processing module reads the pixel data of the original remote sensing image in the memory according to the read-related information to obtain the data to be processed; downsamples the data to be processed according to the downsampling multiple to obtain the pixel data of the sliced image, and writes the pixel data of the sliced image back to the memory according to the write-related information.

[0010] Optionally, the processing-related information further includes the number of overlapping pixels of the slices and the size data of the sliced image. The number of overlapping pixels of the slices is used to indicate the number of pixels repeatedly read by the processing module at the edges of adjacent sliced images when reading the original remote sensing image.

[0011] When reading the pixel data of the original remote sensing image, the processing module determines that after reading enough pixel data for one row of the sliced image based on the size data of the sliced image, and then repeatedly reads the original remote sensing image according to the number of overlapping pixels of the slices.

[0012] Optionally, the read-related information includes the original starting address of the original remote sensing image stored in the memory, the number of pixels read in each read operation, the number of transfers read in each read operation, the correction amount of the read address after determining that enough pixels in a small row have been read, the correction amount of the read address after determining that enough pixels in a medium row have been read, and the correction amount of the read address after determining that enough pixels in a large row have been read. A small row represents the pixel data of one row of the sliced image, a medium row represents the pixel data of one row of the original remote sensing image, a large row represents the total number of pixels included in the sliced images obtained by dividing one row of the original remote sensing image, and the number of row divisions is the number of sliced images that one row of the original remote sensing image can be divided into.

[0013] Optionally, the write-related information includes the target starting address where the sliced image needs to be stored in the memory, the size data of the sliced image, the number of transfers written in each write operation, the correction amount of the write address after determining that enough pixels in a small row have been read, the correction amount of the write address after determining that enough pixels in a medium row have been read, the correction amount of the write address after determining that enough pixels in a large row have been read, the small row comparison reference quantity, the medium row comparison reference quantity, and the large row comparison reference quantity. The small row comparison reference quantity is the number of sliced images that one row of the original remote sensing image can be divided into, and is used as a comparison reference quantity for determining the state of having read enough pixels in a small row. The medium row comparison reference quantity is the number of pixels included in one column of the sliced image, and is used as a comparison reference quantity for determining the state of having read enough pixels in a medium row. The large row comparison reference quantity is the number of sliced images that one column of the original remote sensing image can be divided into, and is used as a comparison reference quantity for determining the state of having read enough pixels in a large row.

[0014] Optionally, the processing module includes a read processing sub-module, a cache sub-module, a write processing sub-module, and a judgment sub-module.

[0015] Optionally, the read processing sub-module determines the original starting address of the original remote sensing image stored in the memory according to the read-related information, and reads the pixel data of the original remote sensing image starting from the original starting address to obtain the data to be processed;

[0016] The cache sub-module caches the data to be processed, downsamples the data to be processed according to the downsampling multiple to obtain the pixel data of the sliced image, and provides a read port for the write processing sub-module;

[0017] The write processing sub-module reads the pixel data of the sliced image through the read port and writes the pixel data of the sliced image back to the memory.

[0018] Optionally, the judgment sub-module corrects the read address according to the number of pixels read from the original remote sensing image by the read processing sub-module. When the read processing sub-module reads enough pixel data of a small row or a medium row or a large row in the original remote sensing image, the read address is corrected so that the read processing sub-module performs a read operation from the corrected read address.

[0019] Optionally, after the judgment sub-module determines that the write processing sub-module has written enough pixel data for one row of the sliced image, it enters the small row judgment state, corrects the read address of the read processing sub-module according to the correction amount of the read address after determining that a small row is read in the read-related information, and corrects the write address of the write processing sub-module according to the correction amount of the write address after determining that a small row is read in the write-related information;

[0020] In addition, the judgment sub-module determines whether the read processing sub-module has read enough pixel data for the number of small rows corresponding to the small row comparison reference amount according to the small row comparison reference amount. If so, it enters the medium row judgment state. If not, it continues to perform read operations, cache operations, and write operations in a loop through the read processing sub-module, the cache sub-module, and the write processing sub-module according to the corrected read address and write address.

[0021] Optionally, when entering the medium row judgment state, the judgment sub-module corrects the read address of the read processing sub-module according to the correction amount of the read address after determining that a medium row is read in the read-related information, and corrects the write address of the write processing sub-module according to the correction amount of the write address after determining that a medium row is read in the write-related information;

[0022] Moreover, the determination sub-module determines whether the read processing sub-module has read enough pixel data for the number of middle rows corresponding to the middle row comparison reference quantity according to the middle row comparison reference quantity. If so, it enters the large row status determination. If not, it continues to perform read operations, caching operations, and write operations in a loop through the read processing sub-module, cache sub-module, and write processing sub-module according to the corrected read address and write address.

[0023] Optionally, when entering the large row status determination, the determination sub-module corrects the read address of the read processing sub-module according to the correction amount of the read address after determining that the large row has been read enough in the read-related information; and corrects the write address of the write processing sub-module according to the correction amount of the write address after determining that the large row has been read enough in the write-related information.

[0024] Moreover, the determination sub-module determines whether the read processing sub-module has read enough pixel data for the number of large rows corresponding to the large row comparison reference quantity according to the large row comparison reference quantity. If so, it determines that the preprocessing operation on the original remote sensing image has been completed. If not, it continues to perform read operations, caching operations, and write operations in a loop through the read processing sub-module, cache sub-module, and write processing sub-module according to the corrected read address and write address.

[0025] Optionally, the IP core further includes a state machine module.

[0026] The state machine module sets 12 states, which are respectively the idle state, parameter configuration state, read address sending state, read data receiving state, read data waiting state, write address sending state, write data sending state, write data waiting state, small row status determination state, middle row status determination state, large row status determination state, and address transformation state. The above states are used to control the operation of the IP core.

[0027] Optionally, the state machine module encodes the above states using Gray code to determine the state codes of the above states.

[0028] Optionally, the state machine module sets the transition conditions between the above states to determine the order among the state transition situations corresponding to the 12 states.

[0029] According to the above order, Gray code encoding is performed on the above states included in each state transition situation as the state codes of the above states, so that in each state transition situation, when the previous state changes to the next state, the total number of bit changes of the Gray code is minimized.

[0030] Optionally, using 4-bit Gray code, there are kinds of permutation results for the Gray code encoding corresponding to the 12 states.

[0031] For each permutation result, the state machine module determines the Gray code encoding of each state corresponding to the permutation result in each state transition situation.

[0032] For each state transition situation in the order described above, determine the bit change between the Gray code encoding of the previous state and the Gray code encoding of the next state in the transition situation as the bit difference corresponding to the transition situation.

[0033] Take the sum of the bit differences corresponding to the state transition situations as the total bit loss of the permutation result.

[0034] Among the permutation results, select the permutation result with the smallest total bit change loss.

[0035] The above at least one technical solution adopted in this specification can achieve the following beneficial effects:

[0036] The IP core for remote sensing image preprocessing provided in this specification includes a parameter configuration module and a processing module. The parameter configuration module reads the initialization parameters configured by the user. The initialization parameters include read-related information for reading the data of the original remote sensing image, write-related information for storing the sliced images obtained after preprocessing, and the downsampling factor. The processing module reads the pixel data of the original remote sensing image in the memory according to the read-related information to obtain the data to be processed, downsamples the data to be processed according to the downsampling factor to obtain the pixel data of the sliced images, and writes the pixel data of the sliced images back to the memory according to the write-related information.

[0037] The IP core of this specification determines the read-related information and write-related information according to the user's needs, can flexibly adapt to remote sensing image preprocessing tasks of different sizes, and at the same time combines the downsampling operation and the slicing operation in the preprocessing operation. By reading the original remote sensing image only once, the sliced images can be obtained, which greatly improves the preprocessing efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] The drawings described herein are used to provide a further understanding of this specification and form a part of this specification. The schematic embodiments of this specification and their descriptions are used to explain this specification and do not constitute an improper limitation of this specification. In the drawings:

[0039] Figure 1 is a schematic diagram of the IP core structure for remote sensing image preprocessing in this specification;

[0040] Figure 2 is a diagram of an image slicing result provided in this specification;

[0041] Figure 3 corresponding to what is provided in this specification Figure 2Schematic diagram of the storage range;

[0042] Figure 4 Schematic diagram of preprocessing of a remote sensing image with an overlapping part provided in this specification;

[0043] Figure 5 Schematic diagram of the structure of another IP core provided in this specification;

[0044] Figure 6 Schematic diagram of the function of a parameter configuration module provided in an embodiment of this specification;

[0045] Figure 7 Schematic diagram of the function of a read processing sub-module provided in an embodiment of this specification;

[0046] Figure 8 Schematic diagram of the function of a cache sub-module provided in an embodiment of this specification;

[0047] Figure 9 Schematic diagram of the function of a write processing sub-module provided in an embodiment of this specification;

[0048] Figure 10 Schematic diagram of the function of a judgment sub-module provided in an embodiment of this specification;

[0049] Figure 11 Schematic diagram of the structure of an IP core provided in an embodiment of this specification. Detailed implementation manners

[0050] To make the purpose, technical solutions and advantages of this specification clearer, the technical solutions of this specification will be clearly and completely described below in conjunction with specific embodiments of this specification and the corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of this specification, rather than all the embodiments. Based on the embodiments in this specification, all other embodiments obtained by those of ordinary skill in the art without creative efforts belong to the scope protected by this application.

[0051] The following will detail the technical solutions provided in each embodiment of this specification in conjunction with the drawings.

[0052] Figure 1 Schematic diagram of the structure of an IP core for preprocessing remote sensing images in this specification. As Figure 1 shown, the IP core in this specification has a parameter configuration module and a processing module. Among them, the IP core is connected to the memory through a bus.

[0053] In Figure 1In the example, the bus can be an AXI4-FULL bus, and the memory connected to the bus can be a Double Data Rate Synchronous Dynamic Random Access Memory (DDR memory). Then, the IP core can be connected to the DDR memory through the internal bus resources of the FPGA.

[0054] A parameter configuration module receives the initialization parameters configured by the user. The initialization parameters include read-related information for reading the data of the original remote sensing image, write-related information for storing the sliced images obtained after preprocessing, and the downsampling factor.

[0055] The read-related information includes the original starting address where the original remote sensing image is stored in the memory, and the write-related information includes the target starting address where the sliced images need to be stored in the memory. The IP core reads the pixel data of the original remote sensing image from the memory according to the read-related information, scales the read pixel data inside the IP core, and after slicing, stores the pixel data of the sliced images back in the memory.

[0056] The processing module reads the pixel data of the original remote sensing image from the memory according to the read-related information to obtain the data to be processed, downsamples the data to be processed according to the downsampling factor to obtain the pixel data of the sliced images. Then, according to the write-related information, it writes the pixel data of the sliced images back to the memory.

[0057] In the IP core provided in this specification, the initialization parameters configured by the user include the downsampling factor, which is used to downsample the pixel data of the original remote sensing image read by the IP core, so as to write the pixel data of the original remote sensing image back to the memory at intervals, synchronously realizing the downsampling and slicing of the original remote sensing image.

[0058] The read-related information in this specification is used for the processing module to read the pixel data of the original remote sensing image. The read-related information at least includes the original starting address where the original remote sensing image is stored in the memory, so that the processing module can read the pixel data of the original remote sensing image stored in the memory into the IP core through the bus according to the original starting address, for preprocessing operations such as downsampling and slicing of the original remote sensing image data.

[0059] In the original remote sensing image, the pixel data in the original remote sensing image is stored continuously row by row or column by column. After image slicing, an original remote sensing image can be sliced into multiple small sliced images, and the pixel data in each sliced image needs to be stored continuously. Therefore, during the process of writing the pixel data of the sliced images back to the memory, write address offset calculation is required.

[0060] In one embodiment of this specification, the write address offset calculation is controlled through three-layer loops, and the write-related information includes the correction amount of the write address after determining that enough small rows have been read, the correction amount of the write address after determining that enough medium rows have been read, and the correction amount of the write address after determining that enough large rows have been read.

[0061] In this embodiment, small rows represent the pixel data of one row of the sliced image, medium rows represent the pixel data of one row of the original remote sensing image, and large rows represent the total pixel data included in the sliced images with the number of row partitions of the original remote sensing image, where the number of row partitions is the number of sliced images that one row of the original remote sensing image can be sliced into. The correction amount of the write address after determining that enough small rows have been read is the total number of pixels included in the sliced image, the correction amount of the write address after determining that enough medium rows have been read is the number of pixels in one row of the sliced image, and the correction amount of the write address after determining that enough large rows have been read is the number of pixels corresponding to the large row, that is, the total number of pixels included in the sliced images with the number of row partitions of the original remote sensing image.

[0062] Among them, taking the length and width of the sliced image as the same as an example, that is When, using Store_row_off to represent the correction amount of the write address after determining that enough small rows have been read, using Store_brow_off to represent the correction amount of the write address after determining that enough medium rows have been read, and using Store_chn_off to represent the correction amount of the write address after determining that enough large rows have been read, then:

[0063]

[0064]

[0065]

[0066] In the above formula, represents the total number of pixels included in the sliced image, represents the total number of pixels included in the sliced images with the number of row partitions of the original remote sensing image.

[0067] The three-layer loops are respectively used to control the address offsets of small rows, medium rows, and large rows. The address offset corresponding to the small row is the correction amount of the write address after determining that enough small rows have been read, the address offset corresponding to the medium row is the correction amount of the write address after determining that enough medium rows have been read, and the address offset corresponding to the large row is the correction amount of the write address after determining that enough large rows have been read.

[0068] When the processing module determines that the number of read pixels is sufficient for the number of pixels corresponding to a minor row, it corrects the write address according to the correction amount of the write address after judging the minor row included in the write-related information. When it determines that the number of read pixel data is sufficient for the number of pixels corresponding to a middle row, it corrects the write address according to the correction amount of the write address after judging the middle row included in the write-related information. When it determines that the number of read pixels is sufficient for the number of pixels corresponding to a major row, it corrects the write address according to the correction amount of the write address after judging the major row included in the write-related information.

[0069] In this specification, the address correction amount corresponding to a minor row is the number of pixels included in the sliced image. The address correction amount corresponding to a middle row is the number of pixels in one row of the sliced image. The address correction amount corresponding to a major row is the total number of pixel data included in the sliced images with the row division number of the original remote sensing image.

[0070] The correction of the write address in the three-layer loop is initially based on the target starting address where the sliced image is stored in the memory. When it is determined that the number of read pixels is sufficient for the number of pixels corresponding to a minor row, the method for correcting the write address is to add the correction amount of the write address after judging the minor row to the target starting address to obtain the corrected write address, and then continue the write operation. When it is determined that the number of read pixels is sufficient for the number of pixels corresponding to a middle row, the method for correcting the write address is to add the correction amount of the write address after judging the middle row to the target starting address to obtain the corrected write address, and then continue the write operation. When it is determined that the number of read pixels is sufficient for the number of pixels corresponding to a major row, the method for correcting the write address is to add the correction amount of the write address after judging the major row to the target starting address to obtain the corrected write address, and then continue the write operation.

[0071] Take Figure 2 as an example. Figure 2 This is an image slicing result diagram provided in this specification. Figure 2 It shows an example where an original remote sensing image is sliced into 9 sliced images in a 3×3 format. Each slice contains 4 rows of pixel data. Each sliced image is labeled as "Slice m-n", where m represents the row order of the slice and n represents the column order of the slice. In Figure 2 the number of pixel rows included in Slice 1-1, Slice 1-2, and Slice 1-3 are marked, where ri represents the i-th row of pixel data.

[0072] Figure 3 This is the storage interval schematic diagram provided in this specification corresponding to Figure 2 . Figure 3 Shown above is Figure 2 the schematic diagram of the storage interval in the memory of each pixel data in the original remote sensing image of Figure 3 Shown below is Figure 2Schematic diagram of the storage range of pixels in each slice image included. As Figure 3 shown, in the storage of the original remote sensing image, the pixel data of each row is stored continuously. In the storage of the pixel data of the slice image, the pixel data of each row of each slice image is stored continuously.

[0073] The IP core of this specification reads the pixel data of the original remote sensing image in the storage order of each pixel data in the original remote sensing image. Therefore, when writing the read data to be processed back to the memory, in order to obtain the slice image, it cannot be written back completely in the read order. The write address needs to be corrected during the reading process so that the processing module can write the pixel data of the slice image back to the correct address position.

[0074] Corresponding to Figure 3 , slice 1-1 r1, slice 1-2 r1, slice 1-1 r2, etc. are each a small row; slice 1-1r1, slice 1-2 r1, slice 1-3 r1 together form a middle row. Because Figure 3 one row of the original remote sensing image is sliced into 3 slice images in , the pixel data included in all 3 slice images of one row is used as a large row. For example, the pixel data included in slice 1-1, slice 1-2, and slice 1-3 is the pixel data of a large row.

[0075] If the starting storage address of the slice image in the memory is the target starting address, during the reading process of the original remote sensing image, when the pixel data of slice 1-1 r1 is read, it is stored from the target starting address. When slice 1-2 r1 is read, the processing module offsets the read address by the correction amount of the read address after determining that a small row is read from the target starting address, and the storage position is the storage range of slice 1-2 in the memory. When slice 1-3 r1 is read, the processing module offsets the write address by the correction amount of the read address after determining that a small row is read again.

[0076] So far, the processing module has completed the reading of the pixel data of one row of the original remote sensing image, that is, the reading of the pixel data of a middle row. The processing module reads in the storage order of the pixel data of the original remote sensing image. The next one to be read is slice 1-1 r2, and this slice 1-1 r2 needs to be stored in the storage space corresponding to slice 1-1, that is, after slice 1-1r1. Therefore, the processing module offsets the read address by the correction amount of the read address after determining that a middle row is read from the target starting address, and stores slice 1-1 r2.

[0077] Further, after the processing module finishes reading slices 1-3 r4, that is, it has completed reading a large row in the original remote sensing image. The processing module offsets the read address by the correction amount of the read address after determining that a large row has been read from the target start address, and stores the next slice 2-1 r1 at a storage location that is the correction amount of the read address after determining that a large row has been read.

[0078] In the IP core provided in this specification, it supports users to configure the downsampling ratio according to the input requirements of the machine learning model, so as to synchronously complete the downsampling of the original remote sensing image during the preprocessing process. Then, the original remote sensing image preprocessed by the IP core of this specification is an image that has been sliced and downsampled. After applying such an image to the inference calculation of the machine learning model, the computational load of the model can be reduced and the inference rate can be accelerated.

[0079] Specifically, the processing module obtains the memory access address bit width of the bus, and this memory access address bit width can be preconfigured in the internal storage of the IP core. According to the memory access address bit width and the pixel bit depth corresponding to the original remote sensing image, the number of pixels transmitted by the bus at one time is determined, and sampling is performed among the number of pixels transmitted by the bus at one time according to the downsampling ratio to obtain the pixel data of the sliced image corresponding to one transmission.

[0080] In the IP core for remote sensing image preprocessing provided in this specification, a pipeline and address operation mechanism is adopted, encapsulating the operations of downsampling and slicing the original remote sensing image and executing them simultaneously, so that reading the original image once can generate the required sliced image, improving the efficiency of the entire IP core and ensuring the real-time performance of on-orbit large-scale remote sensing image preprocessing.

[0081] That is, the two operations of downsampling and slicing the original remote sensing image can be completed simultaneously, reducing the number of times of reading the original remote sensing image, improving the preprocessing efficiency, and ensuring the real-time performance of on-orbit large-scale remote sensing image preprocessing.

[0082] At the same time, the IP core of this specification provides an interface that can be configured by users. In view of the different scales of remote sensing images and the different scales of input pictures required by the vision model, a flexible configuration module is adopted, which is adapted according to three types of variables: remote sensing images, hardware configurations, and model inputs. The IP core is initialized by receiving the configuration of the user through the parameter configuration module of the IP core, so that the IP core in this specification can flexibly adapt to various different remote sensing image sizes and processing tasks, as well as the input requirements of machine learning models. The IP core in this specification can handle various situations after configuration, greatly increasing the reusability of the module. The flexible call of this method is provided in the form of a configurable IP, and it can be adapted to various spaceborne platforms after simple configuration. While being applicable to adapting to different camera payloads and AI application requirements, it can also be multi-core instantiated to exchange resources for speed, truly achieving plug-and-play.

[0083] In addition, the IP core in this specification is designed based on an FPGA chip. Compared with remote sensing image preprocessing chips based on CPUs or GPUs, since the IP core provided in this specification can directly operate on the memory without going through caching and software scheduling, by utilizing the characteristics of precise control of FPGA timing and the burst read / write characteristics of a large-capacity RAM, image downsampling and image slicing are considered together and processed within a predictable time, facilitating the control of the overall process response time.

[0084] During the preprocessing process in this specification, non-overlapping grid slicing can be performed on the original remote sensing image. Denote the size of the original remote sensing image as and the size of the sliced image as . The number of sliced images that can be cut from one row of the original remote sensing image is m, and the number of sliced images that can be cut from one column of the original remote sensing image is n. Then the original remote sensing image can be divided into a total of sliced images. In the non-overlapping grid slicing of this embodiment, , , where " " is rounding up.

[0085] Since the sliced images obtained in this specification are used as input data for a machine learning model for feature learning. To prevent missing edge targets of the sliced images during the recognition process of the machine learning model and affecting the accuracy of tasks such as target recognition and target detection, in an embodiment of this specification, an overlapping part can be set at the edge of the sliced images. Denote the number of overlapping pixels O between adjacent slices in the original remote sensing image, and the number of target overlapping pixels o in the sliced image obtained after preprocessing. When the downsampling factor is s, there is a relationship .

[0086] Among them, m and n can be determined according to the following formula:

[0087]

[0088]

[0089] " " in the above formula is rounding up. It may cause the total number of pixels of the sliced images calculated to be greater than the original remote sensing image. In this case, the part exceeding the original image can be filled with random pixels. Since the size of the original remote sensing image is known data for the machine learning model, even if a target is detected in the part exceeding the original image, it can be filtered according to the size of the original remote sensing image without affecting the detection result.

[0090] In this embodiment, the processing-related information in the initialized parameters configured by the user further includes the number of overlapping pixels O in the slice, which is used to indicate the number of pixels repeatedly read at the edge of adjacent slice images when the processing module reads the original remote sensing image. The parameter configuration module determines the number of target overlapping pixels o in the slice image obtained after preprocessing based on the equivalent relationship between the number of overlapping pixels O in the slice and the downsampling multiple s. Alternatively, the user can configure the number of overlapping pixels O in the slice and the number of target overlapping pixels o in the processing-related information, and then the parameter configuration module can directly obtain the number of overlapping pixels O in the slice, the number of target overlapping pixels o, and the downsampling multiple s configured by the user (the symbols are continued to be used hereinafter).

[0091] When the processing module reads the pixel data of the original remote sensing image, based on the size data of the slice image, after determining that the pixel data of one row of the slice image has been read enough, the original remote sensing image is repeatedly read according to the number of overlapping pixels in the slice.

[0092] Figure 4 It is a schematic diagram of preprocessing a remote sensing image with an overlapping part provided in this specification. Figure 4 The preprocessing process shown includes the downsampling and slicing processes. In Figure 4 , the original remote sensing image is divided into a total of slice images. The slice images in the original remote sensing image are the slice images without downsampling, which are marked in the original remote sensing image as "original slice m-n". It can be seen that when there is no downsampling, the size of the slice image is , and the number of overlapping pixels in each slice image in the original remote sensing image is O.

[0093] For the slice image after downsampling, taking the slice image corresponding to the original slice 1-1 as an example, it is shown below. It can be seen that after downsampling, the size of the slice image is Figure 4 , and the number of pixels in the overlapping part of each slice image is the number of target overlapping pixels o.

[0094] Based on this embodiment, when the processing module reads the pixel data of the original remote sensing image, since the slice image contains an overlapping part, the read operation also needs to correct the address. In this specification, the read address is also corrected through three-layer loops, with small rows, middle rows, and large rows as the boundaries.

[0095] Then the read-related information may include the correction amount of the read address after determining that enough small rows have been read, the correction amount of the read address after determining that enough medium rows have been read, and the correction amount of the read address after determining that enough large rows have been read. The correction amount of the read address after determining that enough small rows have been read is the difference between the number of pixels in one row of the sliced image without downsampling and the number of overlapping pixels of the slice in the original remote sensing image. The correction amount of the read address after determining that enough medium rows have been read is the product of the number of pixels in one row of the original remote sensing image and the downsampling factor. The correction amount of the read address after determining that enough large rows have been read is the product of the number of pixels in one row of the original remote sensing image and the number of non-overlapping pixels of the sliced image. The number of non-overlapping pixels is the difference between the number of pixels in one column of the sliced image in the original remote sensing image without downsampling and the number of overlapping pixels of the slice in the original remote sensing image.

[0096] Among them, let Load_row_off represent the correction amount of the read address after determining that enough small rows have been read, let Load_brow_off represent the correction amount of the read address after determining that enough medium rows have been read, and let Load_chn_off represent the correction amount of the read address after determining that enough large rows have been read. Then:

[0097]

[0098]

[0099]

[0100] In the above formula, represents the number of pixels in one row of the sliced image without downsampling, represents the product of the number of pixels in one row of the original remote sensing image and the downsampling factor, represents the number of non-overlapping pixels of the sliced image.

[0101] When the processing module determines that the number of read pixels is enough for the number of pixels corresponding to one small row, it corrects the read address according to the correction amount of the read address after determining that enough small rows have been read. When it determines that the read pixel data is enough for the number of pixels corresponding to one medium row, it corrects the read address according to the correction amount of the read address after determining that enough medium rows have been read. When it determines that the number of read pixels is enough for one large row, it corrects the read address according to the correction amount of the read address after determining that enough large rows have been read.

[0102] In an embodiment of this specification, the IP core in this specification can be designed separately according to functions. Then the processing module may include a read processing sub-module, a cache sub-module, a write processing sub-module, and a judgment sub-module. The structural diagram of an IP core corresponding to this embodiment can be as Figure 5 shown, Figure 5 which is the structural schematic diagram of another IP core provided in this specification.

[0103] In this embodiment, the read-related information includes the original starting address Load_addr stored in the memory for the original remote sensing image, the number of pixels Load_size read in each read operation, the number of transfers Load_length read in each read operation, the correction amount Load_row_off for the read address after determining that a small row has been read enough, the correction amount Load_brow_off for the read address after determining that a medium row has been read enough, and the correction amount Load_chn_off for the read address after determining that a large row has been read enough.

[0104] Among them, the determination methods for each data in the read-related information are as follows:

[0105] Load_addr is the starting address for storing the original remote sensing image in the memory, with a bit width of 32 bits;

[0106] Load_size = w × s, which is the number of pixels read in each read operation, with a bit width of 12 bits, where w is the length of the sliced image after downsampling, and s is the downsampling factor;

[0107] , which is the number of transfers read in each read operation, with a bit width of 10 bits, bw is the memory access address bit width of the bus, and this bus memory access address bit width can be pre-configured in the internal memory of the IP core. 8 is the bit depth of one pixel adopted in this specification;

[0108] Load_row_off = w × s - O, which is the correction amount for the read address after determining that a small row has been read enough, with a bit width of 12 bits, and O represents the number of non-overlapping pixels of the sliced image in the original remote sensing image without downsampling;

[0109] Load_brow_off = W × s, which is the correction amount for the read address after determining that a medium row has been read enough, with a bit width of 32 bits, and W represents the number of pixels in one row of the original remote sensing image;

[0110] Load_chn_off = W × (h × s × O), which is the correction amount for the read address after determining that a large row has been read enough, with a bit width of 32 bits, and h represents the height of the sliced image after downsampling.

[0111] In this embodiment, the write-related information includes the target starting address Store_addr in the memory where the sliced image needs to be stored, the size data Store_size of the sliced image, the number of transfers Store_length written in each write operation, the correction amount Store_row_off of the write address after determining that a small row has been read enough, the correction amount Store_brow_off of the write address after determining that a medium row has been read enough, the correction amount Store_chn_off of the write address after determining that a large row has been read enough, the small row comparison reference amount Store_row_num, the medium row comparison reference amount Store_brow_num, and the large row comparison reference amount Store_chn_num.

[0112] Among them, the determination methods of the data in the write-related information are as follows:

[0113] Store_addr is the target starting address in the memory where the sliced image needs to be stored, with a bit width of 32 bits;

[0114] Store_size = w, that is, it is equal to the number of pixels in a row of the sliced image. When the length and width of the sliced image are equal, it is the length or width of the sliced image, with a bit width of 10 bits;

[0115] , the number of transfers written in each write operation, with a bit width of 10 bits, bw is the memory access address bit width of the bus, and this bus memory access address bit width can be pre-configured in the internal storage of the IP core. 8 is the bit depth of a pixel adopted in this specification;

[0116] Store_row_off = w × w, the correction amount of the write address after determining that a small row has been read enough, with a bit width of 20 bits;

[0117] Store_brow_off = w, the correction amount of the write address after determining that a medium row has been read enough, with a bit width of 10 bits;

[0118] Store_chn_off = w × w × n, the correction amount of the write address after determining that a large row has been read enough, with a bit width of 32 bits;

[0119] Store_row_num = n, the small row comparison reference amount, used as the comparison reference amount for determining the state of having read enough small rows, with a bit width of 10 bits;

[0120] Store_brow_num = w, the medium row comparison reference amount, with a bit width of 10 bits, used as the comparison reference amount for determining the state of having read enough medium rows;

[0121] Store_chn_num = m, the large row comparison reference amount, with a bit width of 10 bits, used as the comparison reference amount for determining the state of having read enough large rows.

[0122] The bit widths of the respective data in the above read-related information and the bit widths of the respective data in the write-related information are shown in the form of examples and do not constitute a limitation on the IP core of this specification. In actual applications, the bit widths of the respective data can be flexibly set according to actual requirements.

[0123] In this specification, the data bit width of the user configuration interface of the parameter configuration module can be set according to the total bit widths of the respective data in the read-related information and the total bit widths of the respective data in the write-related information, so that the data bit width of the user configuration interface can ensure the transmission of both the read-related information and the write-related information, that is, the data bit width of the user configuration interface is not less than the total bit width of the respective data in the read-related information and at the same time not less than the total bit width of the respective data in the write-related information.

[0124] Figure 6 It is a functional schematic diagram of a parameter configuration module provided by an embodiment of this specification. As Figure 6 shown, the read-related information and the write-related information can be transmitted into the internal register of the IP core from the bus in two clock cycles. Specifically, when the configuration signal of the IP core is valid, the parameter configuration module reads the read-related information in the first clock cycle and reads the write-related information in the second clock cycle.

[0125] Of course, when the data lines of the IP core are sufficient, the data bit width of the user configuration interface can also be made not less than the sum of the total bit widths of the respective data in the read-related information and the write-related information, and when the configuration signal is valid, the read-related information and the write-related information can be transmitted into the internal register of the IP core in one clock cycle.

[0126] After the read-related information is transmitted into the IP core, the parameter configuration module distributes the respective data in the read-related information to the corresponding registers in order from low to high according to the appearance order of the respective data in the read-related information, stores them inside the IP core, and is used for subsequent preprocessing operations. Similarly, the parameter configuration module distributes the respective data in the write-related information to the corresponding registers in order from low to high according to the appearance order of the respective data in the write-related information, locks them inside the IP core, and is used for subsequent preprocessing operations.

[0127] According to the bit widths of the respective data in the read-related information and the bit widths of the respective data in the write-related information given above in this specification, it can be calculated that the total data bit width occupied by the read-related information is 130 bit, and the total data bit width occupied by the write-related information is 144 bit. Then in one example, the data bit width of the user configuration interface can be set to 160 bit.

[0128] After the read-related information and the write-related information are configured, it enters the read processing sub-module. The processing flow chart of this read processing sub-module is as Figure 7 shown, Figure 7This is a schematic diagram of the function of the read processing sub-module provided in the embodiments of this specification.

[0129] The read processing sub-module determines the original starting address of the storage of the original remote sensing image in the memory according to the read-related information, and reads the pixel data of the original remote sensing image starting from the original starting address to obtain the data to be processed.

[0130] When the AXI4-FULL bus is used for data transmission between the IP core and the memory, after receiving the configuration completion of the read-related information and the write-related information, it enters the read address sending state, performs a read address handshake on the read address channel, and receives the data of the 32-bit original starting address in the above example after the handshake is successful.

[0131] Since the memory access address bit width of the bus is bw and the bus burst length is denoted as brstlen, the number of pixels transmitted per clock cycle is: . As described above, the number of transfers read in each read operation , then the read burst (brust) satisfies the following formula:

[0132]

[0133] Because the data read in each burst needs to be stored in the internal cache FIFO of the IP core through the cache sub-module, after each burst read operation, it is necessary to determine whether the FIFO is full. If it is full, it is necessary to wait for the cache to free up storage space.

[0134] After each burst read operation of the read processing sub-module, the sub-module needs to determine whether it has completed the transmission of Load_length pixels, that is, whether it has completed the data transmission of the number of pixels in one row of the sliced image before downsampling. If it has been completed, through the judgment sub-module, it enters the next state. If it has not been completed, it is necessary to correct the read address and continue the read operation. The method of correcting the read address is specifically to increase on the basis of the current read address .

[0135] When the judgment sub-module jumps to the read address sending state, if the jump reason is that the small row has been read enough, through the judgment sub-module, according to the correction amount of the read address after judging that the small row has been read enough, Load_row_off = w × s × O, the read address of the read processing sub-module is corrected.

[0136] When the judgment sub-module jumps to the read address sending state, if the jump reason is that the middle row has been read enough, through the judgment sub-module, according to the correction amount of the read address after judging that the middle row has been read enough, Load_brow_off = W × s, the read address of the read processing sub-module is corrected.

[0137] When the judgment sub-module jumps to the read address sending state, if the jump reason is that enough large rows have been read, the judgment sub-module corrects the read address of the read processing sub-module according to the correction amount of the read address Load_chn_off = W×(h×s×O) after judging that enough medium rows have been read.

[0138] Figure 8 This is a schematic diagram of the function of a cache sub-module provided in the embodiments of this specification. As Figure 8 shown, the cache sub-module caches the data to be processed, downsamples the data to be processed according to the downsampling multiple, obtains the pixel data of the sliced image, and provides a read port for the write processing sub-module, so that the write processing sub-module reads the pixel data of the sliced image and writes the pixel data of the sliced image back to the memory.

[0139] It can be seen that the cache sub-module in this specification has three functions. One is to downsample the original remote sensing image, the second is to store the pixel data after downsampling, and the third is to provide a read port to enable the write processing sub-module to write the pixel data after downsampling, that is, the pixel data of the sliced image, back to the memory.

[0140] The above operation of downsampling the data to be processed can be completed by the cache sub-module in the processing module. That is, the cache sub-module obtains the memory access address bit width of the bus, and the memory access address bit width can be pre-configured in the memory inside the IP core. According to the memory access address bit width and the pixel bit depth corresponding to the original remote sensing image, the number of pixels transferred by the bus at one time is determined, and sampling is performed on the number of pixels transferred by the bus at one time according to the downsampling multiple to obtain the pixel data of the sliced image corresponding to one transfer.

[0141] In one embodiment, the cache sub-module maintains an 8-bit register to store the number of reads (transfers) in the read processing module when reading data, and records the number of reads (transfers) as read_index. A register for storing downsampled data is set, denoted as resize_data. The data read by the read data sub-module is stored in the rdata register. The bit width of resize_data is equal to the memory access address bit width bw of the bus, and the bit width of the rdata register is also bw.

[0142] Taking the downsampling multiple s = 8 and the memory access address bit width bw = 128 as an example, the execution process of the cache sub-module is specifically described below. The downsampling process can be shown in Table 1 below:

[0143] Table 1

[0144]

[0145] The pixel bit depth in this specification is 8. When the memory access bit width of the bus is 128 bit, the number of pixels that can be transmitted each time is 128 / 8 = 16. Since the downsampling multiple s = 8, that is, one pixel is sampled from every 8 pixels, then among the 16 pixels transmitted in one time, 2 pixels will be sampled. Table 1 shows the process of downsampling by the cache sub-module during the 0th to 7th burst transmissions.

[0146] Taking the 0th burst transmission in Table 1 as an example (the sampling process of subsequent burst transmissions is similar to the 0th time and will not be explained separately), the cache sub-module reads 128 bit of data through the bus and caches it in the rdata register. This 128 bit of data contains 16 pixels. Starting from the low bit to the high bit, one pixel (8 bit) is sampled from every 8 pixels (64 bit).

[0147] resize_data0[7:0] <= rdata[7:0] means assigning the low eight-bit data in the rdata register to the low eight-bit of the resize_data register, that is, sampling the 0th pixel (counting from 0) among the 16 pixels read from the bus and storing it in the resize_data register. resize_data0[15:8] <= rdata[71:64] means assigning the data from the 64th bit to the 71st bit in the rdata register to the 8th bit to the 15th bit of the resize_data register, that is, sampling the 8th pixel among the 16 pixels read from the bus and storing it in the resize_data register.

[0148] Both the read port and the write port of the cache sub-module are of bw bit width, and the read enable signal, write enable signal, write full signal, and read empty signal are configured. To make the read and write efficiency of the cache FIFO in the cache sub-module higher, the depth of the FIFO can be set to the number of bytes that can accommodate load_size.

[0149] When the read processing sub-module reads a row of the sliced image without downsampling, that is, w×s number of pixels, the IP core enters the write address issue state, and the write processing sub-module performs a write operation. Figure 9 It is a schematic diagram of the function of the write processing sub-module provided in the embodiments of this specification.

[0150] As Figure 9 shown, when data is transmitted between the IP core and the memory using the AXI4-FULL bus, after entering the write address issue state, the write processing sub-module performs a write address handshake in the write address channel according to the AXI4-FULL protocol. After the handshake is successful, it receives a 32-bit address, and this 32-bit address represents the target start address of the sliced image to be written to the memory in this write transaction.

[0151] Next, according to the AXI4-FULL protocol, data is written to the position in the DDR starting from the above 32-bit address in the write data channel. Since the memory access address bit width of the bus is bw and the bus burst length is denoted as brstlen, the number of pixels transmitted per clock cycle is: As described above, the number of transfers written in each write operation , then the write burst (brust) satisfies the following formula:

[0152]

[0153] Because the data read in each burst needs to be stored in the internal cache of the IP core through the cache sub-module, after each burst write operation, it is necessary to determine whether the FIFO is empty. If it is empty, it is necessary to stop and wait for the FIFO to receive the data sent by the read processing sub-module.

[0154] After each burst write operation of the write processing sub-module, it is judged whether the transmission of Store_length pixels has been completed, that is, whether the data transmission of the number of pixels in one row of the downsampled sliced image has been completed. If it is completed, through the judgment sub-module, it enters the next state. If it is not completed, it is necessary to correct the write address and continue the write operation. The method of correcting the write address is specifically to increase on the basis of the current write address .

[0155] When the judgment sub-module jumps to the write address sending state, if the jump reason is that enough small rows have been read, then through the judgment sub-module, according to the correction amount of the write address Store_row_off = w×w after judging that enough small rows have been read, the read address of the read processing sub-module is corrected.

[0156] When the judgment sub-module jumps to the write address sending state, if the jump reason is that enough middle rows have been read, then through the judgment sub-module, according to the correction amount of the write address Store_brow_off = w after judging that enough middle rows have been read, the write address of the write processing sub-module is corrected.

[0157] When the judgment sub-module jumps to the write address sending state, if the jump reason is that enough large rows have been read, then through the judgment sub-module, according to the correction amount of the write address Store_chn_off = w×w×n after judging that enough middle rows have been read, the write address of the write processing sub-module is corrected.

[0158] The slicing process of the original remote sensing image in this specification is a process of circular reading and caching and writing, as described above Figure 2 For the relevant content, in this specification, the process of circular reading and caching and writing is controlled through three-layer loops, and this loop control is implemented through the judgment sub-module.

[0159] Figure 10This is a schematic diagram of the function of a judgment sub-module provided in the embodiments of this specification. As Figure 10 shown, when the write processing sub-module writes enough pixel data for one line of sliced image data, that is, w pixel data, it enters the small line number judgment state, and then continues to judge whether it has read enough pixel data for one medium line. If so, it enters the medium line judgment state. If not, it enters the read address sending state and continues to loop for read cache writing operations.

[0160] After entering the medium line judgment state, it continues to judge whether it has read enough pixel data for one large line. If so, it enters the large line judgment state. If not, it enters the read address sending state and continues to loop for read cache writing operations.

[0161] After entering the large line judgment state, it continues to judge whether it has read enough pixel data for all the large lines included in the original remote sensing image. If so, it determines that the preprocessing operation is completed and enters the idle state. If not, it enters the read address sending state and continues to loop for read cache writing operations.

[0162] The judgment sub-module corrects the read address according to the number of pixels read from the original remote sensing image by the read processing sub-module. When the read processing sub-module reads enough pixel data for one small line or one medium line or one large line in the original remote sensing image, the read processing sub-module performs read operations from the corrected read address.

[0163] Specifically, when the judgment sub-module determines that the write processing sub-module has written enough pixel data for one line of the sliced image, it enters the small line number judgment state, corrects the read address of the read processing sub-module according to the correction amount of the read address after judging that the small line is read enough in the read-related information, and corrects the write address of the write processing sub-module according to the correction amount of the write address after judging that the small line is read enough in the write-related information.

[0164] In addition, when in the small line number judgment state, the judgment sub-module judges whether the read processing sub-module has read enough pixel data for the number of small lines corresponding to the small line comparison reference amount according to the small line comparison reference amount in the write-related information. If so, it enters the medium line judgment state. If not, it continues to loop for read operations, cache operations, and write operations through the read processing sub-module, cache sub-module, and write processing sub-module according to the corrected read address and write address.

[0165] When the judgment sub-module enters the medium line judgment state, the judgment sub-module corrects the read address of the read processing sub-module according to the correction amount of the read address after judging that the medium line is read enough in the read-related information, and corrects the write address of the write processing sub-module according to the correction amount of the write address after judging that the medium line is read enough in the write-related information.

[0166] Moreover, when determining the middle row state, the determination sub-module determines whether the read processing sub-module has read enough pixel data for the number of middle rows corresponding to the middle row comparison reference amount in the write-related information. If so, it enters the determination of the large row state. If not, it continues to loop through the read processing sub-module, cache sub-module, and write processing sub-module to perform read operations, cache operations, and write operations according to the corrected read address and write address.

[0167] When the determination sub-module enters the determination of the large row state, the determination sub-module corrects the read address of the read processing sub-module according to the correction amount of the read address after determining that enough large rows have been read in the read-related information; and corrects the write address of the write processing sub-module according to the correction amount of the write address after determining that enough large rows have been read in the write-related information.

[0168] Moreover, when determining the large row state, the determination sub-module determines whether the read processing sub-module has read enough pixel data for the number of large rows corresponding to the large row comparison reference amount according to the large row comparison reference amount. If so, it is determined that the preprocessing operation on the original remote sensing image has been completed. If not, it continues to loop through the read processing sub-module, cache sub-module, and write processing sub-module to perform read operations, cache operations, and write operations according to the corrected read address and write address.

[0169] In an embodiment of this specification, the IP core in this specification further includes a state machine module. Figure 11 It is a schematic structural diagram of an IP core provided in the embodiment of this specification, as Figure 11 shown. This IP core includes a state machine module, a parameter configuration module, and a processing module. Among them, the processing module includes a read processing sub-module, a cache sub-module, a write processing sub-module, and a determination sub-module. This IP core is connected to the memory through a bus and can receive the configuration signal of the user and output the current state.

[0170] The state machine module sets 12 states according to the functions that each module included in the above IP core needs to implement, and sets the transition conditions between each state. These 12 states are used to control the operation of each module in the IP core.

[0171] Among them, when the bus is an AXI4-FULL bus, the 12 states can be respectively set as an idle state, a parameter configuration state, a read address sending state, a read data receiving state, a read data waiting state, a write address sending state, a write data sending state, a write data waiting state, a small row determination state, a middle row determination state, a large row determination state, and an address transformation state.

[0172] For easy description, a comparison table of the English abbreviations and Chinese names of these 12 states is provided, as shown in Table 2:

[0173] Table 2

[0174]

[0175] The transition conditions between the 12 states are shown in the state transition table of Table 3:

[0176] Table 3

[0177]

[0178] The state machine module can determine the state codes of the above 12 states. Through the change of the state codes, the state machine realizes the state transition. The state code can be encoded by one-hot code, Gray code, and binary code, and this specification does not make specific restrictions.

[0179] Because the IP core provided in this specification is applied to the preprocessing of remote sensing images in the space environment, the space environment is prone to single-event effects, causing single-event upsets in the FPGA, resulting in the state code being disordered and affecting the normal operation of the IP core. Therefore, this specification provides a coding method for a highly reliable state machine module to reduce the impact of single-event upsets in space on the hardware.

[0180] The state machine module determines the order between the state transition situations corresponding to each state according to the transition conditions between the above 12 states shown in Table 3. According to this order, Gray code encoding is performed on each state included in each state transition situation as the state code of each state, so that in each state transition situation, when the previous state changes to the next state, the total number of bit changes of the Gray code is minimized.

[0181] Specifically, as can be seen from the above Table 3, there are a total of 17 state transitions in which the previous state and the next state change in each state transition situation. 4-bit Gray code can be used to encode each state. The 4-bit Gray code has different encodings. Among the 16 encodings, one encoding is assigned to each of the 12 states, so there are permutation results for the Gray code encodings corresponding to each state.

[0182] For each permutation result, the state machine module determines the Gray code encoding of each state corresponding to this permutation result in each state transition situation. According to this order, for each transition situation in turn, the bit change between the Gray code encoding of the previous state and the Gray code encoding of the next state in this transition situation is determined as the bit difference corresponding to this transition situation. The sum of the bit differences corresponding to each state transition situation is used as the total bit loss of this permutation result. Among the permutation results, the permutation result with the smallest total bit change loss is selected.

[0183] Among them, the total bit loss The determination method can be expressed by the following formula:

[0184]

[0185] Wherein, represents the exclusive OR calculation by bit, represents the Gray code of the previous state in the i-th state transition, represents the Gray code of the subsequent state in the i-th state transition, permutations represents a certain permutation generated among all possible permutations, and the order of the permutation corresponds one-to-one with the positive integer x, represents the sum of each bit of the exclusive OR result, and counts the number of 1s in the exclusive OR result.

[0186] Total bit change loss is the smallest, that is, to find a target combination x such that the value of is the smallest.

[0187] Through this embodiment, a preferred coding scheme with the smallest value can be obtained, which is specifically as follows:

[0188] IDLE (0010) -> INIT_PARAM (1010): 1 bit change(s)

[0189] INIT_PARAM (1010) -> RADDR_SEND (1110): 1 bit change(s)

[0190] RADDR_SEND (1110) -> RDATA_RECV (0110): 1 bit change(s)

[0191] RDATA_RECV (0110) -> RRESP_WAIT (1111): 2 bit change(s)

[0192] RRESP_WAIT (1111) -> RADDR_SEND (1110): 1 bit change(s)

[0193] RRESP_WAIT (1111) -> WADDR_SEND (1011): 1 bit change(s)

[0194] WADDR_SEND (1011) -> WDATA_SEND (0011): 1 bit change(s)

[0195] WDATA_SEND (0011) ->WRESP_WAIT (0001): 1 bit change(s)

[0196] WRESP_WAIT (0001) ->WADDR_SEND (1011): 2 bit change(s)

[0197] WRESP_WAIT (0001) ->JUDGE_ROW (0101): 1 bit change(s)

[0198] JUDGE_ROW (0101) ->JUDGE_BROW (0100): 1 bit change(s)

[0199] JUDGE_ROW (0101) ->INCR_ADDR (1100): 2 bit change(s)

[0200] JUDGE_BROW (0100) ->JUDGE_CHN (0000): 1 bit change(s)

[0201] JUDGE_BROW (0100) ->INCR_ADDR (1100): 1 bit change(s)

[0202] JUDGE_CHN (0000) ->IDLE (0010): 1 bit change(s)

[0203] JUDGE_CHN (0000) ->INCR_ADDR (1100): 2 bit change(s)

[0204] INCR_ADDR (1100) ->RADDR_SEND (1110): 1 bit change(s)

[0205] The state codes determined in the preferred solution are shown in the parentheses after each of the above states. The bit changes corresponding to each state transition are marked behind each state transition. The total bit change loss of this coding scheme .

[0206] Through this embodiment, in view of the bit flips easily caused by single event effects in the space environment, a hardened state machine using Hamming coding is adopted to ensure that when the program runs wild, it can return to the default state for continued processing.

[0207] It should also be noted that the term "comprising", "including" or any other variant thereof is intended to cover non-exclusive inclusion, such that a process, method, commodity or device comprising a series of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, commodity or device. Without further limitation, an element defined by the statement "comprising an..." does not exclude the presence of additional identical elements in the process, method, commodity or device comprising said element.

[0208] Each embodiment in this specification is described in a progressive manner. For the identical or similar parts among the embodiments, reference can be made to each other, and the differences between each embodiment and other embodiments are emphasized. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and reference can be made to the relevant part of the method embodiment for the related content.

[0209] The above description is only for the embodiments of this specification and is not intended to limit this specification. For those skilled in the art, various modifications and changes can be made to this specification. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of this specification shall be included within the scope of the claims of this application.

Claims

1. An IP core for remote sensing image preprocessing, characterized in that, The IP core is connected to the memory through a bus. The IP core includes a parameter configuration module and a processing module, where: The parameter configuration module reads the initialization parameters configured by the user. The initialization parameters include read-related information for reading data of the original remote sensing image, write-related information for storing the sliced images obtained after preprocessing, and processing-related information indicating the preprocessing operation. The processing-related information includes the downsampling ratio; The processing module reads the pixel data of the original remote sensing image in the memory according to the read-related information to obtain the data to be processed; downsamples the data to be processed according to the downsampling ratio to obtain the pixel data of the sliced image, and writes the pixel data of the sliced image back to the memory according to the write-related information; Wherein, the processing-related information further includes the number of overlapping pixels of the slices and the size data of the sliced image. The number of overlapping pixels of the slices is used to indicate the number of pixels repeatedly read by the processing module at the edge of adjacent sliced images when reading the original remote sensing image; When reading the pixel data of the original remote sensing image, the processing module determines that after reading enough pixel data for one row of the sliced image based on the size data of the sliced image, and then repeatedly reads the original remote sensing image according to the number of overlapping pixels of the slices; The read-related information includes the original starting address of the original remote sensing image stored in the memory, the number of pixels read in each read operation, the number of transmissions read in each read operation, the correction amount of the read address after determining that enough small rows have been read, the correction amount of the read address after determining that enough medium rows have been read, and the correction amount of the read address after determining that enough large rows have been read. The small row refers to the pixel data of one row of the sliced image, the medium row refers to the pixel data of one row of the original remote sensing image, the large row refers to the total number of pixels of the sliced images obtained by dividing one row of the original remote sensing image, and the number of row divisions is the number of sliced images that one row of the original remote sensing image can be divided into.

2. The IP core according to claim 1, wherein The write-related information includes the target starting address where the sliced image needs to be stored in the memory, the size data of the sliced image, the number of transmissions written in each write operation, the correction amount of the write address after determining that enough small rows have been read, the correction amount of the write address after determining that enough medium rows have been read, the correction amount of the write address after determining that enough large rows have been read, the small row comparison reference amount, the medium row comparison reference amount, and the large row comparison reference amount; The small row comparison reference amount is the number of sliced images that one row of the original remote sensing image can be divided into, and is used as a comparison reference amount for determining the state of having read enough small rows. The medium row comparison reference amount is the number of pixels included in one column of the sliced image, and is used as a comparison reference amount for determining the state of having read enough medium rows. The large row comparison reference amount is the number of sliced images that one column of the original remote sensing image can be divided into, and is used as a comparison reference amount for determining the state of having read enough large rows.

3. The IP core according to claim 2, characterized in that, The processing module includes a read processing sub-module, a cache sub-module, a write processing sub-module, and a judgment sub-module.

4. The IP core according to claim 3, characterized in that, The reading processing sub-module determines the original starting address of the storage of the original remote sensing image in the memory according to the reading-related information, and reads the pixel data of the original remote sensing image starting from the original starting address to obtain the data to be processed; The caching sub-module caches the data to be processed, downsamples the data to be processed according to the downsampling multiple to obtain the pixel data of the sliced image, and provides a read port for the writing processing sub-module; The writing processing sub-module reads the pixel data of the sliced image through the read port and writes the pixel data of the sliced image back to the memory.

5. The IP core according to claim 4, wherein The judging sub-module, according to the number of pixels read from the original remote sensing image by the reading processing sub-module, when the reading processing sub-module reads enough pixel data of a small row or a medium row or a large row in the original remote sensing image, corrects the read address so that the reading processing sub-module performs a read operation from the corrected read address.

6. The IP core according to claim 5, wherein After the judging sub-module determines that the writing processing sub-module has written enough pixel data for one row of the sliced image, it enters the small row judging state, corrects the read address of the reading processing sub-module according to the correction amount of the read address after judging that the small row is read enough in the reading-related information, and corrects the write address of the writing processing sub-module according to the correction amount of the write address after judging that the small row is read enough in the writing-related information; Moreover, the judging sub-module judges whether the reading processing sub-module has read enough pixel data for the number of small rows corresponding to the small row comparison reference amount according to the small row comparison reference amount. If so, it enters the medium row judging state. If not, it continues to loop through the reading processing sub-module, the caching sub-module, and the writing processing sub-module to perform read operations, caching operations, and write operations according to the corrected read address and write address.

7. The IP core according to claim 6, characterized in that, When entering the medium row judging state, the judging sub-module corrects the read address of the reading processing sub-module according to the correction amount of the read address after judging that the medium row is read enough in the reading-related information, and corrects the write address of the writing processing sub-module according to the correction amount of the write address after judging that the medium row is read enough in the writing-related information; Moreover, the judging sub-module judges whether the reading processing sub-module has read enough pixel data for the number of medium rows corresponding to the medium row comparison reference amount according to the medium row comparison reference amount. If so, it enters the large row judging state. If not, it continues to loop through the reading processing sub-module, the caching sub-module, and the writing processing sub-module to perform read operations, caching operations, and write operations according to the corrected read address and write address.

8. The IP core according to claim 7, wherein When entering the large row judging state, the judging sub-module corrects the read address of the reading processing sub-module according to the correction amount of the read address after judging that the large row is read enough in the reading-related information; corrects the write address of the writing processing sub-module according to the correction amount of the write address after judging that the large row is read enough in the writing-related information; Moreover, the determination sub-module determines whether the read processing sub-module has read pixel data of the number of large rows corresponding to the large row comparison reference amount according to the large row comparison reference amount. If so, it is determined that the preprocessing operation on the original remote sensing image is completed. If not, the read operation, caching operation, and writing operation are continuously cycled through the read processing sub-module, caching sub-module, and writing processing sub-module according to the corrected read address and write address.

9. The IP core according to claim 2, wherein The IP core further includes a state machine module; The state machine module sets 12 states, and each state is respectively: idle state, parameter configuration state, read address sending state, read data receiving state, read data waiting state, write address sending state, write data sending state, write data waiting state, small row judgment state, middle row judgment state, large row judgment state, and address transformation state. The above states are used to control the operation of the IP core.

10. The IP core according to claim 9, characterized in that, The state machine module encodes each of the above states using Gray code to determine the state codes of each of the above states.

11. The IP core according to claim 10, characterized in that, The state machine module sets the transition conditions between the above states to determine the order between the state transition situations corresponding to the 12 states; According to the above order, Gray code encoding is performed on each of the states included in each of the state transition situations as the state codes of each of the states, so that in each of the state transition situations, when the previous state changes to the next state, the total number of bit changes of the Gray code is minimized.

12. The IP core according to claim 11 uses a 4-bit Gray code, and there are permutation results for the Gray code encoding corresponding to the 12 states; The state machine module determines the Gray code encoding of each of the states corresponding to each permutation result in each of the state transition situations; According to the above order, for each state transition situation in turn, the bit change between the Gray code encoding of the previous state and the Gray code encoding of the next state in this state transition situation is determined as the bit difference corresponding to this state transition situation; The sum of the bit differences corresponding to each of the state transition situations is used as the total bit loss of this permutation result; Among the permutation results, the permutation result with the smallest total bit change loss is selected.

Images