Image correction method and device

Abstract

The invention relates to the technical field of image processing, and discloses an image correction method and device, and the method comprises the steps: obtaining original coordinate data of original pixels belonging to different first categories; determining correction coordinate data of a correction pixel corresponding to the original pixel, and determining a second category to which the correction pixel belongs according to a classification rule; caching the coordinate data to a corresponding first storage space according to a first category to which the original pixel belongs and a second category to which the corresponding correction pixel belongs; and taking the first storage spaces corresponding to the same second category in each first storage space group as one group, respectively acquiring each group of to-be-processed coordinate data, respectively determining a correction pixel value corresponding to each correction pixel, and generating a correction image according to the correction pixel value. According to the method, the N * N first storage spaces are utilized to realize acceleration of each processing path, so that the problem of blockage is effectively avoided, an acceleration effect close to N times is realized, and in-chip storage resources can be saved.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to an image correction method and device. Background Art

[0002] Image distortion is mainly caused by factors such as camera lens distortion and non-ideal optical imaging, which leads to various deformations and distortions of the image, which may affect the accuracy and reliability of the image.

[0003] Currently, the deployment of image dedistortion algorithms mostly uses ASIC (Application Specific Integrated Circuit) chips. This method has a simpler algorithm development project, but to achieve faster processing speeds, higher-end chips must be selected. Usually, image dedistortion is only one part of an image processing project. In image processing projects based on programmable logic devices such as FPGA (Field Programmable Gate Array), if an ASIC+FPGA solution is used, the high-speed dedistortion algorithm requires high computing power. In addition to high solution complexity, this heterogeneous solution is also costly. Summary of the invention

[0004] In view of this, the present invention provides an image correction method and device to solve the problem that programmable logic device dedistortion consumes more resources.

[0005] In a first aspect, the present invention provides an image correction method, applied to a programmable logic device, the method comprising:

[0006] Acquiring original coordinate data of original pixels belonging to different first categories; the first category to which the original pixels belong is determined by classifying the original coordinate data of the original pixels according to a preset classification rule;

[0007] Determining, according to the original coordinate data of the original pixel, the corrected coordinate data of the corrected pixel corresponding to the dedistorted original pixel, and classifying the corrected coordinate data of the corrected pixel according to the classification rule to determine the second category to which the corrected pixel belongs;

[0008] According to the first category to which the original pixel belongs and the second category to which the corresponding corrected pixel belongs, the corresponding coordinate data is cached in the corresponding first storage space; wherein a corresponding first storage space group is pre-configured for each first category, and the first storage space group includes the first storage space corresponding to each second category respectively; the coordinate data includes the original coordinate data of the original pixel and the corrected coordinate data of the corrected pixel;

[0009] Taking the first storage spaces corresponding to the same second category in each of the first storage space groups as a group, respectively acquiring the coordinate data to be processed in each group of the first storage spaces;

[0010] The correction pixel values ​​corresponding to the correction pixels are determined according to the coordinate data to be processed in each group, and a correction image is generated according to the correction pixel values.

[0011] In a second aspect, the present invention provides an image correction device, applied to a programmable logic device, the device comprising:

[0012] A coordinate acquisition module, used to acquire original coordinate data of original pixels belonging to different first categories; the first category to which the original pixels belong is determined by classifying the original coordinate data of the original pixels according to a preset classification rule;

[0013] a classification module, configured to determine, based on the original coordinate data of the original pixel, the corrected coordinate data of the corrected pixel corresponding to the original pixel after the original pixel is dedistorted, and classify the corrected coordinate data of the corrected pixel according to the classification rule to determine the second category to which the corrected pixel belongs;

[0014] A coordinate cache module, configured to cache corresponding coordinate data to a corresponding first storage space according to the first category to which the original pixel belongs and the second category to which the corresponding corrected pixel belongs; wherein a corresponding first storage space group is pre-configured for each first category, and the first storage space group includes first storage spaces corresponding to each second category respectively; the coordinate data includes original coordinate data of the original pixel and corrected coordinate data of the corrected pixel;

[0015] A grouping acquisition module, used to take the first storage spaces corresponding to the same second category in each of the first storage space groups as a group, and respectively acquire the coordinate data to be processed in each group of the first storage spaces;

[0016] The processing module is used to determine the correction pixel value corresponding to each correction pixel according to each group of the coordinate data to be processed, and generate a correction image according to the correction pixel value.

[0017] In a third aspect, the present invention provides a programmable logic device, comprising: a memory and a processor, the memory and the processor being communicatively connected to each other, the memory storing computer instructions, and the processor executing the image correction method of the first aspect or any corresponding embodiment thereof by executing the computer instructions.

[0018] In a fourth aspect, the present invention provides a computer-readable storage medium having computer instructions stored thereon, the computer instructions being used to enable a computer to execute the image correction method of the first aspect or any corresponding embodiment thereof.

[0019] In a fifth aspect, the present invention provides a computer program product, comprising computer instructions for causing a computer to execute the image correction method of the first aspect or any corresponding embodiment thereof.

[0020] The present invention divides the original pixels in the original image into N categories according to their coordinate data, so that the dedistortion processing can be performed in parallel according to the parallelism degree N, and there is no need to store N copies of the original image, which greatly reduces the required storage resources; and for irregular correction pixels after dedistortion, N×N first storage spaces are used to accelerate each processing path, effectively avoiding the problem of blocking, and parallel processing can be achieved. The programmable logic device as a whole can basically dedistort the original image at an N-fold rate, achieving an acceleration effect close to N times. In addition, while realizing multiple parallel dedistortion processing, it can save on-chip storage resources, so that programmable logic devices such as FPGA can be applied to dedistortion processing of large-size, high-frame-rate images. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the specific embodiments of the present invention or the technical solutions in the related technologies, the drawings required for use in the specific embodiments or the related technical descriptions will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0022] Figure 1 is a schematic flow chart of an image correction method according to an embodiment of the present invention;

[0023] Figure 2 is a schematic diagram of cached coordinate data according to an embodiment of the present invention;

[0024] Figure 3 is a flow chart of another image correction method according to an embodiment of the present invention;

[0025] Figure 4 is a system architecture diagram of a programmable logic device according to an embodiment of the present invention;

[0026] Figure 5 is a detailed flow chart of an image correction method according to an embodiment of the present invention;

[0027] Figure 6 is a structural block diagram of an image correction device according to an embodiment of the present invention;

[0028] Figure 7 It is a schematic diagram of the hardware structure of the programmable logic device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0029] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0030] The FPGA-based real-time image distortion correction system is widely used in machine vision, UAV remote sensing and other fields due to its highly parallel processing capabilities and fast response.

[0031] Traditional FPGA image de-distortion processing, due to limited hardware resources, mostly uses single-speed processing, which is slow and not suitable for high frame rate, large-size, and large-distortion image processing scenarios. Due to the irregularity of the coordinate mapping of the de-distortion algorithm, the use of parallel processing requires multiple logical resources, such as lookup tables, BRAM (Block Random Access Memory), URAM (Ultra Random Access Memory), etc. For example, for a grayscale image of 1920×1080 size, if the maximum distortion range is 100 pixels, at least 100 rows of pixels need to be cached during single-speed processing, which requires approximately 47 36K BRAMs; and if N-fold parallel processing is to be achieved, N copies of the image need to be stored, which requires a total of 47×N 36K BRAMs. Conventional FPGA chips are difficult to meet this parallel processing requirement.

[0032] The image correction method provided by the embodiment of the present invention divides the original pixels in the original image into N categories according to their coordinate data, so that the dedistortion processing can be performed in parallel according to the parallelism degree N, and there is no need to store N copies of the original image, which greatly reduces the required storage resources; and for the irregular corrected pixels after dedistortion, N×N first storage spaces are used to accelerate each processing path, effectively avoiding the problem of blocking, and parallel processing can be achieved. The programmable logic device as a whole can basically dedistort the original image at N times the rate, achieving an acceleration effect close to N times.

[0033] According to an embodiment of the present invention, an embodiment of an image correction method is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0034] In this embodiment, an image correction method is provided, which can be applied to a programmable logic device with parallel processing capabilities, such as an FPGA (Field Programmable Gate Array), a CPLD (Complex Programmable Logic Device), etc. Figure 1 is a flow chart of an image correction method according to an embodiment of the present invention. Figure 1 As shown, the process includes the following steps.

[0035] Step S101, obtaining original coordinate data of original pixels belonging to different first categories; the first category to which the original pixel belongs is determined by classifying the original coordinate data of the original pixel according to a preset classification rule.

[0036] In this embodiment, the original image is a distorted image that needs to be corrected to achieve dedistortion; for example, the original image is an image acquired by a camera. The original image includes multiple pixels, and for ease of description, the pixels in the original image are referred to as original pixels.

[0037] Moreover, each original pixel corresponds to a corresponding position in the original image, and the position can be expressed in the form of coordinates; specifically, in the original image, each original pixel has a first coordinate in the first direction and a second coordinate in the second direction, and the coordinate data of the original pixel, that is, the original coordinate data, can be determined based on the first coordinate and the second coordinate.

[0038] For example, if the first direction is the row direction and the second direction is the column direction, the first coordinate is the row coordinate and the second coordinate is the column coordinate. Conversely, if the first direction is the column direction and the second direction is the row direction, the first coordinate is the column coordinate and the second coordinate is the row coordinate.

[0039] To simplify the description, this embodiment is mainly described by taking the first coordinate as the row coordinate and the second coordinate as the column coordinate as an example. Specifically, for the original pixel in the uth row and the vth column in the original image, its original coordinate data can be expressed as: (u, v); where u is the first coordinate (row coordinate) and v is the second coordinate (column coordinate).

[0040] In this embodiment, the parallelism N of the parallel de-distortion processing of the image is preset so that the programmable logic device can perform de-distortion processing on N original pixels in parallel. Wherein, N≥2, for example, N=2, 3, 4, etc., which can be determined based on the requirements of image size, FPGA logic resources, algorithm speed, etc.

[0041] Furthermore, a classification rule for classifying the coordinate data is preconfigured, and the classification rule is used to classify the coordinate data into N categories. Specifically, the classification rule is used to classify the original coordinate data of the original pixels to classify the original pixels into N categories, thereby determining the category corresponding to each original pixel, that is, the first category.

[0042] Optionally, the classification rule is used to classify the first coordinate into N categories. For example, if the first coordinate is a row coordinate, the classification rule is used to classify the row coordinates of each pixel.

[0043] For example, when a programmable logic device obtains the raw coordinate data of a raw pixel, it can obtain the raw coordinate data of one raw pixel in each clock cycle; for the obtained raw coordinate data, each obtained raw coordinate data is classified in turn according to the classification rule, so that one of the N categories to which each raw coordinate data belongs can be determined, and this category is called the first category.

[0044] It can be understood that since the parallelism of the programmable logic device is N, after determining the first category to which an original pixel belongs, the original pixel can be subsequently processed according to the first category to which the original pixel belongs, without waiting for the next original pixel to be classified.

[0045] Optionally, in order to facilitate rapid classification of programmable logic devices, classification is implemented by taking a remainder. Specifically, the classification rule is: taking N as a divisor, taking a remainder on the first coordinate, and different remainders correspond to different categories.

[0046] For example, the original coordinate data is (u, v), and the first coordinate u is modulo processed, and the result u%N can be 0, 1, 2, ..., N-1, etc., so that the original pixels can be classified simply and quickly; where % represents the modulo operation. For example, N = 4, then the original pixels correspond to four first categories: u%4 = 0, u%4 = 1, u%4 = 2, u%4 = 3.

[0047] Step S102, determining the corrected coordinate data of the corrected pixel corresponding to the original pixel after the original pixel is dedistorted according to the original coordinate data of the original pixel, and classifying the corrected coordinate data of the corrected pixel according to the classification rule to determine the second category to which the corrected pixel belongs.

[0048] In this embodiment, for each category of original pixels, the original coordinate data is dedistorted respectively in a parallel processing manner to determine the corresponding coordinate data. Specifically, for each category of original pixels, coordinate mapping processing is performed respectively in a parallel processing manner to determine the corrected coordinate data having a mapping relationship with the original coordinate data. The corrected coordinate data is used to represent the coordinate data of the corresponding pixel in the image generated after the original image is corrected (hereinafter referred to as the corrected image), and the pixel is called the corrected pixel. Accordingly, the coordinate data of the corrected pixel is called the corrected coordinate data. Similar to the original coordinate data, the corrected coordinate data also includes coordinates in two directions, that is, the corrected coordinate data includes the first coordinate and the second coordinate of the corrected pixel.

[0049] By dedistorting the original image, the corrected pixel corresponding to the original pixel can be determined, and the coordinate data of these two pixels have a mapping relationship. For example, the original coordinate number of the original pixel is (u, v), and after coordinate mapping processing, the dedistorted coordinate data (iu, iv) corresponding to the original coordinate data (u, v) can be determined. The coordinate data (iu, iv) is the corrected coordinate data of the corrected pixel.

[0050] For example, the coordinate mapping process may be implemented based on a coordinate mapping formula, which may be as follows:

[0051]

[0052] Among them, r 2 =u 2 +v 2 , k1, k2, k3, p1, p2 are all camera internal parameters, and each camera internal parameter can be determined by calibration.

[0053] In addition, since the camera intrinsic parameters are generally fixed, for a specific camera, the mapping relationship between its original coordinate number (u, v) and the corrected coordinate data (iu, iv) is also fixed. Therefore, a mapping table for recording the mapping relationship between each pixel coordinate data can be stored in advance. The corresponding corrected coordinate data (iu, iv) can also be determined by looking up the table to realize coordinate mapping processing.

[0054] This embodiment does not limit the coordinate mapping processing method and the coordinate mapping formula used.

[0055] By classifying the original coordinate data, the original coordinate data of different categories can be processed in parallel; however, since image distortion is manifested as squeezing or stretching of the image, and the closer to the edge, the more obvious the change, the distortion model is nonlinear. As shown in the coordinate mapping formula output above, the mapping relationship between the two coordinates is nonlinear; for the original coordinate data of a certain category (i.e., the first category), after the nonlinear coordinate mapping processing is performed on it, the mapped corrected coordinate data is generally irregular, which will affect the speed of parallel processing.

[0056] Therefore, for the corrected pixels, the corrected coordinate data is also classified according to the same classification rule, so that the corrected pixels can also be divided into N categories. For the convenience of distinguishing and describing, the category to which the corrected pixels belong is called the second category. It can be understood that since the same classification rule is used to classify the original pixels and the corrected pixels, the first category and the second category determined by the classification are both one of the N categories; and since the mapping relationship is nonlinear, for an original pixel belonging to a certain first category, the second category to which the corresponding corrected pixel belongs may be any one.

[0057] Optionally, if the classification rule is used to classify the first coordinate into N categories, the second category to which the correction pixel belongs is determined by classifying the first coordinate of the correction pixel according to the classification rule. Specifically, the classification rule is: taking N as a divisor and performing a modulo process on the first coordinate.

[0058] For example, the classification rule is implemented using the above-mentioned remainder processing, the original coordinate data (u, v) is mapped to the corrected coordinate data (iu, iv), and u%N represents the first category, and iu%N represents the second category; assuming that the first category to which the original coordinate data (u, v) belongs is u%N=0, then the second category iu%N to which the corresponding corrected pixel belongs may be any value among 0, 1, 2, etc.

[0059] Step S103, cache the corresponding coordinate data to the corresponding first storage space according to the first category to which the original pixel belongs and the second category to which the corresponding corrected pixel belongs; wherein a corresponding first storage space group is pre-configured for each first category, and the first storage space group includes the first storage space corresponding to each second category; the coordinate data includes the original coordinate data of the original pixel and the corrected coordinate data of the corrected pixel.

[0060] If the corrected pixels obtained by mapping the original pixels are directly processed, since the corrected coordinate data of multiple corrected pixels can be determined in parallel, and the categories of the corrected pixels are disordered, multiple corrected pixel data belonging to the same second category (for example, iu%N=0) may be determined in one processing cycle (for example, one clock cycle), resulting in a blocking problem when the corrected pixel data of the second category (for example, iu%N=0) is subsequently processed. There will also be a blocking problem for other corrected pixels of the second category, resulting in a low overall processing speed and failing to achieve a parallel processing effect with a parallel degree of N. In this embodiment, the problem of blocking can be effectively solved by reclassifying the corrected pixels and setting multiple first storage spaces for each processing path.

[0061] Specifically, for the first category to which the original pixel belongs, a set including multiple storage spaces can be pre-configured for each first category. For ease of description, the storage space is referred to as a first storage space, and the set is referred to as a first storage space group; and each first storage space in the first storage space group corresponds to a second category. As described above, the number of types of the first category and the second category is N, so N first storage space groups can be pre-configured, and each first storage space group includes N first storage spaces, that is, a total of N×N first storage spaces are pre-configured.

[0062] Among them, the first storage space group corresponds to the first category one by one, and the first storage space in the first storage space group corresponds to the second category one by one. Therefore, according to the first category to which the original pixel belongs, the first storage space group corresponding to the original pixel can be determined; and according to the second category to which the corrected pixel corresponding to the original pixel belongs, the first storage space corresponding to the corrected pixel in the first storage space group corresponding to the original pixel can be determined, and then the original coordinate data of the original pixel and the corrected coordinate data of the corrected pixel are cached in the corresponding first storage space. For the convenience of description, the original coordinate data of the original pixel and the corrected coordinate data of the corrected pixel are collectively referred to as coordinate data.

[0063] Step S104 , taking the first storage spaces corresponding to the same second category in each first storage space group as a group, respectively acquiring the coordinate data to be processed in each group of first storage spaces.

[0064] In this embodiment, although N×N first storage spaces are provided, since the degree of parallelism is N, generally only N coordinate data are processed at a certain moment; and since the camera distortion range is generally small, for example, the distortion range of a general industrial camera is less than 2%, for all correction pixels, the number of pixels belonging to each second category is not much different, for example, the number of correction pixels of iu%4=0 is not much different from the number of correction pixels of iu%4=1 or iu%4=2; therefore, for N first storage space groups, the first storage spaces corresponding to the same second category are taken as a group, and generally only one first storage space in the group corresponding to the same second category stores data to be processed. At this time, the coordinate data in each group of the first storage spaces are obtained respectively according to the regrouping, that is, the coordinate data to be processed, and then the coordinate data to be processed of each group are continued to be processed in parallel.

[0065] Optionally, a second storage space may be set for each second type, and the coordinate data to be processed may be cached based on the second storage space. The above step S104 "taking the first storage spaces corresponding to the same second category in each first storage space group as a group, respectively obtaining the coordinate data to be processed in each group of first storage spaces" may include: taking the first storage spaces corresponding to the same second category in each first storage space group as a group, respectively reading the coordinate data to be processed in each group of first storage spaces; caching the read coordinate data to be processed in the corresponding second storage space; wherein the corresponding second storage space is pre-configured for each second category.

[0066] In this embodiment, N second storage spaces may be preconfigured, wherein the second storage spaces correspond to the second categories one by one. The coordinate data to be processed of each group is read respectively, and the read coordinate data to be processed is cached in the corresponding second storage space, so as to facilitate the subsequent processing of the coordinate data to be processed in each second storage space.

[0067] Figure 2 A schematic diagram of cached coordinate data is shown. Figure 2 As shown, the parallelism N=3, wherein A, B, and C represent three first categories, and a, b, and c represent three second categories; at this time, three first storage space groups can be pre-configured, namely, first storage space group A, first storage space group B, and first storage space group C; and the first storage space group A includes first storage space Aa, first storage space Ab, and first storage space Ac, similarly, the first storage space group B includes first storage space Ba, first storage space Bb, and first storage space Bc, and the first storage space group C includes first storage space Ca, first storage space Cb, and first storage space Cc. In addition, three second storage spaces are pre-configured, namely, second storage space a, second storage space b, and second storage space c.

[0068] Among them, classification is performed by taking the remainder, and the remainder results 0, 1, and 2 correspond to the first categories A, B, and C, respectively, and also correspond to the second categories a, b, and c. Figure 2 As shown, the original coordinate data (u1, v1), the original coordinate data (u2, v2), and the original coordinate data (u3, v3) correspond to different first categories, and u1%3=0, u2%3=1, and u3%3=2, then the three original coordinate data correspond to the first storage space group A, the first storage space group B, and the first storage space group C. In addition, the three original coordinate data are mapped to the corrected coordinate data (iu1, iv1), the corrected coordinate data (iu2, iv2), and the corrected coordinate data (iu3, iv3), respectively.

[0069] Assuming that iu1%3=1, iu2%3=0, iu3%3=3, the correction coordinate data (iu1, iv1) belongs to the second type of the second category (i.e., the second category b), the correction coordinate data (iu2, iv2) belongs to the first type of the second category (i.e., the second category a), and the correction coordinate data (iu3, iv3) belongs to the third type of the second category (i.e., the second category c). Therefore, the three correction coordinate data correspond to the first storage space Ab, the first storage space Ba, and the first storage space Cc, respectively, so the corresponding coordinate data can be stored in these three first storage spaces; for example, the coordinate data stored in the first storage space Ab is: (u1, v1, iu1, iv1).

[0070] For the second storage space a, the first storage space Aa, the first storage space Ba, and the first storage space Ca can be read, so that the coordinate data (u2, v2, iu2, iv2) in the first storage space Ba can be read. Similarly, for the second storage space b, the first storage space Ab, the first storage space Bb, and the first storage space Cb can be read, so that the coordinate data (u1, v1, iu1, iv1) in the first storage space Ab can be read. For the second storage space c, the first storage space Ac, the first storage space Bc, and the first storage space Cc can be read, so that the coordinate data (u3, v3, iu3, iv3) in the first storage space Cc can be read.

[0071] In this embodiment, a first storage space group including N first storage spaces is set for each processing path, which is equivalent to processing the data of the processing path at N times the rate, thereby effectively avoiding blocking of a processing path; and, the first storage space belonging to the same second category is read by the second storage space, and multi-level storage space is used to ensure that subsequent parallel processing is still performed with a parallelism of N, and the required processing resources will not increase. The overall acceleration effect of nearly N times can be achieved.

[0072] Among them, due to the distortion of the image, it may cause a second storage space to read coordinate data from multiple (generally up to 2) first storage spaces at some moments, and at this time it needs to be processed in a serial manner; but since the distortion range of industrial cameras is small (less than 2%), the probability of this situation occurring is low. Compared with the traditional method of using a large amount of storage resources for parallel processing, this method can still approach the full parallel processing speed to a large extent. The specific speed depends on the size of the image distortion, but generally it can achieve a processing speed of not less than 80% of the full parallel rate, that is, using fewer storage resources, it can achieve an effect close to full parallel processing.

[0073] It should be noted that although this embodiment uses a large number of storage spaces, since the first storage space and the second storage space are only used to cache data, the data that has been read can be cleared in time, so the size of each storage space can be very small and will not occupy too many storage resources. For example, the first storage space and the second storage space can cache data in a FIFO (First Input First Output) manner.

[0074] Step S105 , determining the correction pixel values ​​corresponding to the respective correction pixels according to the coordinate data to be processed of each group, and generating a correction image according to the correction pixel values.

[0075] In this embodiment, the coordinate data in each processing path is classified according to the second category corresponding to the first coordinate in the corrected coordinate data. Therefore, for each corrected coordinate data of the second category, the pixel values ​​corresponding to each corrected pixel, i.e., the corrected pixel values, can be determined separately in a parallel processing manner, and then an image, i.e., the corrected image, is generated according to the corrected pixel values ​​of each corrected pixel, thereby achieving dedistortion of the original image and obtaining the corrected image.

[0076] The image correction method provided in this embodiment is divided into N categories according to the coordinate data of the original pixels in the original image, so that the dedistortion processing can be performed in parallel according to the parallelism degree N, and there is no need to store N copies of the original image, which greatly reduces the required storage resources; and for the irregular correction pixels after dedistortion, N×N first storage spaces are used to accelerate each processing path, effectively avoiding the problem of blocking, and parallel processing can be achieved. The programmable logic device as a whole can basically dedistort the original image at N times the rate, achieving an acceleration effect close to N times. While realizing multiple parallel dedistortion processing, this method can save on-chip storage resources, so that programmable logic devices such as FPGA can be applied to dedistortion processing of large-size, high-frame-rate images.

[0077] In this embodiment, an image correction method is provided, which can be applied to a programmable logic device with parallel processing capability, such as FPGA, CPLD, etc. Figure 3 is a flow chart of an image correction method according to an embodiment of the present invention. Figure 3 As shown, the process includes the following steps.

[0078] Step S301, obtaining original coordinate data of original pixels belonging to different first categories; the original coordinate data includes a first coordinate and a second coordinate of the original pixel; the first category to which the original pixel belongs is determined by classifying the first coordinate of the original pixel according to a preset classification rule; the classification rule is used to classify the first coordinate into N categories, where N is a parallelism of not less than 2.

[0079] For details, please see Figure 1 Step S101 of the illustrated embodiment will not be described in detail here.

[0080] Optionally, when acquiring the original coordinate data of the original pixel, the original coordinate data of each original pixel in the original image may be acquired sequentially. For example, the original coordinate data of each original pixel in the original image may be acquired sequentially along a first direction, so that when the original coordinate data is subsequently classified according to the first coordinate, the continuously acquired original coordinate data may be divided into different processing paths to achieve parallel processing.

[0081] The original coordinate data of the original pixel in the original image to be corrected may be first obtained; then the first coordinate of the original pixel may be classified according to a preset classification rule to determine the first category to which the original pixel belongs.

[0082] Alternatively, the original coordinate data of each original pixel may be generated in the form of counts. Specifically, the above step S301 "obtaining the original coordinate data of original pixels belonging to different first categories" may include: with a parallelism of N, executing the process of generating the original coordinate data in parallel.

[0083] Furthermore, generating original coordinate data may specifically include steps A1 to A3.

[0084] Step A1, presetting and determining the starting coordinates of the first coordinates and the width of the first coordinates, as well as the starting coordinates of the second coordinates and the width of the second coordinates.

[0085] Step A2, in the current counting cycle, under the constraints of the width of the first coordinate and the width of the second coordinate, determine the first count of the first coordinate and the second count of the second coordinate corresponding to the current count; the step of the first count is N, and the step of the second count is 1.

[0086] Step A3, determining the first coordinate of the original pixel acquired in the current counting cycle according to the starting coordinate of the first coordinate and the first count; determining the second coordinate of the original pixel acquired in the current counting cycle according to the starting coordinate of the second coordinate and the second count.

[0087] Since the original coordinate data is the number of rows and columns of the original pixels and is arranged in order, the original coordinate data corresponding to each original pixel is generated in the form of counting in this embodiment to achieve the acquisition of the original coordinate data. For example, the original coordinate data can be generated after the processing start signal is detected.

[0088] Specifically, when the original coordinate data needs to be generated, the starting coordinate uv_hstart of the current first coordinate, the width uv_hsize of the first coordinate, and the starting coordinate uv_vstart of the second coordinate, and the width uv_vsize of the second coordinate are configured. For a high-resolution original image, it can be divided into multiple image blocks (Block), and the original coordinate data of the original pixels in each image block are generated respectively; accordingly, the starting coordinate and the corresponding width need to be determined based on the position of the image block in the original image.

[0089] Furthermore, the counting step of the first coordinate is set to N, and the counting step of the second coordinate is set to 1. Each clock cycle can be regarded as a counting cycle, and in the current counting cycle, the corresponding count, i.e., the current count, can be determined. Furthermore, according to the two count steps set, the count of the first coordinate corresponding to the current count (i.e., the first count) and the count of the second coordinate (i.e., the second count) can be determined.

[0090] The width of the first coordinate is used to constrain the first count of the first coordinate, and the first count of the first coordinate does not exceed the width of the first coordinate. The width of the second coordinate is used to constrain the second count of the second coordinate, and the second count of the second coordinate does not exceed the width of the second coordinate. For example, if the original coordinate data of each original pixel in the original image is generated in sequence along the first direction, as the current count increases, the first count remains unchanged, and the second count increases by step 1; when the second count exceeds the width of the second coordinate, the second count returns to zero, and the first count increases by step N. This is repeated until the first count also reaches the width of the first coordinate.

[0091] In each counting cycle, the first count can be added based on the starting coordinate of the first coordinate to generate the first coordinate of the original pixel. Similarly, the second count can be added based on the starting coordinate of the second coordinate to generate the second coordinate of the original pixel, thereby generating the original coordinate data of the original pixel. The original coordinate data to be processed can be generated simply and quickly in a counting manner without reading data.

[0092] In this embodiment, corresponding modules may be configured in a programmable logic device, and image dedistortion may be achieved based on each module. Figure 4 A system architecture diagram of a programmable logic device is shown, which performs parallel processing using the parallelism N=4 as an example, and includes a coordinate main module U0, a mapping main module U1, a cache main module U2, a pixel processing main module U3 and an image generation main module U4.

[0093] like Figure 4 As shown, if N=4, for the coordinate main module U0, four coordinate units can be instantiated, and each coordinate unit generates original coordinate data of each category in parallel.

[0094] For each coordinate unit, the initial value of the first count can be set to N different values, such as 0, 1, 2, ..., N-1, etc. For example, if N=4, the initial values ​​of the first count in the four coordinate units can be set to 0, 1, 2, and 3 respectively. The initial values ​​of the second count can all be set to 0.

[0095] Taking N=4 as an example, each coordinate unit sets the starting coordinate uv_hstart of the row coordinate (first coordinate), the width uv_hsize of the first coordinate, and the starting coordinate uv_vstart of the column coordinate (second coordinate), the width uv_vsize of the second coordinate, and each coordinate unit sets the initial count value code of the row coordinate (i.e. the initial value of the first count), the codes of the four coordinate units can be 0, 1, 2, 3 respectively, the row count step is 4, and the column count step is 1. When the row width of the original image is not an integer multiple of 4, it can be padded or truncated to an integer multiple of 4.

[0096] The row counts of the four coordinate units are generated according to the rule of 0, 1, 2, and 3 modulo 4, for example: 0, 4, 8, 12..., 1, 5, 9, 13..., 2, 6, 10, 14..., 3, 7, 11, 15... The row coordinates can be determined by adding the starting coordinate uv_hstart of the row coordinates to the counts of each row.

[0097] Step S302, coordinate mapping processing is performed on each category of original pixels respectively; the coordinate mapping processing is used to determine the corrected coordinate data of the corrected pixels corresponding to the original pixels after dedistortion according to the original coordinate data of the original pixels; the corrected coordinate data includes the first coordinate and the second coordinate of the corrected pixels.

[0098] like Figure 4 As shown, N=4, for the mapping main module U1, four mapping units can be instantiated, and each mapping unit performs coordinate mapping processing on the generated original coordinate data (u, v) in a parallel manner to determine the corresponding corrected coordinate data (iu, iv).

[0099] Among them, each original coordinate data (u, v) can be input into the corresponding mapping unit to output the corresponding corrected coordinate data (iu, iv). In addition, since the original coordinate data (u, v) will be used later, the mapping unit can transparently transmit the original coordinate data (u, v).

[0100] Step S303: classify the first coordinates of the corrected pixel according to the classification rule to determine the second category to which the corrected pixel belongs.

[0101] For details, please see Figure 1 The relevant description of step S102 of the illustrated embodiment will not be repeated here.

[0102] Step S304, cache the corresponding coordinate data to the corresponding first storage space according to the first category to which the original pixel belongs and the second category to which the corresponding corrected pixel belongs; wherein a corresponding first storage space group is pre-configured for each first category, and the first storage space group includes the first storage space corresponding to each second category; the coordinate data includes the original coordinate data of the original pixel and the corrected coordinate data of the corrected pixel.

[0103] For details, please see Figure 1 Step S103 of the illustrated embodiment will not be described in detail here.

[0104] Step S305 , taking the first storage spaces corresponding to the same second category in each first storage space group as a group, respectively acquiring the coordinate data to be processed in each group of first storage spaces.

[0105] For details, please see Figure 1 Step S104 of the illustrated embodiment will not be described in detail here.

[0106] In some optional implementations, as described above, step S305 can be implemented by reading the coordinate data to be processed in each group of first storage spaces, and storing the read coordinate data to be processed in the second storage space. The process of reading the coordinate data to be processed can specifically include: determining whether there is a non-empty first storage space in the first storage space corresponding to the same second category in each first storage space group; if there is a non-empty first storage space, reading the coordinate data in the non-empty first storage space.

[0107] like Figure 4As shown, the cache main module U2 is configured with 4 first storage space groups, each of which includes 4 first storage spaces, and is also configured with 4 second storage spaces. For each mapping unit, it may output four second categories of correction coordinate data (iu, iv): iu%4=0, 1, 2, 3; and each second storage space only reads the correction coordinate data of the same second category. Specifically, for each second storage space, it can obtain data from four first storage spaces from 4 channels, and quickly read the data in the way of reading whichever is not empty; when there are multiple first storage spaces that are not empty, the second storage spaces can be processed serially, which will not affect other processing paths.

[0108] Each group of 4 first storage spaces caches 4 pixel row coordinates iu%4 as 0, 1, 2, 3 coordinate data, including u, v, iu, iv; each second storage space caches 4 pixel row coordinates u%4 as 0, 1, 2, 3 coordinate data, including u, v, iu, iv. Figure 4 As shown, the four second storage spaces read and store the correction coordinate data (iu, iv) of iu%4=0, iu%4=1, iu%4=2, and iu%4=3 respectively.

[0109] Step S306, reading the original image data corresponding to the original image; the original image data includes the original pixel value and original coordinate data of each original pixel.

[0110] In this embodiment, the programmable logic device also needs to read the original image data corresponding to the original image, and the original image data can be the data of the entire original image or the data of a certain image block in the original image. The original image data includes the original pixel value and original coordinate data of each original pixel, and the original coordinate data is in one-to-one correspondence with the original coordinate data obtained in the above step S301.

[0111] It can be understood that if the original coordinate data is obtained by generating in step S301, step S306 can be executed in parallel to read the original coordinate data of each original pixel from the original image. If the original coordinate data is also obtained by reading in step S301, the original coordinate data can be subsequently classified, mapped, etc. after the original image data is obtained in step S306. Therefore, step S301 and step S306 can be executed successively or in parallel, and this embodiment does not limit the execution order.

[0112] Step S307 , according to the first category to which each original pixel belongs, cache the original pixel values ​​of the original pixels into corresponding first memories respectively; wherein a corresponding first memory is pre-configured for each first category.

[0113] In this embodiment, in order to ensure parallel processing, N first memories are further set, and each first memory corresponds to a first category; as mentioned above, each original pixel corresponds to a unique first category, so based on the first category to which the original pixel value belongs, the corresponding original pixel value can be cached in the corresponding first memory respectively.

[0114] like Figure 4 As shown, the original image data can be stored in a cache, such as a DDR (Double Data Rate) cache; the original image data is read from the cache, and the original pixel values ​​can be cached based on the corresponding first category. N=4, then the four first memories respectively store the original pixel values ​​corresponding to u%4=0, u%4=1, u%4=2, and u%4=3. Among them, the first memories store each original pixel value in sequence so that the corresponding storage address can be determined based on the coordinate data.

[0115] Step S308 , determining the correction pixel values ​​corresponding to the respective correction pixels according to the coordinate data to be processed of each group, and generating a correction image according to the correction pixel values.

[0116] Specifically, the above step S308 of “determining the correction pixel value corresponding to each correction pixel according to each group of coordinate data to be processed” includes steps S3081 and S3082.

[0117] Step S3081, determining a corresponding first storage address according to the corrected coordinate data in the coordinate data to be processed.

[0118] Step S3082, obtaining the original pixel value corresponding to the first storage address from the first memory corresponding to the coordinate data to be processed, and using the original pixel value corresponding to the first storage address as the corrected pixel value of the corrected pixel in the coordinate data to be processed.

[0119] In this embodiment, although each first memory corresponds to a corresponding first category, since the first category and the second category are determined based on the same classification rule, each first memory also corresponds to a unique second category. Wherein, for each group of coordinate data to be processed, for example, the coordinate data to be processed read from the corresponding second storage space, since the corrected coordinate data (iu, iv) therein is regular, they all correspond to only the same second category, and the original coordinate data (u, v) may correspond to multiple rules, so the corrected pixel value is determined based on the corrected coordinate data (iu, iv) of the rule.

[0120] Specifically, for the corrected coordinate data (iu, iv) in the coordinate data to be processed, a corresponding storage address, i.e., a first storage address, can be determined based on the corrected coordinate data (iu, iv). The first storage address is used to store the original pixel value whose original coordinate data is the corrected coordinate data (iu, iv). The original pixel value is used as the corrected pixel value of the corresponding corrected pixel, and a dedistorted corrected image can be generated based on each corrected pixel value.

[0121] For example, the original pixel in the 1st row and 1st column of the original image (i.e., u=v=1) has coordinate mapping results of iu=6, iv=7, i.e., the corrected coordinate data is (6,7). A first storage address can be generated based on the corrected coordinate data (6,7), and the first storage address is used to store the original pixel value P in the 6th row and 7th column of the original image. 6,7 , the original pixel value P 6,7 When the corrected image is subsequently generated, the pixel value of the corrected pixel in the first row and first column of the corrected image is set to the original pixel value P 6,7 , and finally the corrected pixels can be obtained.

[0122] In this embodiment, a first memory is set for each processing path to ensure that each corrected pixel value can be determined in parallel. In addition, the first storage address is determined based on the corrected coordinate data to query the pixel value in the first memory, which can be applied to the regular corrected coordinate data in each group of data to be processed to avoid blocking.

[0123] In some optional implementations, the step S308 of “generating a corrected image according to the corrected pixel values” may include steps B1 to B3.

[0124] Step B1, caches the corrected pixel value to the corresponding third storage space according to the first category to which the original pixel in the coordinate data to be processed belongs and the second category to which the corrected pixel in the coordinate data to be processed belongs; wherein a corresponding third storage space group is pre-configured for each second category, and the third storage space group includes a third storage space corresponding to each first category.

[0125] Step B2: taking the third storage spaces corresponding to the same first category in each third storage space group as a group, and respectively obtaining the corrected pixel values ​​in each group of third storage spaces.

[0126] Step B3, determining the corresponding second storage address according to the original coordinate data in the coordinate data to be processed, recording the pixel value corresponding to the second storage address as the correction pixel value, and generating a correction image determined based on each correction pixel value.

[0127] In this embodiment, it is necessary to determine the position of the corrected pixel value in the corrected image based on the original coordinate data (u, v), but for each set of coordinate data to be processed, the original coordinate data (u, v) belongs to different first categories. If the corrected pixel is generated directly based on the original coordinate data (u, v), there may also be a blocking problem. Similar to the above-mentioned method of setting the first storage space group, another storage space group is also set, namely the third storage space group, but the third storage space group corresponds one-to-one to each second category; and the third storage space group includes N third storage spaces, and each third storage space corresponds one-to-one to the first category. There are a total of N×N third storage spaces.

[0128] According to the second category to which the corrected pixel in the coordinate data to be processed belongs, the corresponding third storage space group can be determined, and according to the first category to which the original pixel in the coordinate data to be processed belongs, the third storage space corresponding to the original pixel in the corresponding third storage space group can be further determined, and then the corresponding corrected pixel value is stored in the third storage space. The original coordinate data can also be stored in the third storage space, so that the original coordinate data in the third storage space can be directly read, and then the corresponding second storage address is determined, and the second storage address is the address for recording the pixel value of a pixel in the corrected image.

[0129] After determining the second storage address, the corrected pixel value can be recorded to the second storage address; since the second storage address is generated based on the original coordinate data, the corrected pixel value can be used as the pixel value of the pixel corresponding to the original coordinate data in the corrected image, thereby obtaining a de-distorted corrected image.

[0130] Optionally, step B2 of “taking the third storage spaces corresponding to the same first category in each third storage space group as a group and respectively obtaining the corrected pixel values ​​in each group of third storage spaces” may include steps B21 to B22.

[0131] In step B21, the third storage spaces corresponding to the same first category in each third storage space group are taken as a group, and the corrected pixel values ​​in each group of third storage spaces are read respectively.

[0132] Step B22, caching the read corrected pixel value into a corresponding fourth storage space; wherein a corresponding fourth storage space is pre-configured for each first category.

[0133] In this embodiment, similar to the above-mentioned setting of the second storage space, a corresponding fourth storage space is further set for each group of the third storage space, and the fourth storage space corresponds to the first category one by one, that is, the fourth storage space is used to store data belonging to the same first category, that is, the corrected pixel value. Among them, the fourth storage space can also store the original coordinate data in the coordinate data to be processed, and the first category to which the original coordinate data belongs is consistent with the first category corresponding to the fourth storage space.

[0134] The third storage space and the fourth storage space can also cache data in a FIFO manner without occupying large storage resources.

[0135] Optionally, the above step B3 of “recording the pixel value corresponding to the second storage address as the corrected pixel value” may include step B31.

[0136] Step B31, alternately read the second storage address according to the first buffer and the second buffer, and write the corrected pixel value to the second storage address of the second memory, wherein when the first buffer performs a read operation, the second buffer performs a write operation; when the first buffer performs a write operation, the second buffer performs a read operation.

[0137] In this embodiment, a memory for storing the corrected image, namely, a second memory, is provided, and the generated second storage address is a certain address in the second memory.

[0138] In addition, two buffers for alternating reading and writing are set in each processing path, namely, the first buffer. When the first buffer reads the second storage address, the second buffer is used to write another second storage address read by itself to the second storage address of the second memory. Conversely, when the first buffer writes a second storage address to the second storage address of the second memory, the second buffer is used to read another second storage address. By alternating reading and writing between the two buffers, reading and writing can be delayed without delay, thereby ensuring reading and writing efficiency.

[0139] like Figure 4 As shown, in the pixel processing main module U3, a corresponding first address calculation unit and a first memory are set for each processing path, and the first address calculation unit is used to calculate the first storage address corresponding to the correction coordinate data, and then read the correction pixel value from the corresponding first memory.

[0140] In the image generation main module U4, a third storage space group (including N third storage spaces), a fourth storage space and a second address calculation unit are set for each processing path. According to the first category u%4 to which the original coordinate data (u, v) in the coordinate data to be processed belongs, the corrected pixel value and the original coordinate data (u, v) are stored in the corresponding third storage space.

[0141] Furthermore, the fourth storage space reads and records the data stored in the third storage spaces belonging to the same first category in each third storage space group, and the data may include the corrected pixel value and the original coordinate data. Afterwards, the corresponding second address calculation unit obtains the original coordinate data and the corrected pixel value in the third storage space, and determines the second storage address based on the original coordinate data, and then the corrected pixel value may be stored in the corresponding second storage address.

[0142] like Figure 4 As shown, the image generation main module U4 may also include a first buffer and a second buffer that are read alternately to ensure reading and writing efficiency.

[0143] It is understandable that the second storage space and the fourth storage space may not be provided, and the corresponding address calculation unit may directly read the corresponding data from each storage space group (the first storage space group, the third storage space group) and perform address calculation processing.

[0144] Figure 5 A detailed flow chart of the image correction method is shown in FIG. Figure 4 The architecture shown in the figure, the process of the programmable logic device dedistorting the original image is as follows.

[0145] The programmable logic device receives the raw image data to be processed and writes it into a cache, which may be, for example, a DDR cache or other on-chip or off-chip memory.

[0146] It is determined in real time whether there is enough data in the cache. If the data is sufficient, the original image data can be read by block, that is, a request to read the image block is initiated; if it is not the first time to read the image block, it is also necessary to determine whether the currently read image block has been processed and whether the current image block is the last image block. If the last image block has been processed, the process ends.

[0147] When reading the data of the image block, the pixel values ​​can be read in parallel, for example, 16 pixel values ​​are read per clock cycle, and the read pixel values ​​are cached in parallel to the corresponding storage area (such as FIFO, etc.). And the first category to which each original pixel belongs can be determined according to the u%4 rule, so that the corresponding input is written into the corresponding first memory. The first memory can be, for example, RAM, such as URAM or distributed RAM, etc., or it can be replaced by off-chip expansion storage, which is not limited in this embodiment.

[0148] Furthermore, when a data block read request is initiated, a start signal may be generated, so that each coordinate unit generates original coordinate data (u, v) in parallel based on the start signal. Each coordinate unit calculates corrected coordinate data (iu, iv) corresponding to the original coordinate data (u, v) in parallel.

[0149] For each processing path, the second category is determined according to the iu%4 rule, and the coordinate data is written into the corresponding first storage space. The second storage space reads the non-empty first storage space, and writes the read coordinate data into the second storage space.

[0150] For each processing path, each first address calculation unit calculates the first storage address corresponding to the correction coordinate data (iu, iv), and reads the corresponding first memory based on the first storage address to obtain the correction pixel value. In addition, for each processing path, the corresponding first category is determined again according to the u%4 rule, and written into the corresponding third storage space, and then the correction pixel value is written into the fourth storage space, and finally the second address calculation unit is used to calculate the second storage address, and the correction pixel value is written into the second memory through alternating reading and writing of the first buffer and the second buffer, and finally the entire correction image is stored in the second memory.

[0151] The second memory may also be a RAM, for example, a URAM or a distributed RAM, etc., or an off-chip extended storage may be used instead, which is not limited in this embodiment.

[0152] The image correction method provided in this embodiment sets N first memories, and each first memory records the original pixel values ​​of different categories respectively to ensure parallel processing. The first storage address is generated based on the rule-based correction coordinate data, which can be unaffected by the various first category coordinate data in the second storage space, and the correction pixel value can be determined in parallel, effectively avoiding blocking. When generating a corrected image, N×N third storage spaces are used to accelerate each processing path, effectively avoiding the problem of blocking, and parallel processing can be achieved. The first storage space and the second storage space can form a cascade storage structure, and the third storage space and the fourth storage space can also form a cascade storage structure. This cascade storage structure can ensure stable data reading and writing.

[0153] In this embodiment, an image correction device is also provided, which is used to implement the above-mentioned embodiments and preferred implementation modes, and the descriptions that have been made will not be repeated. As used below, the term "module" can be a combination of software and / or hardware that implements a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceivable.

[0154] This embodiment provides an image correction device, which is applied to a programmable logic device, such as Figure 6 As shown, including:

[0155] The coordinate acquisition module 601 is used to acquire original coordinate data of original pixels belonging to different first categories; the first category to which the original pixels belong is determined by classifying the original coordinate data of the original pixels according to a preset classification rule;

[0156] A classification module 602 is used to determine, according to the original coordinate data of the original pixel, the corrected coordinate data of the corrected pixel corresponding to the original pixel after the original pixel is dedistorted, and classify the corrected coordinate data of the corrected pixel according to the classification rule to determine the second category to which the corrected pixel belongs;

[0157] The coordinate cache module 603 is used to cache the corresponding coordinate data to the corresponding first storage space according to the first category to which the original pixel belongs and the second category to which the corresponding corrected pixel belongs; wherein a corresponding first storage space group is pre-configured for each first category, and the first storage space group includes the first storage space corresponding to each second category; the coordinate data includes the original coordinate data of the original pixel and the corrected coordinate data of the corrected pixel;

[0158] A group acquisition module 604 is used to take the first storage spaces corresponding to the same second category in each of the first storage space groups as a group, and respectively acquire the coordinate data to be processed in each group of the first storage spaces;

[0159] The processing module 605 is used to determine the correction pixel value corresponding to each correction pixel according to each group of the coordinate data to be processed, and generate a correction image according to the correction pixel value.

[0160] In some optional embodiments, the original coordinate data includes the first coordinate and the second coordinate of the original pixel; the corrected coordinate data includes the first coordinate and the second coordinate of the corrected pixel; the classification rule is used to classify the first coordinate into N categories, where N is a parallelism not less than 2;

[0161] The first category to which the original pixel belongs is determined by classifying the first coordinate of the original pixel according to the classification rule;

[0162] The second category to which the correction pixel belongs is determined by classifying the first coordinates of the correction pixel according to the classification rule.

[0163] In some optional implementations, the coordinate acquisition module 601 acquires original coordinate data of original pixels belonging to different first categories, including:

[0164] The process of generating the original coordinate data is performed in parallel with a parallelism degree N, and the generating the original coordinate data includes:

[0165] Presetting and determining the starting coordinates of the first coordinate and the width of the first coordinate, as well as the starting coordinates of the second coordinate and the width of the second coordinate;

[0166] In a current counting cycle, under the constraints of the width of the first coordinate and the width of the second coordinate, determining a first count of the first coordinate and a second count of the second coordinate corresponding to the current count; the step of the first count is N, and the step of the second count is 1;

[0167] The first coordinates of the original pixel acquired in the current counting cycle are determined according to the starting coordinates of the first coordinates and the first count; the second coordinates of the original pixel acquired in the current counting cycle are determined according to the starting coordinates of the second coordinates and the second count.

[0168] In some optional implementations, the classification rule is: taking N as the divisor, performing remainder processing on the first coordinate, and different remainders correspond to different categories.

[0169] In some optional implementations, the group acquisition module 604 takes the first storage spaces corresponding to the same second category in each of the first storage space groups as a group, and respectively acquires the coordinate data to be processed in each group of the first storage spaces, including:

[0170] Taking the first storage spaces corresponding to the same second category in each of the first storage space groups as a group, respectively reading the coordinate data to be processed in each group of the first storage spaces;

[0171] The read coordinate data to be processed is cached in a corresponding second storage space; wherein a corresponding second storage space is pre-configured for each second category.

[0172] In some optional implementations, the group acquisition module 604 takes the first storage spaces corresponding to the same second category in each of the first storage space groups as a group, and reads the coordinate data to be processed in each group of first storage spaces respectively, including:

[0173] Determine whether there is a non-empty first storage space among the first storage spaces corresponding to the same second category in each of the first storage space groups;

[0174] In the case that there is a non-empty first storage space, the coordinate data in the non-empty first storage space is read, and the coordinate data in the non-empty first storage space is used as the coordinate data to be processed.

[0175] In some optional embodiments, the device further comprises:

[0176] An image reading module, used for reading original image data corresponding to the original image to be corrected; the original image data includes original pixel values ​​and original coordinate data of each original pixel;

[0177] A pixel cache module, configured to cache the original pixel values ​​of the original pixels to corresponding first memories respectively according to the first categories to which the original pixels belong; wherein a corresponding first memory is pre-configured for each first category;

[0178] The processing module 605 determines the correction pixel value corresponding to each correction pixel according to each group of the coordinate data to be processed, including:

[0179] Determine a corresponding first storage address according to the corrected coordinate data in the coordinate data to be processed;

[0180] The original pixel value corresponding to the first storage address is obtained from the first memory corresponding to the coordinate data to be processed, and the original pixel value corresponding to the first storage address is used as the corrected pixel value of the corrected pixel in the coordinate data to be processed.

[0181] In some optional implementations, the processing module 605 generates a corrected image according to the corrected pixel value, including:

[0182] According to the first category to which the original pixel in the coordinate data to be processed belongs and the second category to which the corrected pixel in the coordinate data to be processed belongs, the corrected pixel value is cached in a corresponding third storage space; wherein a corresponding third storage space group is pre-configured for each second category, and the third storage space group includes a third storage space corresponding to each first category respectively;

[0183] Taking the third storage spaces corresponding to the same first category in each of the third storage space groups as a group, respectively acquiring the corrected pixel values ​​in each group of the third storage spaces;

[0184] The corresponding second storage address is determined according to the original coordinate data in the coordinate data to be processed, the pixel value corresponding to the second storage address is recorded as the correction pixel value, and a correction image determined based on each of the correction pixel values ​​is generated.

[0185] In some optional implementations, the processing module 605 takes the third storage spaces corresponding to the same first category in each of the third storage space groups as a group, and respectively obtains the corrected pixel values ​​in each group of the third storage spaces, including:

[0186] The third storage spaces corresponding to the same first category in each of the third storage space groups are taken as a group, and the corrected pixel values ​​in each group of the third storage spaces are read respectively;

[0187] The read corrected pixel value is cached in a corresponding fourth storage space; wherein a corresponding fourth storage space is pre-configured for each first category.

[0188] In some optional implementations, the processing module 605 records the pixel value corresponding to the second storage address as the corrected pixel value, including:

[0189] Alternately reading the second storage address according to the first buffer and the second buffer, and writing the corrected pixel value to the second storage address of the second memory;

[0190] When the first buffer performs a read operation, the second buffer performs a write operation; when the first buffer performs a write operation, the second buffer performs a read operation.

[0191] The further functional description of each of the above modules and units is the same as that of the above corresponding embodiments and will not be repeated here.

[0192] The image correction device in this embodiment is presented in the form of a functional unit, where the unit refers to an ASIC (Application Specific Integrated Circuit) circuit, including a processor and memory that executes one or more software or fixed programs, and / or other devices that can provide the above functions.

[0193] The embodiment of the present invention also provides a programmable logic device having the above Figure 6 The image correction device shown.

[0194] See also Figure 7 , Figure 7 is a schematic diagram of the structure of a programmable logic device provided by an optional embodiment of the present invention, such as Figure 7As shown, the programmable logic device includes: one or more processors 10, a memory 20, and interfaces for connecting various components, including high-speed interfaces and low-speed interfaces. The various components are connected to each other using different buses for communication, and can be installed on a common mainboard or installed in other ways as needed. The processor can process instructions executed in the programmable logic device, including instructions stored in or on the memory to display graphical information of the GUI on an external input / output device (such as a display device coupled to the interface). In some optional embodiments, if necessary, multiple processors and / or multiple buses can be used together with multiple memories. Similarly, multiple programmable logic devices can be connected, and each device provides some necessary operations (for example, as a server array, a group of blade servers, or a multi-processor system). Figure 7 A processor 10 is taken as an example.

[0195] The processor 10 may be a central processing unit, a network processor or a combination thereof. The processor 10 may further include a hardware chip.

[0196] The memory 20 stores instructions executable by at least one processor 10, so that the at least one processor 10 executes the method shown in the above embodiment.

[0197] The memory 20 may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required by at least one function; the data storage area may store data created according to the use of the programmable logic device, etc. In addition, the memory 20 may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some optional embodiments, the memory 20 may optionally include a memory remotely arranged relative to the processor 10, and these remote memories may be connected to the programmable logic device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0198] The memory 20 may include a volatile memory, such as a random access memory; the memory may also include a non-volatile memory, such as a flash memory, a hard disk or a solid state drive; the memory 20 may also include a combination of the above types of memory.

[0199] The programmable logic device further includes a communication interface 30 for the programmable logic device to communicate with other devices or a communication network.

[0200] The embodiment of the present invention also provides a computer-readable storage medium. The method according to the embodiment of the present invention can be implemented in hardware, firmware, or can be implemented as a computer code that can be recorded in a storage medium, or can be implemented as a computer code that is originally stored in a remote storage medium or a non-temporary machine-readable storage medium and will be stored in a local storage medium through a network download, so that the method described herein can be stored in such software processing on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. Among them, the storage medium can be a magnetic disk, an optical disk, a read-only storage memory, a random access memory, a flash memory, a hard disk or a solid-state hard disk, etc.; further, the storage medium can also include a combination of the above types of memories. It can be understood that a computer, a processor, a microprocessor controller, or programmable hardware includes a storage component that can store or receive software or computer code. When the software or computer code is accessed and executed by a computer, a processor, or hardware, the method shown in the above embodiment is implemented.

[0201] A part of the present invention may be applied as a computer program product, such as a computer program instruction, which, when executed by a computer, can call or provide the method and / or technical solution according to the present invention through the operation of the computer. Those skilled in the art should understand that the existence of the computer program instruction in a computer-readable medium includes, but is not limited to, a source file, an executable file, an installation package file, etc., and accordingly, the way in which the computer program instruction is executed by the computer includes, but is not limited to: the computer directly executes the instruction, or the computer compiles the instruction and then executes the corresponding compiled program, or the computer reads and executes the instruction, or the computer reads and installs the instruction and then executes the corresponding installed program. Here, the computer-readable medium may be any available computer-readable storage medium or communication medium accessible to the computer.

[0202] Although the embodiments of the present invention have been described in conjunction with the accompanying drawings, those skilled in the art may make various modifications and variations without departing from the spirit and scope of the present invention, and such modifications and variations should all be included in the protection scope of the present invention.

Claims

1. An image correction method, characterized in that: Applied to a programmable logic device, the method comprises: Acquiring original coordinate data of original pixels belonging to different first categories; the first category to which the original pixels belong is determined by classifying the original coordinate data of the original pixels according to a preset classification rule; Determining, according to the original coordinate data of the original pixel, the corrected coordinate data of the corrected pixel corresponding to the dedistorted original pixel, and classifying the corrected coordinate data of the corrected pixel according to the classification rule to determine the second category to which the corrected pixel belongs; According to the first category to which the original pixel belongs and the second category to which the corresponding corrected pixel belongs, the corresponding coordinate data is cached in the corresponding first storage space; wherein a corresponding first storage space group is pre-configured for each first category, and the first storage space group includes the first storage space corresponding to each second category respectively; the coordinate data includes the original coordinate data of the original pixel and the corrected coordinate data of the corrected pixel; Taking the first storage spaces corresponding to the same second category in each of the first storage space groups as a group, respectively acquiring the coordinate data to be processed in each group of the first storage spaces; The correction pixel values ​​corresponding to the correction pixels are determined according to the coordinate data to be processed in each group, and a correction image is generated according to the correction pixel values.

2. The method according to claim 1, characterized in that The original coordinate data includes the first coordinate and the second coordinate of the original pixel; the corrected coordinate data includes the first coordinate and the second coordinate of the corrected pixel; the classification rule is used to classify the first coordinate into N categories, where N is a parallelism not less than 2; The first category to which the original pixel belongs is determined by classifying the first coordinate of the original pixel according to the classification rule; The second category to which the correction pixel belongs is determined by classifying the first coordinates of the correction pixel according to the classification rule.

3. The method according to claim 2, characterized in that The obtaining of original coordinate data of original pixels belonging to different first categories includes: The process of generating the original coordinate data is performed in parallel with a parallelism degree N, and the generating the original coordinate data includes: Presetting and determining the starting coordinates of the first coordinate and the width of the first coordinate, as well as the starting coordinates of the second coordinate and the width of the second coordinate; In a current counting cycle, under the constraints of the width of the first coordinate and the width of the second coordinate, determining a first count of the first coordinate and a second count of the second coordinate corresponding to the current count; the step of the first count is N, and the step of the second count is 1; The first coordinates of the original pixel acquired in the current counting cycle are determined according to the starting coordinates of the first coordinates and the first count; the second coordinates of the original pixel acquired in the current counting cycle are determined according to the starting coordinates of the second coordinates and the second count.

4. The method according to claim 2, characterized in that: The classification rule is: taking N as the divisor, performing remainder processing on the first coordinate, and different remainders correspond to different categories.

5. The method according to claim 1, characterized in that The step of taking the first storage spaces corresponding to the same second category in each of the first storage space groups as a group and respectively acquiring the coordinate data to be processed in each group of the first storage spaces comprises: Taking the first storage spaces corresponding to the same second category in each of the first storage space groups as a group, respectively reading the coordinate data to be processed in each group of the first storage spaces; The read coordinate data to be processed is cached in a corresponding second storage space; wherein a corresponding second storage space is pre-configured for each second category.

6. The method according to claim 5, characterized in that The first storage spaces corresponding to the same second category in each of the first storage space groups are taken as a group, and the coordinate data to be processed in each group of first storage spaces are read respectively, including: Determine whether there is a non-empty first storage space among the first storage spaces corresponding to the same second category in each of the first storage space groups; In the case that there is a non-empty first storage space, the coordinate data in the non-empty first storage space is read, and the coordinate data in the non-empty first storage space is used as the coordinate data to be processed.

7. The method according to any one of claims 1 to 6, characterized in that The method further comprises: Reading original image data corresponding to the original image to be corrected; the original image data includes original pixel values ​​and original coordinate data of each original pixel; According to the first category to which each of the original pixels belongs, the original pixel values ​​of the original pixels are cached in corresponding first memories respectively; wherein a corresponding first memory is pre-configured for each first category; The step of respectively determining the correction pixel value corresponding to each correction pixel according to the coordinate data to be processed of each group includes: Determine a corresponding first storage address according to the corrected coordinate data in the coordinate data to be processed; The original pixel value corresponding to the first storage address is obtained from the first memory corresponding to the coordinate data to be processed, and the original pixel value corresponding to the first storage address is used as the corrected pixel value of the corrected pixel in the coordinate data to be processed.

8. The method according to claim 7, characterized in that Generating a corrected image according to the corrected pixel values ​​comprises: According to the first category to which the original pixel in the coordinate data to be processed belongs and the second category to which the corrected pixel in the coordinate data to be processed belongs, the corrected pixel value is cached in a corresponding third storage space; wherein a corresponding third storage space group is pre-configured for each second category, and the third storage space group includes a third storage space corresponding to each first category respectively; Taking the third storage spaces corresponding to the same first category in each of the third storage space groups as a group, respectively acquiring the corrected pixel values ​​in each group of the third storage spaces; The corresponding second storage address is determined according to the original coordinate data in the coordinate data to be processed, the pixel value corresponding to the second storage address is recorded as the correction pixel value, and a correction image determined based on each of the correction pixel values ​​is generated.

9. The method according to claim 8, characterized in that The third storage spaces corresponding to the same first category in each of the third storage space groups are taken as a group, and the corrected pixel values ​​in each group of the third storage spaces are respectively obtained, including: The third storage spaces corresponding to the same first category in each of the third storage space groups are taken as a group, and the corrected pixel values ​​in each group of the third storage spaces are read respectively; The read corrected pixel value is cached in a corresponding fourth storage space; wherein a corresponding fourth storage space is pre-configured for each first category.

10. The method according to claim 8, characterized in that The step of recording the pixel value corresponding to the second storage address as the corrected pixel value comprises: Alternately reading the second storage address according to the first buffer and the second buffer, and writing the corrected pixel value to the second storage address of the second memory; When the first buffer performs a read operation, the second buffer performs a write operation; when the first buffer performs a write operation, the second buffer performs a read operation.

11. An image correction device, characterized in that: Applied to a programmable logic device, the device comprises: A coordinate acquisition module, used to acquire original coordinate data of original pixels belonging to different first categories; the first category to which the original pixels belong is determined by classifying the original coordinate data of the original pixels according to a preset classification rule; a classification module, configured to determine, based on the original coordinate data of the original pixel, the corrected coordinate data of the corrected pixel corresponding to the original pixel after the original pixel is dedistorted, and classify the corrected coordinate data of the corrected pixel according to the classification rule to determine the second category to which the corrected pixel belongs; A coordinate cache module, configured to cache corresponding coordinate data to a corresponding first storage space according to the first category to which the original pixel belongs and the second category to which the corresponding corrected pixel belongs; wherein a corresponding first storage space group is pre-configured for each first category, and the first storage space group includes first storage spaces corresponding to each second category respectively; the coordinate data includes original coordinate data of the original pixel and corrected coordinate data of the corrected pixel; A grouping acquisition module, used to take the first storage spaces corresponding to the same second category in each of the first storage space groups as a group, and respectively acquire the coordinate data to be processed in each group of the first storage spaces; The processing module is used to determine the correction pixel value corresponding to each correction pixel according to each group of the coordinate data to be processed, and generate a correction image according to the correction pixel value.

Images