Parallel processing method and device for image integral image and image acceleration processing method

Abstract

The present invention discloses a parallel processing method, device, image processing acceleration method, terminal device and storage medium of an image integral map. The parallel computing method includes the following steps: S1: dividing the image into several small blocks; S2: calculating the local integral map of each small block; S3: calculating the cumulative sum of the rightmost column of the small block by row; S4: calculating the cumulative sum of the bottom row of the small block by column; S5: calculating the integral map of the entire image. The beneficial effect of the present invention is that the algorithm combines the block parallel and binary parallel methods. First, the input image is divided into several small blocks, and each small block calculates the integral map within the block in parallel; then each small block is regarded as a unit, and the cumulative sum of the rightmost column and the bottom row of each small block in both row and column directions is calculated using a binary parallel method; finally, the integral map of the entire image is calculated by combining the cumulative sum between each small block and the integral map within the small block.

Description

Technical Field

[0001] The present invention relates to the field of image processing computing technology, and in particular to a parallel processing method and device for an image integral graph, an image processing acceleration method, a terminal device and a storage medium. Background Art

[0002] Image integral graph is an important method for accelerating image processing. Each point in the integral graph represents the cumulative sum of all pixel values ​​in its upper left rectangle. Through the integral graph, the sum of the rectangular area of ​​the image can be quickly calculated, thereby accelerating common image processing algorithms such as mean filtering, non-local mean filtering, and Harr feature descriptors. However, the calculation of the image integral graph requires the accumulation of rows and columns, which is highly dependent on the pixel position and difficult to parallelize.

[0003] With the development of artificial intelligence technology, the performance of high-concurrency hardware devices such as GPU and NPU has been further improved. High-performance computing and high-concurrency methods suitable for these hardware devices have received more and more attention. Many scholars and engineers have begun to devote themselves to making full use of the high concurrency of these hardware devices to implement and accelerate traditional algorithms. Summary of the invention

[0004] The purpose of this section is to summarize some aspects of embodiments of the present invention and briefly introduce some preferred embodiments. Some simplifications or omissions may be made in this section and the specification abstract and the invention title of this application to avoid blurring the purpose of this section, the specification abstract and the invention title, and such simplifications or omissions cannot be used to limit the scope of the present invention.

[0005] In view of the above problems or problems existing in the prior art, the present invention is proposed.

[0006] Therefore, the purpose of the present invention is to provide a parallel processing method, apparatus, image processing acceleration method, terminal device and storage medium for an image integral map, which can divide the image into several small blocks, calculate the local integral map of each small block, and calculate the cumulative sum of the rightmost column of the small block and the cumulative sum of the bottom row by row and column respectively, so as to calculate the integral map of the entire image.

[0007] In order to solve the above technical problems, the present invention provides the following technical solutions: a parallel processing method of an image integral map, which comprises the following steps:

[0008] S1: Divide the image into several small blocks;

[0009] S2: Calculate the local integral map of each small block;

[0010] S3: Calculate the cumulative sum of the rightmost column of the small block by row;

[0011] S4: Calculate the cumulative sum of the bottom row of the small block by column;

[0012] S5: Calculate the integral image of the entire image.

[0013] As a preferred solution of the parallel processing method of the image integral map of the present invention, wherein: S2 comprises the following steps;

[0014] S2-1: Calculate the cumulative sum by row within each small block divided in step S1;

[0015] S2-2: Calculate the cumulative sum by column within each small block divided in step S1.

[0016] As a preferred solution of the parallel processing method of the image integral map of the present invention, step S3 includes the following steps:

[0017] S3-1: Take the rightmost column value of the small block of the integral image obtained in step S2 by row, and arrange them in column order into array A. The i-th element in array A is the rightmost column value of the i-th small block in each row of the integral image, i∈[0, f(W / N)); if the length of array A is not equal to 2 n , then the length of array A is expanded to 2 n , the expansion value is filled with 0, where the size of the small block is NxN, representing rows and columns, respectively, f(W / N) columns, W represents the width of the image, f represents the function of rounding up, 2 n Represents 2 to the power of n, where n is a positive integer;

[0018] S3-2: Set n-1 loops. In each loop P1, first for each thread t1, the total number of threads of thread t1 includes 2 n-1- P1 A thread, the second P1 (2t1+ 1)-1 and 2 P1 (2(t1+1))-1 items are summed and the result is stored in the second P1 (2(t1+1))-1, and then synchronize all threads corresponding to the current array A, P1∈[0, n-1), t1∈[0, 2 n-1- P1 );

[0019] S3-3: The last item of array A is saved to the rightmost column of the last small block of the corresponding row of the integral image, and then set to 0;

[0020] S3-4: Set n-1 loops. In each loop P2, first for each thread t2, the total number of threads of thread t2 includes 2 n-1- P2 A thread, the second P2 (2t2+1)-1 term and 2 P2 (2(t2+1))-1 items are swapped, and the two items are summed and the result is stored in the 2ndP2 In the (2(t2+1))-1 item, all threads corresponding to the current array A are synchronized, P2∈[n-2, 0], t2∈[0, 2 n-1- P2 );

[0021] S3-5: In each thread t3, the total number of threads of thread t3 includes 2 n-1 A thread assigns the t3+1th item in array A to the rightmost column of the t3th block in each row of the integral image, t3∈[0, 2 n-1 );

[0022] As a preferred solution of the parallel processing method of the image integral map of the present invention, wherein: step S4 includes the following steps;

[0023] The S4 step includes the following steps:

[0024] S4-1: add the Jth item in the bottom row of each small block to the lower right corner item of the left adjacent small block of the corresponding small block, and save the result to the Jth item in the bottom row of the corresponding small block, J∈[0, N-1);

[0025] S4-2: For the integral image obtained in step S3, the bottom row values ​​of the small blocks are taken out by column and the cumulative sum is calculated, and the result is saved in the bottom row of the corresponding small block in the integral image.

[0026] As a preferred solution of the parallel processing method of the image integral map of the present invention, wherein: step S5 includes the following steps;

[0027] S5-1: for each small block of the integral image obtained in step S4, add the value Vb of the Kth row and jth column to the value Vt of the bottom row of the corresponding column of the upper adjacent small block and the value Vl of the rightmost column of the corresponding row of the left adjacent small block, and save the result to the Kth row and jth column of the corresponding small block in the integral image, K∈[0, N-1), j∈[0, N-1);

[0028] S5-2: For the value Vb' of the Kth row and N-1th column of each small block, add Vb' to the lower right corner value Vt' of the upper adjacent small block, and save the result to the Kth row and N-1th column of the corresponding small block in the integral graph, K∈[0, N-1);

[0029] As a preferred solution of the parallel processing method of the image integral image of the present invention, wherein: in step S1, the size of the small block is NxN, and the small block image is divided into f(H / N) rows and f(W / N) columns, wherein H and W represent the length and width of the image respectively, and f represents a rounding-up function.

[0030] The present invention also provides an image processing acceleration method, including steps of a parallel processing method of an image integral map.

[0031] The present invention also provides a parallel processing device for image integral images, comprising:

[0032] A block division unit, used to divide the image into several small blocks;

[0033] A local integral map generating unit, used for calculating the local integral map of each small block;

[0034] A row accumulation unit is used to calculate the cumulative sum of the rightmost column of the small block by row;

[0035] The column accumulation unit is used to calculate the cumulative sum of the bottom row of the small block by column;

[0036] The overall calculation unit is used to calculate the integral image of the entire image.

[0037] The present invention also provides a terminal device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the computer program is executed by the processor, the steps of the parallel processing method of the image integral map are implemented.

[0038] The present invention also provides a storage medium, including a computer program stored on the storage medium, and the steps of the parallel processing method of the image integral map are implemented when the computer program is executed by a processor.

[0039] Beneficial effects of the present invention: The present invention divides an image into several small blocks, calculates the local integral map of each small block, and calculates the cumulative sum of the rightmost column and the cumulative sum of the bottom row of the small block by row and column respectively, thereby calculating the integral map of the entire image; the present invention combines the block parallel and binary parallel algorithms, uses binary parallel to improve the parallelism of inter-block accumulation during block parallel, and uses block parallel to reduce idle threads in binary parallel, so that many threads can be used for parallel calculation in each step of calculating the image integral map, and fully utilizes the high concurrency of hardware devices such as GPU, thereby maximizing the calculation speed of the image integral map. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for describing the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative labor. Among them:

[0041] Figure 1 It is a schematic diagram of the interface of step S1 of the parallel processing method of the image integral image;

[0042] Figure 2 It is a schematic diagram of the interface of step S2 of the parallel processing method of the image integral map;

[0043] Figure 3 It is a schematic diagram of the interface of step S3 of the parallel processing method of the image integral map;

[0044] Figure 4 It is a schematic diagram of the interface of step S4 of the parallel processing method of the image integral image;

[0045] Figure 5 It is a schematic diagram of the interface of step S5 of the parallel processing method of the image integral image;

[0046] Figure 6 This is an example diagram of the binary parallel interface of the parallel processing method of the image integral graph when the array length is 8;

[0047] Figure 7 Schematic diagram of the flow of the parallel processing method of the image integral map. DETAILED DESCRIPTION

[0048] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are described in detail below in conjunction with the accompanying drawings.

[0049] In the following description, many specific details are set forth to facilitate a full understanding of the present invention, but the present invention may also be implemented in other ways different from those described herein, and those skilled in the art may make similar generalizations without violating the connotation of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0050] Secondly, the term "one embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The term "in one embodiment" that appears in different places in this specification does not necessarily refer to the same embodiment, nor does it refer to a separate or selective embodiment that is mutually exclusive with other embodiments.

[0051] Reference Figure 7 , an embodiment of a parallel processing method for an image integral map is provided, a parallel processing method for an image integral map comprising the following steps,

[0052] S1: Divide the image into several small blocks;

[0053] S2: Calculate the local integral map of each small block;

[0054] S3: Calculate the cumulative sum of the rightmost column of the small block by row;

[0055] S4: Calculate the cumulative sum of the bottom row of the small block by column;

[0056] S5: Calculate the integral image of the entire image.

[0057] refer to Figure 1 In step S1, the image is divided into several small blocks. Specifically, according to the preset small block size NxN, the image is divided into f(H / N) rows and f(W / N) columns. Wherein, H and W represent the length and width of the image respectively, and f represents a function of rounding up. Preferably, in order to achieve the best parallel computing speed, the small block N should not be too large or too small. In the embodiment of the present invention, N is selected from 4, 8, and 16.

[0058] Further, see Figure 2 , S2 includes the following steps;

[0059] S2-1: Calculate the cumulative sum of each row in each small block divided in step S1 (refer to Figure 2 small horizontal arrow in the middle);

[0060] S2-2: Calculate the cumulative sum by column within each small block divided in step S1 (refer to Figure 2 small vertical arrow in the middle).

[0061] Preferably, a local integral image is calculated inside each small block. Specifically, the calculation of the local integral image inside each small block in step S2 includes two steps, firstly, the cumulative sum is calculated by row for each small block divided in step S1, and then the cumulative sum is calculated by column. Preferably, in order to achieve the best parallel computing speed, in an embodiment of the present invention, N threads are allocated to each small block, wherein each thread t, t∈[0, N) first calculates the cumulative sum of the t-th row, and then calculates the cumulative sum of the t-th column, and saves the result in the integral image.

[0062] Reference Figure 3 , S3: Calculate the cumulative sum of the rightmost column of the small block by row; Specifically, calculating the cumulative sum of the rightmost column of the small block by row in step S3 is to accumulate the rightmost column of each small block by row. Preferably, in order to achieve the best parallel computing speed, in the embodiment of the present invention, a binary parallel method is used for calculation,

[0063] Better, binary parallel method process interface example diagram reference Figure 6 ; Among them, the binary parallelization includes the following steps:

[0064] For the integral image obtained in step S2, the rightmost column value of the small block is taken out row by row and arranged in column order into array A. The i-th element in array A is the rightmost column value of the i-th small block in each row of the integral image, i∈[0, f(W / N)); if the length of array A is not equal to 2 n , then expand the length of array A to 2 n , the expansion value is filled with 0, where the size of the small block is NxN, representing rows and columns, respectively, f(W / N) columns, W represents the width of the image, f represents the function of rounding up, 2 nRepresents 2 raised to the power of n, where n is a positive integer.

[0065] Preferably, in order to accelerate parallel computing, the present invention allocates 2^(n-1) threads to each row of array A taken out.

[0066] Furthermore, step S3 further includes the following steps:

[0067] S3-2: Set n-1 loops. In each loop P1, first for each thread t1, the total number of threads of thread t1 includes 2 n-1- P1 A thread, the second P1 (2t1+ 1)-1 and 2 P1 (2(t1+1))-1 items are summed and the result is stored in the second P1 (2(t1+1))-1, and then synchronize all threads corresponding to the current array A, P1∈[0, n-1), t1∈[0, 2 n-1- P1 ). Further, step S3 also includes the following steps:

[0068] S3-3: The last item of array A is saved to the rightmost column of the last small block of the corresponding row of the integral image, and then set to 0.

[0069] Furthermore, step S3 further includes the following steps:

[0070] S3-4: Set n-1 loops. In each loop P2, first for each thread t2, the total number of threads of thread t2 includes 2 n-1- P2 A thread, the second P2 (2t2+1)-1 term and 2 P2 (2(t2+1))-1 items are swapped, and the two items are summed and the result is stored in the 2nd P2 In the (2(t2+1))-1 item, all threads corresponding to the current array A are synchronized, P2∈[n-2, 0], t2∈[0, 2 n-1- P2 ).

[0071] Furthermore, step S3 further includes the following steps:

[0072] S3-5: In each thread t3, the total number of threads of thread t3 includes 2 n-1 A thread assigns the t3+1th item in array A to the rightmost column of the t3th block in each row of the integral image, t3∈[0, 2 n-1 ).

[0073] Further, step S4 includes the following steps:

[0074] Reference Figure 4, S4-1: add the Jth item in the bottom row of each small block to the lower right corner item of the adjacent small block on the left side of the corresponding small block, and save the result to the Jth item in the bottom row of the corresponding small block, J∈[0, N-1);

[0075] S4-2: For the integral image obtained in step S3, the bottom row values ​​of the small blocks are taken out by column and the cumulative sum is calculated, and the result is saved in the bottom row of the corresponding small block in the integral image.

[0076] In one embodiment, the accumulated sum is calculated using a binary parallel method in step S3.

[0077] Further, see Figure 5 , step S5 includes the following steps;

[0078] S5-1: For the value Vb of the Kth row and jth column in each small block of the integral image obtained in step S4, add it to the bottom row value Vt of the corresponding column of the upper adjacent small block and the rightmost column value Vl of the corresponding row of the left adjacent small block, and save the result to the Kth row and jth column of the corresponding small block in the integral image, K∈[0, N-1), j∈[0, N-1).

[0079] Furthermore, step S5 further includes the following steps:

[0080] S5-2: For the value Vb' of the Kth row and N-1th column of each small block, add Vb' to the lower right corner value Vt' of the upper adjacent small block, and save the result to the Kth row and N-1th column of the corresponding small block in the integral graph; the N-1th row of each small block is not processed, K∈[0, N-1).

[0081] It should be noted that when there is no adjacent small block on the upper side or the left side of the small block under consideration, Vt or Vl is set to 0 accordingly before calculation.

[0082] The result obtained through step 5 is the integral image of the original image.

[0083] refer to Figures 1 to 7 The present invention also provides an implementation of an image acceleration method, including steps of a parallel processing method for an image integral map, by which the optimal parallel computing speed can be achieved.

[0084] refer to Figure 1~2 , the present invention also provides an implementation of a parallel processing device for an image integral map, the parallel processing device for an image integral map comprises a block division unit for dividing an image into a plurality of small blocks;

[0085] A local integral map generating unit, used for calculating the local integral map of each small block;

[0086] A row accumulation unit is used to calculate the cumulative sum of the rightmost column of the small block by row;

[0087] The column accumulation unit is used to calculate the cumulative sum of the bottom row of the small block by column;

[0088] The overall calculation unit is used to calculate the integral image of the entire image.

[0089] The present invention also provides an embodiment of a terminal device, which includes a processor and a computer program stored in a memory and executable on the processor, and the steps of the parallel processing method of the image integral map are implemented when the computer program is executed by the processor.

[0090] The present invention also provides an embodiment of a storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps of the parallel processing method of the image integral map are implemented.

[0091] In summary, the present invention divides the image into several small blocks, calculates the local integral map of each small block, and calculates the cumulative sum of the rightmost column of the small block and the cumulative sum of the bottom row by row and column respectively, so as to calculate the integral map of the entire image; the present invention combines the block parallel and binary parallel algorithms, and uses binary parallel to improve the parallelism of inter-block accumulation during block parallelization, and uses block parallelism to reduce idle threads in binary parallelization, so that many threads can be used for parallel calculation in each step of calculating the image integral map, and fully utilizes the high concurrency of hardware devices such as GPU, thereby maximizing the calculation speed of the image integral map.

[0092] Importantly, it should be noted that the construction and arrangement of the present application shown in a number of different exemplary embodiments are only exemplary. Although only a few embodiments are described in detail in this disclosure, it should be readily understood by those who refer to this disclosure that many modifications are possible (e.g., the size, scale, structure, shape and proportion of various elements, and parameter values ​​(e.g., temperature, pressure, etc.), mounting arrangements, use of materials, color, changes in orientation, etc.) without substantially departing from the novel teachings and advantages of the subject matter described in the application. For example, an element shown as integrally formed may be composed of multiple parts or elements, the position of the element may be inverted or otherwise changed, and the nature or number or position of the discrete element may be altered or changed. Therefore, all such modifications are intended to be included within the scope of the present invention. The order or sequence of any process or method steps may be changed or reordered according to an alternative embodiment. In the claims, any "device plus function" clause is intended to cover the structure described herein that performs the function, and is not only structurally equivalent but also equivalent structure. Other substitutions, modifications, changes and omissions may be made in the design, operating conditions and arrangement of the exemplary embodiments without departing from the scope of the present invention. Therefore, the invention is not limited to a specific embodiment, but extends to numerous modifications still falling within the scope of the appended claims.

[0093] Additionally, in order to provide a concise description of exemplary embodiments, all features of an actual embodiment may not be described (ie, those features that are not relevant to the best mode presently contemplated for carrying out the invention or those features that are not relevant to implementing the invention).

[0094] It will be appreciated that in the development of any actual implementation, as in any engineering or design project, numerous implementation-specific decisions may be made. Such a development effort may be complex and time-consuming, but will be a routine task of design, fabrication, and production for those of ordinary skill having the benefit of this disclosure without undue experimentation.

[0095] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention, which should all be included in the scope of the claims of the present invention.

Claims

1. A parallel processing method for image integral images, characterized in that: The following steps are included: S1: Divide the image into several small blocks; S2: Calculate the local integral map of each small block; S3: Calculate the cumulative sum of the rightmost column of the small block by row; S4: Calculate the cumulative sum of the bottom row of the small block by column; S5: Calculate the integral image of the entire image; The S2 comprises the following steps: S2-1: Calculate the cumulative sum by row within each small block divided in step S1; S2-2: Calculate the cumulative sum by column within each small block divided in step S1; The step S3 comprises the following steps: S3-1: Take the rightmost column value of the small block of the integral image obtained in step S2 by row, and arrange them in column order into array A. The i-th element in array A is the rightmost column value of the i-th small block in each row of the integral image, i∈[0, f(W / N)); if the length of array A is not equal to 2 n , then the length of array A is expanded to 2 n , the expansion value is filled with 0, where the size of the small block is NxN, representing rows and columns, respectively, f(W / N) columns, W represents the width of the image, f represents the function of rounding up, 2 n Represents 2 to the power of n, where n is a positive integer; S3-2: Set n-1 loops. In each loop P1, first for each thread t1, the total number of threads of thread t1 includes 2 n -1- P1 A thread, the second P1 (2t1+ 1)-1 and 2 P1 (2(t1+1))-1 items are summed and the result is stored in the second P1 (2(t1+1))-1, then synchronize all threads corresponding to the current array A, P 1∈[0, n-1), t1∈[0, 2 n-1- P1 ); S3-3: The last item of array A is saved to the rightmost column of the last small block of the corresponding row of the integral image, and then set to 0; S3-4: Set n-1 loops. In each loop P2, first for each thread t2, the total number of threads of thread t2 includes 2 n -1- P2 A thread, the second P2 (2t2+1)-1 term and 2 P2 (2(t2+1))-1 items are swapped, and the two items are summed and the result is stored in the 2nd P2 In the (2(t2+1))-1 item, all threads corresponding to the current array A are synchronized, P2∈[n-2, 0], t2∈[0, 2 n-1- P2 ); S3-5: In each thread t3, the total number of threads of thread t3 includes 2 n-1 A thread assigns the t3+1th item in array A to the rightmost column of the t3th block in each row of the integral image, t3∈[0, 2 n-1 ); The S4 step includes the following steps: S4-1: add the Jth item in the bottom row of each small block to the lower right corner item of the left adjacent small block of the corresponding small block, and save the result to the Jth item in the bottom row of the corresponding small block, J∈[0, N-1); S4-2: For the integral image obtained in step S3, the bottom row values ​​of the small blocks are taken out by column and the cumulative sum is calculated, and the result is saved to the bottom row of the corresponding small block in the integral image; The step S5 comprises the following steps: S5-1: for each small block of the integral image obtained in step S4, add the value Vb of the Kth row and jth column to the value Vt of the bottom row of the corresponding column of the upper adjacent small block and the value Vl of the rightmost column of the corresponding row of the left adjacent small block, and save the result to the Kth row and jth column of the corresponding small block in the integral image, K∈[0, N-1), j∈[0, N-1); S5-2: For the value Vb' of the Kth row and N-1th column of each small block, add Vb' to the lower right corner value Vt' of the upper adjacent small block, and save the result to the Kth row and N-1th column of the corresponding small block in the integral graph, K∈[0, N-1).

2. The parallel processing method of image integral images according to claim 1, characterized in that: In the step S1, the size of the small block is NxN, and the small block image is divided into f(H / N) rows and f(W / N) columns, where H and W represent the length and width of the image respectively, and f represents a rounding function.

3. A terminal device, characterized in that: The terminal device includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the computer program is executed by the processor, the steps of the parallel processing method of the image integral map as described in any one of claims 1 to 2 are implemented.

4. A storage medium, characterized in that: The storage medium stores a computer program, which, when executed by a processor, implements the steps of the parallel processing method for image integral images according to any one of claims 1 to 2.

Images