A multi-processor parallel processing method and device for panoramic image stitching

Abstract

The present invention provides a multi-processor parallel processing method and apparatus for panoramic image stitching, which uses at least two parallel processors to synchronously process and stitch and then output the panoramic image. The method includes: dividing the panoramic image to be output into N regions, and allocating the image processing tasks of each processor; synchronously sending the data of multiple captured images to each processor; each processor parallelly processes the multiple captured images according to the image processing tasks to obtain output images; according to the order of the N regions of the panoramic image to be output, the processors respectively output the output images to the corresponding regions in the buffer; when all the output images are output to the buffer, the stitching of the panoramic image is completed. The apparatus includes an input image buffer, a processor array, and an output image buffer, and the processor array includes at least two parallel processors. The multi-processor parallel image stitching method designed by the present invention has the advantages of strong operability, scientific rationality, high efficiency and speed.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and particularly to a multi-processor parallel processing method and device for panoramic image stitching. Background Art

[0002] The panoramic image stitching technology uses multiple ordinary cameras to generate a spherical panoramic view of 360 degrees horizontally and 360 degrees vertically, which can bring a broader perspective and richer visual impact to viewers. The panoramic image stitching technology is increasingly applied to various scenarios such as scenic area virtual display and digital exhibition halls.

[0003] Currently, most of the panoramic image stitching technologies are implemented through a single processor. Due to the limited processing power of the single processor, and with the continuous increase in the requirements for the pixel resolution and output frame rate of the output image, the resolution and frame rate of the output image obtained through single-processor processing cannot meet the requirements of users. Summary of the Invention

[0004] With the continuous increase in the requirements for the pixel resolution and output frame rate of the output image, the present invention provides a multi-processor parallel processing method and device for panoramic image stitching. The panoramic image stitching method and device provided by the present invention perform sub-task processing on the input images by using multiple processors. On the basis of reducing the task load of each processor, the panoramic image stitching with higher performance requirements is completed through the cooperation of each processor.

[0005] The technical solutions for achieving the invention purpose are as follows:

[0006] In a first aspect, the present invention provides a multi-processor parallel processing method for panoramic image stitching, which uses N parallel processors to synchronously process multiple captured images and then outputs a panoramic image, where N is an integer greater than or equal to 2.

[0007] The multi-processor parallel processing method for panoramic image stitching includes the following steps:

[0008] S1. Divide the panoramic image to be output into N regions and allocate the image processing tasks of each processor;

[0009] S2. Synchronously send the data of multiple captured images to each processor;

[0010] S3. Each processor processes the multiple captured images respectively according to its image processing task to obtain its own output image;

[0011] S4. According to the order of the N regions of the panoramic image to be output, each processor outputs its output image to the corresponding region in the buffer for caching;

[0012] S5. After the output images of the N processors are all output to the buffer, the stitching of the panoramic image is completed.

[0013] The method for multi-processor parallel processing for panoramic image stitching designed by the present invention can reasonably allocate tasks to multiple parallel processors according to the pixel requirements of panoramic image stitching, reduce the task load of each processor, and improve the image processing speed. Each processor synchronously inputs its output image into the buffer, and the stitching of the panoramic image can be realized. The method designed by the present invention has the advantages of strong operability, scientific rationality, high efficiency and speed.

[0014] In an embodiment of the present invention, in the above step S1, the method of dividing the panoramic image to be output into N regions and allocating the image processing tasks of each processor is as follows: based on the number of pixel rows Rx and the number of pixel columns C of the panoramic image to be output, and according to the processor numbers 1, 2, 3, ……, N−1, N, along the row numbers 0, 1, 2, 3, ……, R−1 of the pixels of the panoramic image to be output, the panoramic image to be output is sequentially divided into 1, 2, 3, ……, N−1, N regions corresponding to the processor numbers in sequence, and each region corresponds to the image processing task of a processor.

[0015] Further, among the above processors, the sizes of the first N−1 processors' regions are the same, and the size of the Nth processor's region is less than or equal to the sizes of the first N−1 regions, that is, the processing amounts of the image processing tasks of the first N−1 processors are the same, and the processing amount of the image processing task of the Nth processor is less than or equal to the processing amounts of the image processing tasks of the first N−1 processors.

[0016] Furthermore, when the processing amounts of the image processing tasks of the N processors are all the same, the calculation formula for the pixel value of the output image of the kth processor in the panoramic image is: P(i, j) = f(S(u, v)), where k is an integer in the range of [1, N], P is the pixel value of the i-th row and j-th column of the output image, i is an integer in the range of [(k−1)xR / N, kxR / N), j is an integer in the range of [0, C), f is the spatial mapping relationship between the output image and the input captured image, and S is the pixel value of the u-th row and v-th column of the input captured image.

[0017] Furthermore, among the N processors, the address range of the output image generated by the kth processor in the buffer, that is, in the panoramic image, is: [(k−1)xRxC / N, kxRxC / N).

[0018] In a second aspect, the present invention provides a multi-processor parallel processing device for panoramic image stitching, including an input image buffer, a processor array, and an output image buffer. The processor array includes at least 2 parallel processors.

[0019] Among them, the input image buffer is connected to each processor in the processor array, and the input image buffer is used to store multiple captured images.

[0020] Among them, the processor is used to receive multiple captured images output by the input image buffer, and perform stitching processing on the multiple captured images based on the assigned image processing tasks to generate the output image of the processor.

[0021] Among them, the processors in the processor array are all connected to the output image buffer, and are used to output their respective output images to the image buffer for storage.

[0022] Furthermore, the method for dividing the image processing tasks of the above N processors is as follows: based on the number of pixel rows Rx pixel columns C of the panoramic image to be output, and the processor numbers 1, 2, 3, ……, N−1, N, along the row numbers 0, 1, 2, ……, R−1 of the pixels of the panoramic image to be output, the panoramic image to be output is sequentially divided into 1, 2, ……, N−1, N regions corresponding to the processor numbers in sequence, and each region corresponds to the image processing task of a processor; after the processor generates the output image, the respective output images are output to the corresponding regions in the image buffer for storage.

[0023] Even further, the above processor array includes N processors, N is an integer greater than or equal to 2, the processing amounts of the image processing tasks of the first N−1 processors are the same, and the processing amount of the image processing task of the Nth processor is less than or equal to the processing amounts of the image processing tasks of the first N−1 processors.

[0024] Even further, when the processing amounts of the image processing tasks of the N processors are the same, the calculation formula for the pixel value of the output image of the kth processor in the panoramic image is: P(i, j) = f(S(u, v)), where k is an integer in [1, N], P is the pixel value of the i-th row and j-th column of the output image, i is an integer in the range of [(k−1)xR / N, kxR / N), j is an integer in the range of [0, C), f is the spatial mapping relationship between the output image and the input captured image, and S is the pixel value of the u-th row and v-th column of the input captured image.

[0025] Even further, among the above N processors, the range of the corresponding region of the output image generated by the kth processor in the buffer, that is, the address range in the panoramic image: [(k−1)xRxC / N, kxRxC / N).

[0026] Compared with the prior art, the beneficial effects of the present invention are as follows: The method of multi-processor parallel processing for panoramic image stitching designed by the present invention can reasonably allocate tasks to multiple parallel processors according to the pixel requirements of panoramic image stitching, reduce the task load of each processor, improve the image processing speed, and synchronously input the output images of each processor into the buffer to achieve panoramic image stitching. The method designed by the present invention has the advantages of strong operability, scientific rationality, high efficiency and speed. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings in the following description are only for the purpose of more clearly illustrating the technical solutions of the embodiments of the present invention or the prior art, and for those of ordinary skill in the art, other drawings can be obtained according to these drawings without creative efforts.

[0028] Figure 1 It is a flowchart of the method of multi-processor parallel processing for panoramic image stitching of the present invention;

[0029] Figure 2 It is a schematic diagram of the device of multi-processor parallel processing for panoramic image stitching of the present invention;

[0030] Among them, 1. Input image buffer; 2. Processor array; 3. Output image buffer. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0031] The present invention will be further described below in conjunction with specific embodiments, and the advantages and features of the present invention will become clearer as the description progresses. However, these embodiments are exemplary only and do not constitute any limitation to the scope of the present invention. Those skilled in the art should understand that the details and forms of the technical solutions of the present invention can be modified or replaced without departing from the spirit and scope of the present invention, but such modifications and replacements all fall within the protection scope of the present invention.

[0032] In the description of this embodiment, it should be understood that the orientation or positional relationship indicated by the terms "center", "longitudinal", "transverse", "upper", "lower", "front", "rear", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", etc. is based on the orientation or positional relationship shown in the accompanying drawings, and is only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be construed as a limitation to the present invention.

[0033] In addition, terms such as "first", "second", "third", etc. are only used for descriptive purposes and should not be construed as indicating or implying relative importance or implicitly specifying the quantity of the indicated technical features. Thus, features defined with "first", "second", etc. may explicitly or implicitly include one or more of such features. In the description of the present invention, unless otherwise specified, the meaning of "a plurality" is two or more.

[0034] Embodiment 1:

[0035] This specific embodiment discloses a method for multi-processor parallel processing for panoramic image stitching. In this embodiment, for panoramic image stitching, N parallel processors synchronously process multiple captured images and then output a panoramic image, where N is an integer greater than or equal to 2.

[0036] As Figure 1 shown, the method for multi-processor parallel processing for panoramic image stitching includes the following steps:

[0037] S1. Divide the panoramic image to be output into N regions and allocate the image processing tasks for each processor.

[0038] In this step, when the panoramic image to be output is divided into N regions, the method for obtaining the image processing tasks for each processor is as follows: Based on the number of pixel rows Rx and the number of pixel columns C of the panoramic image to be output, and according to the processor numbers 1, 2, 3,..., N - 1, N, along the row numbers 0, 1, 2, 3,..., R - 1 of the pixels of the panoramic image to be output, the panoramic image to be output is sequentially divided into 1, 2, 3,..., N - 1, N regions corresponding to the processor numbers in sequence, and each region corresponds to the image processing task of one processor. For example, the region of the image processing task of the processor numbered 1 is the first region; the region of the image processing task of the processor numbered 2 is the second region, and so on. The region of the image processing task of the processor numbered N - 1 is the (N - 1)th region, and the region of the image processing task of the processor numbered N is the Nth region.

[0039] Among them, the processing amounts of the image processing tasks of each of the above-mentioned processors may be the same or different. In this specific embodiment, in order to balance the processing amount of each processor and make each processing area be able to output the processed output image within the specified time as much as possible, this specific embodiment preferably chooses to evenly divide the panoramic image to be output according to the number of pixel rows R to obtain the image processing tasks for each processor.

[0040] However, since the image processing tasks are obtained based on the number of pixel rows R of the panoramic image to be output, there may be a situation where the image processing tasks of each processor are the same, or there may be a situation where the processing amounts of the image processing tasks of different processors are different. To ensure that the processing amounts of the image processing tasks of each processor are as equal as possible, for example, when the number of pixel rows Rx number of pixel columns C of the panoramic image to be output is 1080x1920, and when the number of processors is 2, 3, 4, 5, 6, 9, etc., 1080 / N is an integer, which indicates that the processing amounts of the image processing tasks obtained by each processor are the same; for another example, when the number of processors is 7, 1080 / 7 has a remainder, which indicates that the processing amount of the image processing task of some processors is different from that of others. To ensure that the processing amounts of most processors are the same, in this specific embodiment, among the above-mentioned processors, the area sizes of the first N−1 processors are the same, and the area size of the Nth processor is less than or equal to the area sizes of the first N−1 processors, that is, the processing amounts of the image processing tasks of the first N−1 processors are the same, and the processing amount of the image processing task of the Nth processor is less than or equal to the processing amounts of the image processing tasks of the first N−1 processors.

[0041] It should be noted here that the acquisition method of the image processing task of each processor is not limited to the above-mentioned one method. For example, a method can be selected in which the processing amounts of the image processing tasks of the first few processors are the same, and the processing amounts of the image processing tasks of the last few processors are different to divide the panoramic image to be output.

[0042] Furthermore, when the processing amounts of the image processing tasks of N processors are all the same, the calculation formula for the pixel value of the output image of the kth processor in the panoramic image is: P(i, j) = f(S(u, v)), where k is an integer in the range of [1, N], P is the pixel value of the i-th row and j-th column of the output image, i is an integer in the range of [(k−1)xR / N, kxR / N), j is an integer in the range of [0, C), f is the spatial mapping relationship between the output image and the input captured image, and S is the pixel value of the u-th row and v-th column of the input captured image.

[0043] Furthermore, among the N processors, the range of the corresponding area of the output image generated by the kth processor in the buffer, that is, the address range in the panoramic image: [(k−1)xRxC / N, kxRxC / N).

[0044] S2. Synchronously send the data of multiple captured images to each processor.

[0045] In this step, the multiple captured images obtained are to be synchronously distributed to each processor.

[0046] S3. Each processor processes multiple captured images according to its image processing task, and obtains its respective output image.

[0047] In this step, each processor processes multiple captured images according to the image processing tasks distributed in step 1. The processing method uses existing general methods and will not be specifically described here.

[0048] S4. According to the order of N regions of the panoramic image to be output, each processor outputs its output image to the corresponding region in the buffer for caching.

[0049] In this step, since each processor corresponds to one of the N regions divided by the output panoramic image, each processor has its own corresponding region for the output image in the buffer.

[0050] S5. After all the output images of the N processors are output to the buffer, the stitching of the panoramic image can be completed.

[0051] The method of multi-processor parallel processing for panoramic image stitching designed by the present invention can reasonably allocate tasks to multiple parallel processors according to the pixel requirements of panoramic image stitching, reduce the task load of each processor, and improve the image processing speed. Each processor synchronously inputs its output image into the buffer, and the stitching of the panoramic image can be achieved. The method designed by the present invention has the advantages of strong operability, scientific rationality, high efficiency and speed.

[0052] Embodiment 2:

[0053] This embodiment provides a device corresponding to the multi-processor parallel processing method for panoramic image stitching in Embodiment 1. As Figure 2 shown, the device includes an input image buffer 1, a processor array 2, and an output image buffer 3. The processor array 2 includes at least two parallel processors.

[0054] Among them, as Figure 2 shown, the input image buffer 1 is connected to each processor in the processor array 2, and the input image buffer 1 is used to store multiple captured images.

[0055] Among them, the processor is used to receive multiple captured images output by the input image buffer 1, and perform stitching processing on the multiple captured images based on the allocated image processing tasks to generate the output image of the processor.

[0056] Among them, as Figure 2 shown, the processors in the processor array 2 are all connected to the output image buffer 3, and are used to output their respective output images to the image buffer 3 for storage.

[0057] Further, the method for dividing the image processing tasks of the above N processors is the same as that in Embodiment 1, and details are not described herein again.

[0058] Furthermore, the above processor array includes N processors, where N is an integer ≥ 2, and the processing amount of the image processing tasks of each processor is the same as that in Embodiment 1, and details are not described herein again.

[0059] Furthermore, the calculation formula for the pixel values of the output images of each processor is the same as that in Embodiment 1, and details are not described herein again.

[0060] Furthermore, the output images of each processor are output to the corresponding area range in the image buffer 3, that is, the address range in the panoramic image is the same as that in Embodiment 1.

[0061] The above are only the preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

[0062] In addition, it should be understood that although this specification is described according to embodiments, not every embodiment only contains an independent technical solution. This narrative manner of the specification is only for clarity. Those skilled in the art should regard the specification as a whole, and the technical solutions in each embodiment can also be appropriately combined to form other embodiments that can be understood by those skilled in the art.

Claims

1. A multi-processor parallel processing method for panoramic image stitching, characterized in that: Use N parallel processors to synchronously process multiple captured images and then output a panoramic image, where N is an integer greater than or equal to 2, including the following steps: S1. Divide the panoramic image to be output into N regions and assign the image processing tasks for each processor; S2. Synchronously send the data of multiple captured images to each processor; S3. Each processor processes multiple captured images respectively according to its image processing task to obtain its own output image; S4. According to the order of the N regions of the panoramic image to be output, each processor outputs its output image to the corresponding region in the buffer for caching; S5. After all the output images of the N processors are output to the buffer, the stitching of the panoramic image is completed; In step S1, the method of dividing the panoramic image to be output into N regions and assigning the image processing tasks for each processor is as follows: Based on the number of pixel rows Rx and the number of pixel columns C of the panoramic image to be output, and according to the processor numbers 1, 2, ……, N - 1, N, along the row numbers 0, 1, 2, ……, R - 1 of the pixels of the panoramic image to be output, the panoramic image to be output is successively divided into 1, 2, ……, N - 1, N regions corresponding to the processor numbers in sequence, and each region corresponds to the image processing task of a processor respectively.

2. The multi-processor parallel processing method for panoramic image stitching according to claim 1, wherein: Among each processor, the region sizes of the first N - 1 processors are the same, and the region size of the Nth processor is less than or equal to the region sizes of the first N - 1 processors.

3. The multi-processor parallel processing method for panoramic image stitching according to claim 2, characterized in that: When the processing amounts of the image processing tasks of the N processors are the same, the calculation formula for the pixel value of the output image of the kth processor in the panoramic image is: P(i,j)=f(S(u,v)), where k is an integer in the range of [1, N], P is the pixel value of the i-th row and j-th column of the output image, i is an integer in the range of [(k - 1)xR / N, kxR / N), j is an integer in the range of [0, C), f is the spatial mapping relationship between the output image and the input captured image, and S is the pixel value of the u-th row and v-th column of the input captured image.

4. The multi-processor parallel processing method for panoramic image stitching according to claim 3, characterized in that: Among the N processors, the address range of the output image generated by the kth processor in the buffer, that is, in the panoramic image: [(k - 1)xRxC / N, kxRxC / N).

5. An apparatus for multi-processor parallel processing in panoramic image stitching, characterized in that: It includes an input image buffer, a processor array, and an output image buffer. The processor array includes at least 2 parallel processors; The input image buffer is connected to each processor in the processor array, and the input image buffer is used to store multiple captured images; The processor is used to receive multiple captured images output by the input image buffer and perform stitching processing on the multiple captured images based on the assigned image processing task to generate the output image of the processor; The processors in the processor array are all connected to the output image buffer and are used to output their respective output images to the image buffer for storage; Among them, the method for dividing the image processing tasks of the N processors is as follows: Based on the number of pixel rows Rx pixel columns C of the panoramic image to be output, and the processor numbers 1, 2, ……, N - 1, N, along the row numbers 0, 1, 2, ……, R - 1 of the pixels of the panoramic image to be output, the panoramic image to be output is sequentially divided into 1, 2, ……, N - 1, N regions corresponding to the processor numbers in sequence, and each region corresponds to the image processing task of one of the processors; After the processors generate the output images, they output their respective output images to the corresponding regions in the image buffer for storage.

6. The apparatus for multi-processor parallel processing of panoramic image stitching according to claim 5, characterized in that: The processor array includes N of the processors, where N is an integer greater than or equal to 2. The processing amounts of the image processing tasks of the first N - 1 processors are the same, and the processing amount of the image processing task of the Nth processor is less than or equal to the processing amounts of the image processing tasks of the first N - 1 processors.

7. The apparatus for multi-processor parallel processing of panoramic image stitching according to claim 6, characterized in that: When the processing amounts of the image processing tasks of the N processors are all the same, the calculation formula for the pixel value of the output image of the kth processor in the panoramic image is: P(i, j) = f(S(u, v)), where k is an integer in the range of [1, N], P is the pixel value of the i-th row and j-th column of the output image, i is an integer in the range of [(k - 1)xR / N, kxR / N), j is an integer in the range of [0, C), f is the spatial mapping relationship between the output image and the input acquisition image, and S is the pixel value of the u-th row and v-th column of the input acquisition image.

8. The apparatus for multi-processor parallel processing of panoramic image stitching according to claim 7, wherein: Among the N processors, the range of the corresponding region in the buffer where the kth processor generates the output image, that is, the address range in the panoramic image: [(k - 1)xRxC / N, kxRxC / N).

Images