Image splicing circuit and method and electronic equipment
By introducing image distortion correction module and stitching module into the image stitching circuit, the problem of low image stitching efficiency caused by CPU dependence is solved, more efficient image stitching is achieved and hardware cost and power consumption is reduced.
Patent Information
- Application Number
- CN202510434780.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-04-08
- Publication Date
- 2025-08-12
AI Technical Summary
In the prior art, image stitching depends on CPU implementation, resulting in low image stitching efficiency and increasing hardware cost and power consumption.
The image distortion correction module and the image stitching module are used to connect the bus to reduce the power consumption and cost of the CPU and improve the image stitching efficiency.
Through the combination of the image distortion correction module and the image stitching module, the power consumption and cost of the CPU are reduced and the efficiency of image stitching is improved.
Smart Images
Figure CN120469790A_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to the field of computer vision technology, and in particular to an image stitching circuit, method, and electronic device. Background Art
[0002] Image stitching is a key research area in computer vision. It aims to stitch multiple overlapping images into a seamless, high-resolution panoramic image, capturing more image information within a single frame. Security equipment is ubiquitous in our daily lives, including transportation, communities, airports, and other public spaces. Panoramic image stitching is also widely used in aerospace, military, and medical fields. With technological advancements, the demand for panoramic images in robotics and automotive applications is also increasing.
[0003] Security monitoring is increasingly demanding real-time performance. Due to the limitations of static images in surveillance, there's a need to rapidly stitch together each frame and display it in real-time as a video without increasing hardware costs to meet daily production needs. Previous image stitching solutions rely on the central processing unit (CPU), resulting in low image stitching efficiency. Summary of the Invention
[0004] The present disclosure provides an image stitching circuit, method, and electronic device to solve the problem of low image stitching efficiency caused by reliance on CPU implementation in related technologies. By stitching the original images together through the image distortion correction module and image stitching module in the image stitching circuit, CPU power consumption can be reduced and image stitching efficiency can be improved.
[0005] A first embodiment of the present disclosure provides an image splicing circuit, the circuit comprising:
[0006] Image distortion correction module, image stitching module and bus;
[0007] The image distortion correction module is connected to the first data input module and the first data output module. The first data input module is used to obtain original images and transmit the original images to the image distortion correction module. The original images are at least two images including the same area. The image distortion correction module is used to perform distortion correction processing on the original images to obtain corrected images. The first data output module is used to obtain the corrected images and transmit the corrected images to the bus.
[0008] The image stitching module is connected to the second data input module and the second data output module. The second data input module is used to obtain the corrected image from the bus. The image stitching module is used to stitch the corrected image to obtain a stitched image. The second data output module is used to output the stitched image.
[0009] In some embodiments of the present disclosure, the image distortion correction module includes: an interpolation coordinate calculation module, an image interpolation calculation module, and a color correction calculation module;
[0010] The interpolation coordinate calculation module is connected to the first data input module, the interpolation coordinate calculation module is connected to the image interpolation calculation module, the image interpolation calculation module is connected to the color correction calculation module, and the color correction calculation module is connected to the first data output module;
[0011] The interpolation coordinate calculation module is used to scale the parameters of the image acquisition device to obtain the scaled parameters. The image interpolation calculation module is used to interpolate the original image based on the scaled parameters. The color correction calculation module is used to perform color correction on the interpolated image to obtain a corrected image.
[0012] In some embodiments of the present disclosure, the interpolation coordinate calculation module includes:
[0013] Coordinate calculation module and coordinate cache module;
[0014] The coordinate calculation module is connected to the first data input module and the coordinate cache module. The coordinate calculation module is used to obtain the interpolation coordinates of the original image. The coordinate calculation module includes a horizontal coordinate calculation module and a vertical coordinate calculation module. The coordinate cache module is used to cache the interpolation coordinates of the original image.
[0015] In some embodiments of the present disclosure, the image interpolation calculation module includes:
[0016] Ping-pong cache module and linear interpolation calculation module;
[0017] The interpolation coordinate calculation module is connected to the ping-pong cache module, which is connected to the linear interpolation calculation module. The ping-pong cache module is used to cache the original image, and the linear interpolation calculation module is used to perform interpolation calculation on the original image based on the interpolation coordinates of the original image.
[0018] In some embodiments of the present disclosure, the image stitching module includes:
[0019] Hierarchical calculation module and reconstruction calculation module;
[0020] The layered calculation module is connected to the second data input module and the reconstruction calculation module, and the reconstruction calculation module is connected to the second data output module. The layered calculation module is used to perform layered processing on the images of the same area in the corrected image to obtain layered images, and the reconstruction calculation module is used to reconstruct the layered images to obtain a spliced image.
[0021] In some embodiments of the present disclosure, the image stitching circuit further includes:
[0022] Exposure compensation module;
[0023] The exposure compensation module is connected to the second data input module and the layer calculation module, and is used to compensate the image in the same area in the corrected image.
[0024] In some embodiments of the present disclosure, the image stitching circuit further includes:
[0025] Hierarchical image caching module;
[0026] The layered image cache module is connected with the layered calculation module and the reconstruction calculation module, and is used for caching layered images.
[0027] In some embodiments of the present disclosure, the exposure compensation module includes:
[0028] Image block gain calculation module, Gaussian filter calculation module, gain interpolation calculation module and image compensation calculation module;
[0029] The image block gain calculation module is connected to the Gaussian filter calculation module, the Gaussian filter calculation module is connected to the gain interpolation calculation module, and the gain interpolation calculation module is connected to the image compensation calculation module. The image block gain calculation module is used to segment the image of the same area in the corrected image and calculate the gain value of each image block in the segmented image block. The Gaussian filter calculation module is used to perform Gaussian filtering on the gain value of each image block. The gain interpolation calculation module is used to calculate the gain value corresponding to each pixel in the image block. The image compensation calculation module is used to multiply each pixel in the image block with the gain value corresponding to each pixel to achieve exposure compensation for the image of the same area in the corrected image.
[0030] In some embodiments of the present disclosure, the hierarchical computing module includes:
[0031] Original image cache module, Gaussian pyramid calculation module, first fusion calculation module and first data selection module;
[0032] The original image cache module is connected to the first fusion calculation module, the Gaussian pyramid calculation module is connected to the first fusion calculation module, and the first fusion calculation module is connected to the first data selection module. The original image cache module is used to cache the original image after exposure compensation. The Gaussian pyramid calculation module is used to perform Gaussian processing on the original image after exposure compensation to obtain a Gaussian image. The first fusion calculation module is used to subtract the original image after exposure compensation from the Gaussian image to obtain a layered image. The first data selection module is used to determine the number of Gaussian pyramid levels of the Gaussian pyramid calculation module and select the layered image according to the number of Gaussian pyramid levels.
[0033] In some embodiments of the present disclosure, the reconstruction calculation module includes:
[0034] A data cache module, an upsampling module, a second fusion calculation module and a second data selection module;
[0035] The data cache module is connected to the upsampling module, the upsampling module is connected to the second fusion calculation module, the second fusion calculation module is connected to the second data selection module, the data cache module is used to cache the layered image, the upsampling module is used to upsample the layered image, the second fusion calculation module is used to fuse the upsampled image with the layered image to obtain a spliced image, and the second data selection module is used to select the spliced image according to the number of Gaussian pyramid levels of the Gaussian pyramid calculation module.
[0036] In some embodiments of the present disclosure, the image stitching circuit further includes:
[0037] Configuration module, the configuration module is used to configure parameters and instructions in the image stitching circuit.
[0038] A second embodiment of the present disclosure provides an image stitching method, the method comprising:
[0039] Acquire original images, where the original images are at least two images including the same area;
[0040] Perform distortion correction on the original image to obtain a corrected image;
[0041] The rectified images are stitched to obtain a stitched image.
[0042] A third aspect of the present disclosure provides an electronic device, including: an image stitching circuit as described in the first aspect of the present disclosure, or an image stitching method as described in the second aspect of the present disclosure.
[0043] In summary, the image stitching circuit proposed in the present disclosure includes an image distortion correction module, an image stitching module and a bus; the image distortion correction module is connected to a first data input module and a first data output module, the first data input module is used to obtain the original image and transmit the original image to the image distortion correction module, the original image is at least two images including the same area, the image distortion correction module is used to perform distortion correction processing on the original image to obtain a corrected image, the first data output module is used to obtain the corrected image and transmit the corrected image to the bus; the image stitching module is connected to a second data input module and a second data output module, the second data input module is used to obtain the corrected image from the bus, the image stitching module is used to stitch the corrected image to obtain a stitched image, and the second data output module is used to output the stitched image. The image stitching circuit determined by the image distortion correction module, the image stitching module and the bus can reduce the power consumption and cost of the CPU and improve the efficiency of image stitching.
[0044] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure. BRIEF DESCRIPTION OF THE DRAWINGS
[0045] The accompanying drawings herein are incorporated into and constitute a part of the specification, illustrate embodiments consistent with the present disclosure, and together with the description are used to explain the principles of the present disclosure, and do not constitute an improper limitation of the present disclosure.
[0046] Figure 1 A schematic structural diagram of an image splicing circuit provided in an embodiment of the present disclosure;
[0047] Figure 2 A schematic structural diagram of a specific image splicing circuit provided in an embodiment of the present disclosure;
[0048] Figure 3 A schematic structural diagram of a specific image splicing circuit provided in an embodiment of the present disclosure;
[0049] Figure 4 A schematic structural diagram of a specific image splicing circuit provided in an embodiment of the present disclosure;
[0050] Figure 5 A schematic structural diagram of an image splicing circuit provided in an embodiment of the present disclosure;
[0051] Figure 6 A schematic structural diagram of an image splicing circuit provided in an embodiment of the present disclosure;
[0052] Figure 7 A schematic structural diagram of an image splicing circuit provided in an embodiment of the present disclosure;
[0053] Figure 8 A schematic structural diagram of an image splicing circuit provided in an embodiment of the present disclosure;
[0054] Figure 9 A schematic structural diagram of an exposure compensation module provided in an embodiment of the present disclosure;
[0055] Figure 10 A schematic diagram of the structure of a hierarchical computing module provided in an embodiment of the present disclosure;
[0056] Figure 11 A schematic diagram of the structure of a hierarchical computing module processing process provided by an embodiment of the present disclosure;
[0057] Figure 12 A schematic structural diagram of a reconstruction calculation module provided in an embodiment of the present disclosure;
[0058] Figure 13 A schematic diagram of a structure for obtaining a stitched image provided by an embodiment of the present disclosure;
[0059] Figure 14 A schematic structural diagram of an image splicing circuit provided in an embodiment of the present disclosure;
[0060] Figure 15 A flowchart of an image stitching method provided in an embodiment of the present disclosure;
[0061] Figure 16 A schematic structural diagram of an image stitching device provided in an embodiment of the present disclosure. DETAILED DESCRIPTION
[0062] The embodiments of the present disclosure are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions.
[0063] Image stitching is a key research area in computer vision. It aims to stitch multiple overlapping images into a seamless, high-resolution panoramic image, capturing more image information within a single frame. Security equipment is ubiquitous in our daily lives, including transportation, communities, airports, and other public spaces. Panoramic image stitching also has widespread applications in aerospace, military, and healthcare. With technological advancements, the demand for panoramic images in robotics and automotive applications is also growing.
[0064] Security monitoring is increasingly demanding real-time performance. Due to the limitations of static images in surveillance, there's a need to rapidly stitch together each frame and display it in real-time as a video without incurring additional hardware costs to meet daily production needs. Previous image stitching solutions rely on the CPU for implementation, resulting in low image stitching efficiency.
[0065] The following is a brief introduction to the image stitching solution in related technologies:
[0066] Field-Programmable Gate Array (FPGA), Digital Signal Processor (DSP), and Advanced RISCMachine (ARM) are used to perform edge computing on the end side to complete the splicing processing. Among them, FPGA / DSP will be placed together with the image sensor, with a short data transmission link, high chip computing performance, and high real-time performance. FPGA is used as a semi-custom circuit chip with relatively high chip flexibility and computing performance. ARM is used as the edge CPU.
[0067] The above solution has the following defects:
[0068] First, FPGA / DSP / ARM is used to perform edge computing on the terminal side to complete the splicing processing. Independent computing chips are used and need to be integrated into a camera device, which increases the volume, material costs, and power consumption.
[0069] Secondly, FPGA and DSP are expensive and consume a lot of power.
[0070] Again, ARM's computing performance is low and cannot meet the requirements of high-performance and low-latency scenarios.
[0071] In order to solve the problems existing in the related art, the present disclosure proposes an image stitching circuit, which integrates an image distortion correction module, an image stitching module and a bus; the image distortion correction module is connected to a first data input module and a first data output module, the first data input module is used to obtain an original image and transmit the original image to the image distortion correction module, the original image is at least two images including the same area, the image distortion correction module is used to perform distortion correction processing on the original image to obtain a corrected image, the first data output module is used to obtain the corrected image and transmit the corrected image to the bus; the image stitching module is connected to a second data input module and a second data output module, the second data input module is used to obtain a corrected image from the bus, the image stitching module is used to stitch the corrected image to obtain a stitched image, and the second data output module is used to output the stitched image. The image stitching circuit determined by the image distortion correction module, the image stitching module and the bus can reduce the power consumption and cost of the CPU and improve the efficiency of image stitching.
[0072] The image stitching circuit proposed in the present disclosure belongs to the field of computer vision technology and can be used for image stitching of security cameras, home cameras, and visual robots (including industrial and home use). It can also be expanded to be applied to the image stitching fields of various other industries, such as medical imaging and autonomous driving. Specifically, it can be used in application-specific integrated circuit (ASIC) chips. Since ASIC chips have a high degree of integration, less volume, fewer materials and electronic components are used in the manufacture of camera equipment, which can save costs and reduce energy consumption.
[0073] The image splicing circuit, method and electronic device provided in this application are described in detail below with reference to the accompanying drawings.
[0074] Figure 1 A schematic diagram of the structure of an image splicing circuit provided in an embodiment of the present disclosure.
[0075] like Figure 1 As shown, the image splicing circuit includes:
[0076] Image distortion correction module 1, image stitching module 2 and bus 3;
[0077] The image distortion correction module 1 is connected to the first data input module and the first data output module. The first data input module is used to obtain the original image and transmit the original image to the image distortion correction module 1. The original image is at least two images including the same area. The image distortion correction module 1 is used to perform distortion correction processing on the original image to obtain a corrected image. The first data output module is used to obtain the corrected image and transmit the corrected image to the bus 3.
[0078] The image stitching module 2 is connected to the second data input module and the second data output module. The second data input module is used to obtain the corrected image from the bus 3. The image stitching module 2 is used to stitch the corrected image to obtain a stitched image. The second data output module is used to output the stitched image.
[0079] In an embodiment of the present disclosure, the original image can be obtained through an image acquisition device. Specifically, the image acquisition device can be a monocular camera, and multiple monocular cameras can be configured to capture multiple original images of the same scene at different angles. The image acquisition device can also be a binocular or multi-camera, and the binocular or multi-camera can simultaneously capture original images of the same scene at different angles.
[0080] In an embodiment of the present disclosure, the original image may be obtained from a saved image file.
[0081] In an embodiment of the present disclosure, the original image is at least two images including the same area, including at least two images including the same left and right area, and the original image is at least two images including the same top and bottom area.
[0082] In an embodiment of the present disclosure, the first data input module may obtain the original image from a storage device, such as a double data rate (DDR) storage device.
[0083] In the embodiment of the present disclosure, taking the storage device as DDR as an example, the second data input module can obtain the corrected image from the DDR through bus 3.
[0084] In an embodiment of the present disclosure, the second data output module may transmit the stitched image to a storage device via bus 3 .
[0085] In an embodiment of the present disclosure, the first data input module and the second data input module can be read data Direct Memory Access (DMA) controllers, and the first data output module and the second data output module can be write data DMA controllers. By adopting the DMA controller, data can be directly transmitted between the peripheral device and the memory without the intervention of the CPU.
[0086] In the embodiments of the present disclosure, the image distortion correction module, the image stitching module, and the image stitching circuit determined by the bus can reduce the power consumption and cost of the CPU and improve the efficiency of image stitching.
[0087] Further, such as Figure 2 As shown, the present disclosure provides a structural schematic diagram of a specific image splicing circuit.
[0088] In some embodiments of the present disclosure, referring to Figure 2 , the image distortion correction module 1 in the present disclosure includes:
[0089] Interpolation coordinate calculation module 11, image interpolation calculation module 12 and color correction calculation module 13;
[0090] The interpolation coordinate calculation module 11 is connected to the first data input module, the interpolation coordinate calculation module 11 is connected to the image interpolation calculation module 12, the image interpolation calculation module 12 is connected to the color correction calculation module 13, and the color correction calculation module 13 is connected to the first data output module;
[0091] The interpolation coordinate calculation module 11 is used to scale the parameters of the image acquisition device to obtain the scaled parameters. The image interpolation calculation module 12 is used to interpolate the original image based on the scaled parameters. The color correction calculation module 13 is used to perform color correction on the interpolated image to obtain a corrected image.
[0092] In the embodiment of the present disclosure, the interpolation coordinate calculation module 11 can obtain the coordinates of the image after distortion correction and scaling, so that the subsequent image interpolation calculation can correctly generate the output image.
[0093] In the embodiment of the present disclosure, a high-quality dedistorted and scaled output image can be generated through the image interpolation calculation module 12 .
[0094] In the embodiment of the present disclosure, the color correction calculation module 13 performs color correction on the interpolated original image, which can ensure that the stitched image has uniform color and brightness, thereby improving the overall quality and visual effect of the stitched panoramic image.
[0095] In some embodiments of the present disclosure, the color correction calculation module 13 will only calculate the original image if color correction is turned on in advance. Specifically, taking the original image as an image with the same left and right areas as an example (left and right images), first, histogram statistics are performed on the input left and right images respectively. After the statistics are completed, the left and right groups of results are normalized to obtain the true correction parameters. After the normalization calculation is completed, the image can be output and image correction can be performed. During this process, the calculation of the image interpolation calculation module 12 will be paused.
[0096] In some embodiments of the present disclosure, the calculations of the three modules, namely, the interpolation coordinate calculation module 11 , the image interpolation calculation module 12 and the color correction calculation module 13 , are all performed in a pipeline manner, and cooperate with each other to save a lot of hardware resources.
[0097] In the embodiments of the present disclosure, Figure 3 As shown, the interpolation coordinate calculation module 11 and the image interpolation calculation module 12 are calculated simultaneously in a pipeline manner, that is, the image interpolation calculation module 12 can be connected to the third data input module, wherein the third data input module can also be connected to a read data DMA controller.
[0098] In some embodiments of the present disclosure, Figure 4 As shown, the present disclosure provides a structural diagram of a specific image stitching circuit, wherein the interpolation coordinate calculation module 11 includes:
[0099] Coordinate calculation module 111 and coordinate cache module 112;
[0100] The coordinate calculation module 111 is connected to the first data input module and the coordinate cache module 112. The coordinate calculation module 111 is used to obtain the interpolation coordinates of the original image. The coordinate calculation module 111 includes a horizontal coordinate calculation module and a vertical coordinate calculation module. The coordinate cache module 112 is used to cache the interpolation coordinates of the original image.
[0101] In an embodiment of the present disclosure, coordinate parameters are read from a storage device, and the coordinate calculation module 111 is divided into two coordinate calculators for horizontal and vertical directions. A multiplier is used to calculate the mapping coordinates of each point, that is, the dedistorted and scaled mapping coordinates can be obtained.
[0102] In the embodiment of the present disclosure, the interpolation coordinate calculation module 11 and the image interpolation calculation module 12 are calculated simultaneously in a pipeline manner. The input and output of these two modules are both buses. In order to avoid excessive data coupling, the bus delay will have a greater performance risk. The coordinate cache module 112 can be used as an intermediate buffer. Specifically, if the mapping coordinates are read faster and the original image reading is delayed, the coordinate cache module 112 can cache the read mapping coordinates to reduce the loss of mapping coordinates.
[0103] In some embodiments of the present disclosure, Figure 5 As shown, the present disclosure provides a structural diagram of a specific image splicing circuit, wherein the image interpolation calculation module 12 includes:
[0104] Ping-pong buffer module 121 and linear interpolation calculation module 122;
[0105] The interpolation coordinate calculation module 11 is connected to the ping-pong buffer module 121, and the ping-pong buffer module 121 is connected to the linear interpolation calculation module 122. The ping-pong buffer module 121 is used to cache the original image, and the linear interpolation calculation module 122 is used to perform interpolation calculation on the original image based on the interpolation coordinates of the original image.
[0106] In an embodiment of the present disclosure, the original image is read from the storage device and written into the ping-pong cache module 121. The ping-pong cache module 121 uses two random access memories (RAM) to implement ping-pong cache. For example, the ping-pong cache module 121 includes ping-pong cache 1 and ping-pong cache 2, which can greatly improve the performance of the subsequent linear interpolation calculation module 122. Specifically, ping-pong cache 1 can be used for reading the current frame, and ping-pong cache 2 can be used for writing the next frame. In this way, efficient data processing can be performed without interrupting the data flow, thereby improving the speed of image interpolation calculation, and thus accelerating the efficiency of image stitching.
[0107] In an embodiment of the present disclosure, a finite state machine may be used to control the precise reading and writing switching of the ping-pong cache module 121 , and the finite state machine is located in the interpolation coordinate calculation module 11 .
[0108] In an embodiment of the present disclosure, the integer coordinates read from the interpolation coordinate calculation module 11 are used to read the original image in the ping-pong cache module 121, and the decimal part is used as the weight of the interpolation calculation to perform interpolation calculation on the original image in the linear interpolation calculation module 122, and finally the dedistorted and scaled image is output.
[0109] In some embodiments of the present disclosure, Figure 6 As shown, the present disclosure provides a structural diagram of a specific image splicing circuit, and the image splicing module 2 includes:
[0110] a layered calculation module 21 and a reconstruction calculation module 22;
[0111] The layered calculation module 21 is connected to the second data input module and the reconstruction calculation module 22, and the reconstruction calculation module 22 is connected to the second data output module. The layered calculation module 21 is used to perform layered processing on the images of the same area in the corrected image to obtain layered images, and the reconstruction calculation module 22 is used to reconstruct the layered images to obtain a spliced image.
[0112] In an embodiment of the present disclosure, the layered calculation module 21 may be a Laplacian pyramid calculation controller. Specifically, the rectified image may be subjected to Laplacian pyramid calculation to obtain layered images of different resolutions.
[0113] In the embodiment of the present disclosure, a high-resolution stitched image can be obtained through the layered calculation module 21 and the reconstruction calculation module 22.
[0114] In some embodiments of the present disclosure, Figure 7 As shown, the present disclosure provides a structural schematic diagram of a specific image splicing circuit, and the image splicing circuit further includes:
[0115] Exposure compensation module 4;
[0116] The exposure compensation module 4 is connected to the second data input module and the layer calculation module 21. The exposure compensation module 4 is used to compensate the image in the same area in the corrected image.
[0117] In an embodiment of the present disclosure, the exposure compensation module 4 can compensate for images requiring Laplacian pyramid calculation, so as to make the transition of the stitched image more natural.
[0118] In the embodiment of the present disclosure, the exposure compensation module 4 will calculate the input image of the same region of the original image or the input image of the same region of the corrected image only when the exposure compensation is turned on.
[0119] In some embodiments of the present disclosure, Figure 8 As shown, the present disclosure provides a structural schematic diagram of a specific image splicing circuit, and the image splicing circuit further includes:
[0120] Layered image cache module 5;
[0121] The layered image cache module 5 is connected to the layered calculation module 21 and the reconstruction calculation module 22 , and is used to cache layered images.
[0122] In an embodiment of the present disclosure, the layered image cache module 5 may transmit the cached layered image to the bus through the fourth data input module and the third data output module, and then transmit the cached layered image to the storage device via the bus.
[0123] In an embodiment of the present disclosure, the layered image cache module 5 can cache the Laplacian image of each layer calculated by the Laplacian pyramid. When the stored layered images exceed the cache capacity, the layered images can be transmitted to the bus through the third data output module.
[0124] In some embodiments of the present disclosure, Figure 9As shown, the present disclosure provides a structural diagram of an exposure compensation module 4, which includes:
[0125] Image block gain calculation module 41, Gaussian filter calculation module 42, gain interpolation calculation module 43 and image compensation calculation module 44;
[0126] The image block gain calculation module 41 is connected to the Gaussian filter calculation module 42, the Gaussian filter calculation module 42 is connected to the gain interpolation calculation module 43, and the gain interpolation calculation module 43 is connected to the image compensation calculation module 44. The image block gain calculation module 41 is used to segment the image of the same area in the corrected image and calculate the gain value of each image block in the segmented image block. The Gaussian filter calculation module 42 is used to perform Gaussian filtering on the gain value of each image block. The gain interpolation calculation module 43 is used to calculate the gain value corresponding to each pixel in the image block. The image compensation calculation module 44 is used to multiply each pixel in the image block with the gain value corresponding to each pixel to achieve exposure compensation for the image of the same area in the corrected image.
[0127] In the embodiment of the present disclosure, the image processed by the Gaussian filter calculation module 42 can make the gain of the block image smoother.
[0128] In the embodiment of the present disclosure, the gain interpolation calculation module 43 may be a single linear interpolation calculation controller, configured to obtain a gain value of each pixel in the block image.
[0129] In some embodiments of the present disclosure, Figure 10 As shown, the present disclosure provides a structural diagram of a specific hierarchical computing module, and the hierarchical computing module 21 includes:
[0130] Original image cache module 211, Gaussian pyramid calculation module 212, first fusion calculation module 213 and first data selection module 214;
[0131] The original image cache module 211 is connected to the first fusion calculation module 213, the Gaussian pyramid calculation module 212 is connected to the first fusion calculation module 213, and the first fusion calculation module 213 is connected to the first data selection module 214. The original image cache module 211 is used to cache the original image after exposure compensation. The Gaussian pyramid calculation module 212 is used to perform Gaussian processing on the original image after exposure compensation to obtain a Gaussian image. The first fusion calculation module 213 is used to subtract the original image after exposure compensation from the Gaussian image to obtain a layered image. The first data selection module 214 is used to determine the number of Gaussian pyramid levels of the Gaussian pyramid calculation module and select a layered image based on the number of Gaussian pyramid levels.
[0132] In an embodiment of the present disclosure, the Gaussian pyramid calculation module 212 includes a downsampling Gaussian filter calculation controller and an upsampling Gaussian filter calculation controller, wherein the downsampling Gaussian filter calculation controller is used to downsample the original image by a factor of 2 to obtain a layered image with higher resolution, and the upsampling Gaussian filter calculation controller is used to upsample the downsampled data by a factor of 2 to obtain a Gaussian image with the same resolution as the original image. The Gaussian image refers to the output image of the upsampling Gaussian filter calculation controller.
[0133] In the embodiments of the present disclosure, Figure 11 As shown, Figure 11 This is a structural diagram of the processing process of the layered calculation module 21, and it is determined that the Gaussian pyramid level is 4 layers, among which the original data buffer is the original image cache module 211 in this application, the fusion calculation controller is the first fusion calculation module 213 in this application, and the data selector is the first data selection module 214 in this application. The Laplace fusion image output by each layer corresponds to the layered image in this application. Since the resolution of the layered image is relatively large, it will be written to the storage device. The resolution of the top-level image is relatively small, so it can be directly cached in the top-level Gaussian image buffer. Specifically, the top-level Gaussian image buffer is a local cache.
[0134] In some embodiments of the present disclosure, Figure 12 As shown, the present disclosure provides a structural diagram of a reconstruction calculation module, and the reconstruction calculation module 22 includes:
[0135] A data cache module 221, an upsampling module 222, a second fusion calculation module 223 and a second data selection module 224;
[0136] The data cache module 221 is connected to the upsampling module 222, the upsampling module 222 is connected to the second fusion calculation module 223, the second fusion calculation module 223 is connected to the second data selection module 224, the data cache module 221 is used to cache the layered image, the upsampling module 222 is used to upsample the layered image, the second fusion calculation module 223 is used to fuse the upsampled image with the layered image to obtain a spliced image, and the second data selection module 224 is used to select the spliced image according to the number of Gaussian pyramid levels of the Gaussian pyramid calculation module.
[0137] In the embodiment of the present disclosure, the reconstruction calculation module 22 is a Gaussian pyramid reconstruction calculation process from top to bottom inversely, such as Figure 13 As shown, Figure 13To obtain the structural diagram of the stitched image, specifically, the data buffer is the data buffer module 221 in this application, the upsampling Gaussian filter calculation controller is the upsampling module 222 in this application, the fusion calculation controller is the second fusion calculation module 223 in this application, and the data selector is the second data selection module 224. The Gaussian image of the top layer is used as input, and the same 2x upsampling calculation and Gaussian filtering are performed. Then, the fusion is completed by pixel addition with the Laplace fusion image of the current layer. In this way, the image of the current layer is reconstructed and used as the input of the next layer. The reconstruction is continued in the same way until the first layer, that is, the original image layer.
[0138] In an embodiment of the present disclosure, the stitched image is written into a storage device to complete the entire stitched image calculation process.
[0139] In some embodiments of the present disclosure, Figure 14 As shown, the present disclosure provides a structural schematic diagram of a specific image splicing circuit, and the image splicing circuit further includes:
[0140] Configuration module 6, configuration module 6 is used to configure parameters and instructions in the image splicing circuit.
[0141] In the embodiment of the present disclosure, the parameters and instructions in the image stitching circuit include input original image, output stitching image address, cache layered image address, address of mapping coordinate parameters, number of pyramid layers and other related parameters.
[0142] In an embodiment of the present disclosure, the configuration module 6 may further configure whether to enable color correction and whether to enable exposure compensation.
[0143] In the embodiment of the present disclosure, the configuration module 6 can make the entire image splicing circuit more flexible and adaptable to various complex application scenarios.
[0144] In the embodiments of the present disclosure, Figure 14 As shown, the configuration module 6 can be connected to the bus slave device interface Bus_slave_if, and the bus is connected to the bus master device interface Bus_Master_if.
[0145] In summary, the image stitching circuit provided by the present invention includes an image distortion correction module, an image stitching module and a bus; the image distortion correction module is connected to a first data input module and a first data output module, the first data input module is used to obtain the original image and transmit the original image to the image distortion correction module, the original image is at least two images including the same area, the image distortion correction module is used to perform distortion correction processing on the original image to obtain a corrected image, the first data output module is used to obtain the corrected image and transmit the corrected image to the bus; the image stitching module is connected to a second data input module and a second data output module, the second data input module is used to obtain the corrected image from the bus, the image stitching module is used to perform stitching processing on the corrected image to obtain a stitched image, and the second data output module is used to output the stitched image. The image stitching circuit determined by the image distortion correction module, the image stitching module and the bus can reduce the power consumption and cost of the CPU and improve the efficiency of image stitching.
[0146] Figure 15 This is a flow chart of an image stitching method provided by an embodiment of the present disclosure. Figure 15 As shown, the image stitching method is applied to the above Figures 1 to 14 The image splicing circuit shown in FIG. 1 includes steps 1501 to 1503 .
[0147] Step 1501: obtaining original images, which are at least two images including the same area;
[0148] Step 1502: performing distortion correction processing on the original image to obtain a corrected image;
[0149] Step 1503: stitch the corrected images to obtain a stitched image.
[0150] In an embodiment of the present disclosure, the original image may be at least two images of the same area.
[0151] In an embodiment of the present disclosure, the original image may be at least two images of the same upper and lower areas.
[0152] In an embodiment of the present disclosure, before stitching the corrected images to obtain the stitched images, color correction and exposure compensation may be optionally performed on the corrected images.
[0153] In the embodiments of the present disclosure, distortion correction processing is performed on the original image to remove the distortion caused by the image acquisition device, such as the camera lens, and at the same time, the two stitched images are aligned to facilitate the subsequent pyramid calculation fusion processing, so that the final fusion effect is more natural.
[0154] In the embodiments of the present disclosure, by performing color correction and exposure compensation on the aforementioned distortion-corrected image, color balance can be achieved, ensuring that images from different sources have uniform color and brightness after stitching, thereby improving the overall quality and visual effect of the panoramic image.
[0155] In summary, according to the image stitching method proposed in the present disclosure, the original image is obtained through the image stitching circuit, and the original image is at least two images including the same area; the original image is subjected to distortion correction processing to obtain a corrected image; the corrected image is stitched to obtain a stitched image. In addition, the corrected image can be selectively subjected to color correction and exposure compensation to compensate for the color and brightness details of the stitched image. By executing the image stitching method through the image stitching circuit in this application, the power consumption and cost of the CPU can be reduced, and the efficiency of image stitching can be improved.
[0156] Figure 16 This is a structural diagram of an image splicing device provided by an embodiment of the present disclosure. Figure 16 As shown, the image stitching device 1600 includes:
[0157] An acquiring unit 1601 is configured to acquire original images, where the original images are at least two images including the same area.
[0158] The correction unit 1602 is used to perform distortion correction processing on the original image to obtain a corrected image;
[0159] The stitching unit 1603 is configured to stitch the rectified images to obtain a stitched image.
[0160] Corresponding to the methods provided in the above-mentioned embodiments, the present disclosure also provides a testing device. Since the device provided in the embodiment of the present disclosure corresponds to the methods provided in the above-mentioned embodiments, the implementation method is also applicable to the device provided in this embodiment and will not be described in detail in this embodiment.
[0161] An embodiment of the present disclosure also provides an electronic device.
[0162] Specifically, the electronic device includes: the image splicing circuit described in any one of the above embodiments, or the image splicing method described in the embodiments.
[0163] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present disclosure. In this specification, the schematic expressions of the above terms may be directed to different embodiments or examples. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, unless they are mutually inconsistent.
[0164] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features being referred to. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one such feature. Throughout the present disclosure, "plurality" means at least two, such as two, three, etc., unless otherwise specifically defined.
[0165] Although the embodiments of the present disclosure have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and alterations may be made to the embodiments without departing from the principles and spirit of the present disclosure, the scope of which is defined by the claims and their equivalents.
Claims
1. An image splicing circuit, characterized in that: The image splicing circuit includes: Image distortion correction module, image stitching module and bus; The image distortion correction module is connected to a first data input module and a first data output module. The first data input module is used to obtain original images and transmit the original images to the image distortion correction module. The original images are at least two images including the same area. The image distortion correction module is used to perform distortion correction processing on the original images to obtain corrected images. The first data output module is used to obtain the corrected images and transmit the corrected images to the bus. The image stitching module is connected to a second data input module and a second data output module. The second data input module is used to obtain the corrected image from the bus. The image stitching module is used to stitch the corrected image to obtain a stitched image. The second data output module is used to output the stitched image.
2. The image splicing circuit according to claim 1, characterized in that: The image distortion correction module includes: Interpolation coordinate calculation module, image interpolation calculation module and color correction calculation module; The interpolation coordinate calculation module is connected to the first data input module, the interpolation coordinate calculation module is connected to the image interpolation calculation module, the image interpolation calculation module is connected to the color correction calculation module, and the color correction calculation module is connected to the first data output module; The interpolation coordinate calculation module is used to scale the parameters of the image acquisition device to obtain the scaled parameters, the image interpolation calculation module is used to interpolate the original image based on the scaled parameters, and the color correction calculation module is used to perform color correction on the interpolated image to obtain the corrected image.
3. The image splicing circuit according to claim 2, characterized in that: The interpolation coordinate calculation module includes: Coordinate calculation module and coordinate cache module; The coordinate calculation module is connected to the first data input module and the coordinate cache module. The coordinate calculation module is used to obtain the interpolation coordinates of the original image. The coordinate calculation module includes a horizontal coordinate calculation module and a vertical coordinate calculation module. The coordinate cache module is used to cache the interpolation coordinates of the original image.
4. The image splicing circuit according to claim 3, characterized in that: The image interpolation calculation module includes: Ping-pong cache module and linear interpolation calculation module; The interpolation coordinate calculation module is connected to the ping-pong cache module, the ping-pong cache module is connected to the linear interpolation calculation module, the ping-pong cache module is used to cache the original image, and the linear interpolation calculation module is used to perform interpolation calculation on the original image based on the interpolation coordinates of the original image.
5. The image splicing circuit according to claim 1, characterized in that: The image stitching module includes: Hierarchical calculation module and reconstruction calculation module; The layered calculation module is connected to the second data input module and the reconstruction calculation module, and the reconstruction calculation module is connected to the second data output module. The layered calculation module is used to perform layered processing on the images of the same area in the corrected image to obtain layered images, and the reconstruction calculation module is used to reconstruct the layered images to obtain the spliced image.
6. The image splicing circuit according to claim 5, characterized in that: The image splicing circuit further includes: Exposure compensation module; The exposure compensation module is connected to the second data input module and the layer calculation module, and is used to compensate the image in the same area in the corrected image.
7. The image splicing circuit according to claim 5, characterized in that: The image splicing circuit further includes: Hierarchical image caching module; The layered image cache module is connected to the layered calculation module and the reconstruction calculation module, and is used to cache the layered image.
8. The image splicing circuit according to claim 6, characterized in that: The exposure compensation module includes: Image block gain calculation module, Gaussian filter calculation module, gain interpolation calculation module and image compensation calculation module; The image block gain calculation module is connected to the Gaussian filter calculation module, which is connected to the gain interpolation calculation module, which is connected to the image compensation calculation module. The image block gain calculation module is used to segment the image in the same area of the corrected image and calculate the gain value of each image block in the segmented image block. The Gaussian filter calculation module is used to perform Gaussian filtering on the gain value of each image block. The gain interpolation calculation module is used to calculate the gain value corresponding to each pixel in the image block. The image compensation calculation module is used to multiply each pixel in the image block by the gain value corresponding to each pixel to achieve exposure compensation for the image in the same area of the corrected image.
9. The image splicing circuit according to claim 8, characterized in that: The hierarchical calculation module includes: Original image cache module, Gaussian pyramid calculation module, first fusion calculation module and first data selection module; The original image cache module is connected to the first fusion calculation module, the Gaussian pyramid calculation module is connected to the first fusion calculation module, and the first fusion calculation module is connected to the first data selection module. The original image cache module is used to cache the exposure-compensated original image. The Gaussian pyramid calculation module is used to perform Gaussian processing on the exposure-compensated original image to obtain a Gaussian image. The first fusion calculation module is used to subtract the exposure-compensated original image from the Gaussian image to obtain a layered image. The first data selection module is used to determine the number of Gaussian pyramid levels of the Gaussian pyramid calculation module and select the layered image based on the number of Gaussian pyramid levels.
10. The image splicing circuit according to claim 9, characterized in that: The reconstruction calculation module includes: A data cache module, an upsampling module, a second fusion calculation module and a second data selection module; The data cache module is connected to the upsampling module, the upsampling module is connected to the second fusion calculation module, the second fusion calculation module is connected to the second data selection module, the data cache module is used to cache the layered image, the upsampling module is used to upsample the layered image, the second fusion calculation module is used to fuse the upsampled image with the layered image to obtain the stitched image, and the second data selection module is used to select the stitched image according to the number of Gaussian pyramid levels of the Gaussian pyramid calculation module.
11. The image splicing circuit according to claim 1, characterized in that: The image splicing circuit further includes: A configuration module is used to configure parameters and instructions in the image stitching circuit.
12. An image stitching method, characterized in that: The method is performed by the image splicing circuit according to any one of claims 1 to 11, and the method includes: Acquire original images, where the original images are at least two images including the same area; Performing distortion correction processing on the original image to obtain a corrected image; The corrected images are stitched to obtain a stitched image.
13. An electronic device, characterized in that: include: The image stitching circuit according to any one of claims 1 to 11, or the image stitching method according to claim 12.