An image stitching system and method based on OpenCV

Abstract

The present invention discloses an image stitching system and method based on OpenCV, and the system includes: an image acquisition module, an image splitting module, an image cache module, an image processing module, an image analysis module and a panoramic image storage module. The method includes the steps of: acquiring multiple video images of the same viewing angle through a camera, and converting the video stream into a frame image by using OpenCV; further decomposing each frame image into pixels; binding the key value of each frame image and the decomposition of the frame image pixels and sending them to the image storage server cache; stitching the acquired images with pixels based on the panoramic video stitching algorithm of OpenCV to generate a panoramic image; calling an image analysis tool to detect whether the image imaging effect is abnormal by analyzing the relevant features of the panoramic image; and storing the stitched panoramic image in a distributed file system for subsequent viewing or calling of specific applications. The present invention analyzes the imaging effect of the image, and the imaging effect of the overall image is better.

Description

Technical Field

[0001] The invention belongs to the technical field of image processing, and in particular relates to an image stitching system and method based on OpenCV. Background Art

[0002] OpenCV is a cross-platform computer vision and machine learning software library released under the BSD license (open source), which can run on Linux, Windows, Android and Mac OS operating systems. It is lightweight and efficient - it consists of a series of C functions and a small number of C++ classes, and provides interfaces for languages ​​such as Python, Ruby, and MATLAB. It implements many common algorithms in image processing and computer vision, including a series of basic methods in image processing. Some methods can be used directly. In addition, we can also perform a series of optimization operations based on the native code and interface of OpenCV.

[0003] The existing image stitching method only processes how to stitch each frame of the image, but does not analyze the imaging effect of the image, resulting in poor imaging effect of the overall image and some images are blurred. Summary of the invention

[0004] Aiming at the problem that the prior art does not analyze the imaging effect of pictures, resulting in poor imaging effect of the overall image, the present invention provides an image stitching system and method based on OpenCV.

[0005] In order to achieve the above technical objectives, the technical solution adopted by the present invention is as follows:

[0006] An image stitching system based on OpenCV, including

[0007] The image acquisition module is a multi-angle surveillance camera that is used to rotate multiple times at different elevation angles to collect video images and use OpenCV to convert video streams into frame images;

[0008] Image splitting module, used to split each frame of image into pixels;

[0009] The image cache module is an image storage server, which is used to disassemble and bind the key value of each frame of the image and the pixel points of the frame of the image and send them to the image storage server cache;

[0010] The image processing module is an image data stitching processing engine used to obtain cached images in the server; based on the panoramic video stitching algorithm of OpenCV, the acquired images are stitched pixel by pixel to generate a panoramic picture;

[0011] The image analysis module calls the image analysis tool to detect whether the image imaging effect is abnormal by analyzing the relevant features of the panoramic image. If the imaging effect is normal, it is sent to the panoramic image storage module for storage. If the imaging effect is abnormal, the pixel points of the frame images of other cameras at the same time are retrieved and sent to the image processing module for splicing and replacing the abnormal frame image pixels;

[0012] The panoramic picture storage module stores the stitched panoramic images in the distributed file system for use by specific applications.

[0013] An image stitching method based on OpenCV, comprising the steps of:

[0014] Capture multiple video images of the same viewpoint through the camera, and use OpenCV to convert the video stream into frame images;

[0015] Further decompose each frame of the image into pixels;

[0016] The key value of each frame of the image and the pixel points of the frame are disassembled and bound and sent to the image storage server cache;

[0017] Based on the OpenCV panoramic video stitching algorithm, the acquired images are stitched pixel by pixel to generate a panoramic picture;

[0018] Call the image analysis tool to detect whether the image imaging effect is abnormal by analyzing the relevant features of the panoramic image;

[0019] If the imaging effect is normal, it is sent to the panoramic picture storage module for storage. If the imaging effect is abnormal, the pixel points of the frame pictures of other cameras at the same time are retrieved and sent to the image processing module to splice and replace the abnormal frame picture pixels;

[0020] The stitched panoramic image is stored in a distributed file system for subsequent viewing or calling of specific applications.

[0021] Furthermore, the number of the cameras is three or more, and the three cameras are arranged in three different planes in the space to be photographed.

[0022] Furthermore, the cache directory structure of the image storage server is stored in the form of a queue.

[0023] Furthermore, the key value of each frame image and the pixel points of the frame image are disassembled and bound and sent to the image storage server cache; at the same time, the key value of each frame image and the pixel points of the frame image are disassembled and bound and sent to the backup image storage server for storage.

[0024] Furthermore, the key value of each frame of the image is unique, and the key value of each frame of the image corresponds to a pixel point of the frame of the image.

[0025] Compared with the prior art, the present invention has the following beneficial effects:

[0026] By adding an image analysis module to the existing OpenCV image stitching system or method, it is possible to promptly detect whether the imaging effect of the stitched panoramic image is abnormal, and then timely process or re-stitch the image with poor imaging effect to achieve zero abnormality in the imaging effect of the panoramic image, thereby improving the imaging quality of the entire image stitching system or method process. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 This is an overall structural flow chart of an image stitching system based on OpenCV in the present invention;

[0028] Figure 2 The present invention is a flow chart of an image stitching method based on OpenCV. DETAILED DESCRIPTION

[0029] In order to facilitate the understanding of those skilled in the art, the present invention is further described below in conjunction with embodiments and drawings. The contents mentioned in the implementation modes are not intended to limit the present invention.

[0030] The frame image described in the present invention is a picture of the screen at a certain moment when the camera captures the image, that is, the frame image is a subset of the captured image; the pixel point refers to the picture of the screen at a certain moment further decomposed into multiple pixel point pictures, and the collection of multiple pixel points is a frame image, that is, the pixel point is a subset of the frame image.

[0031] like Figure 1 As shown, an image stitching system based on OpenCV includes: an image acquisition module, which is a multi-angle monitoring camera, used for rotating multiple circles at different elevation angles to collect video images, and using OpenCV to convert video streams into frame images; an image splitting module, used for disassembling each frame image into pixels; an image cache module, which is a picture storage server, used for binding the key value of each frame image and the disassembly of the frame image pixels and sending them to the picture storage server cache; an image processing module, which is a picture data stitching processing engine, used for obtaining cached images in the server; based on the panoramic video stitching algorithm of OpenCV, the acquired images are stitched with pixels to generate a panoramic image; an image analysis module, which calls the image analysis tool, and detects whether the image imaging effect is abnormal by analyzing the relevant features of the panoramic image. If the imaging effect is normal, it is sent to the panoramic image storage module for storage. If the imaging effect is abnormal, the pixel points of the frame images of other cameras at the same time are retrieved and sent to the image processing module for stitching and replacing the abnormal frame image pixels; the panoramic image storage module stores the stitched panoramic image in the distributed file system for calling specific applications.

[0032] The number of the cameras is 3 or more, and the 3 cameras are arranged in 3 different planes in the space to be photographed. One camera is arranged in the horizontal plane, one camera is arranged in the vertical plane, and one camera is arranged in the Z-axis plane direction; and due to the multi-angle steering characteristics of the camera, it is ensured that there is a viewing angle at any position in the space to be photographed or around the object. At the same time, it is also ensured that at the same time, the images captured by cameras with different viewing angles overlap. When a frame of picture is selected as the splicing background, if some pixels of the frame of picture are blurred or backlit, they can be replaced with the overlapping parts of other viewing angles at the same time, ensuring that the spliced ​​panoramic image is clearer and more complete.

[0033] like Figure 2 As shown, an image stitching method based on opencv includes the following steps: collecting multiple video images of the same viewing angle through a camera, and converting the video stream into frame images by using OpenCV; further decomposing each frame image into pixel points; binding the key value of each frame image and the decomposition of the frame image pixel points and sending them to the image storage server cache; based on the panoramic video stitching algorithm of OpenCV, stitching the acquired images pixel points to generate a panoramic image; calling the image analysis tool to detect whether the image imaging effect is abnormal by analyzing the relevant features of the panoramic image; if the imaging effect is normal, sending it to the panoramic image storage module for storage; if the imaging effect is abnormal, retrieving the pixel points of the frame images of other cameras at the same time and sending them to the image processing module for stitching and replacing the abnormal frame image pixel points; storing the stitched panoramic image in the distributed file system for subsequent viewing or calling of specific applications.

[0034] The cache directory structure of the image storage server is stored in the form of a queue. The data structure of the queue helps the image cache module to relieve storage pressure. When the image processing module extracts the image that has entered the queue first, it deletes its cache data. Once the image is extracted and processed, it will no longer occupy the cache space of the image storage server, thereby relieving the storage pressure of the image cache module. At the same time, the data storage structure of the queue protects the cached photos at the same time from being disrupted in the order of splicing due to storage, resulting in splicing errors.

[0035] The key value of each frame of the picture and the pixel points of the frame of the picture are decomposed and bound and sent to the picture storage server cache; at the same time, the key value of each frame of the picture and the pixel points of the frame of the picture need to be decomposed and bound and sent to the backup picture storage server for storage. The backup picture storage server is a permanent storage server. When the picture analysis tool is called, before re-stitching the picture with abnormal imaging effect, it is necessary to call the same frame picture pixel points or frame pictures of other cameras stored at that moment to replace and splice them, so as to achieve a clearer and more complete panoramic picture. Although the backup picture storage server is a permanent storage server, in this embodiment, the directory structure of the backup picture storage server is also stored in the form of a queue, which reduces the process of manual deletion and increases the storage space of the backup picture storage server, thereby ensuring that high-quality pictures can be fully stored within a certain period of time (1 month or half a year), and then automatically delete pictures that exceed the maximum storage time limit.

[0036] The key value of each frame image is unique, and each frame image key value corresponds to a frame image pixel. It is ensured that the bound frame image pixel has unique attributes, so that after verification, if there is a blurred or backlit frame image area, the same frame image pixel of other cameras at the same time is called to replace it, and the second stitching is completed. This improves the efficiency and quality of verifying and stitching panoramic images.

[0037] Compared with the prior art, the present invention has the following beneficial effects:

[0038] By adding an image analysis module to the existing OpenCV image stitching system or method, it is possible to promptly detect whether the imaging effect of the stitched panoramic image is abnormal, and then timely process or re-stitch the image with poor imaging effect to achieve zero abnormality in the imaging effect of the panoramic image, thereby improving the imaging quality of the entire image stitching system or method process.

[0039] The above is a detailed introduction to an image stitching system and method based on OpenCV provided by the present application. The description of the specific embodiment is only used to help understand the method and its core idea of ​​the present application. It should be pointed out that for ordinary technicians in this technical field, without departing from the principles of the present application, several improvements and modifications can be made to the present application, and these improvements and modifications also fall within the scope of protection of the claims of the present application.

Claims

1. An image stitching method based on opencv, characterized in that: Includes steps: Capture video images through the camera and use opencv to convert the video stream into frame images; Further decompose each frame of the image into pixels; Bind the key value of each frame of the image with the disassembled frame of the image pixel and send it to the image storage server cache; Based on the opencv panoramic video stitching algorithm, the acquired images are stitched pixel by pixel to generate a panoramic picture; Call the image analysis tool to detect whether the image imaging effect is abnormal by analyzing the relevant features of the panoramic image; If the imaging effect is normal, it is sent to the panoramic picture storage module for storage. If the imaging effect is abnormal, the pixel points of the frame pictures of other cameras at the same time are retrieved and sent to the image processing module to splice and replace the abnormal frame picture pixels; The stitched panoramic image is stored in a distributed file system for subsequent viewing or specific application calls; The cache directory structure of the image storage server is stored in the form of a queue; When the image processing module extracts the image in the queue first, it deletes its cache data, and the extracted and processed image will no longer occupy the cache space of the image storage server; The key value of each frame of the picture is bound to the disassembled frame of the picture pixel point and then sent to the picture storage server cache; At the same time, the key value of each frame of the image and the disassembled frame of the image pixel need to be bound and sent to the backup image storage server for storage; When the image analysis tool is called, before re-stitching the image with abnormal imaging effect, it is necessary to call the same frame image pixels of other cameras stored at that moment to replace and stitch them. The directory structure of the backup image storage server is also stored in the form of queues, and the images that exceed the maximum storage time limit are automatically deleted; The key value of each frame of the picture is unique, and the key value of each frame of the picture corresponds to a pixel point of the frame of the picture; For the areas where the frame images are blurred or backlit, the pixels of the same frame images from other cameras at the same time are used to replace them and complete the second stitching.

2. An opencv-based image stitching system for implementing the method of claim 1, characterized in that: include: The image acquisition module is a multi-angle surveillance camera that is used to rotate multiple times at different elevation angles to collect video images and use OpenCV to convert video streams into frame images; Image splitting module, used to split each frame of image into pixels; The image cache module is an image storage server, which is used to bind the key value of each frame of the image and the disassembled frame image pixel points and send them to the image storage server cache; The image processing module is a picture data stitching processing engine used to obtain cached images in the server; Based on the opencv panoramic video stitching algorithm, the acquired images are stitched pixel by pixel to generate a panoramic picture; The image analysis module calls the image analysis tool to detect whether the image imaging effect is abnormal by analyzing the relevant features of the panoramic image. If the imaging effect is normal, it is sent to the panoramic image storage module for storage. If the imaging effect is abnormal, the pixel points of the frame images of other cameras at the same time are retrieved and sent to the image processing module for splicing and replacing the abnormal frame image pixels; The panoramic image storage module stores the stitched panoramic images in the distributed file system for use by specific applications. There are three cameras, which are arranged in three different planes in the space to be photographed.

Images