Method for shooting and outputting high-definition images based on multiple low-cost cameras

By fixing multiple low-cost cameras and pre-processing, registering and splicing images, the problem of high-resolution image shooting costs is solved, and the effect of low-cost cameras to generate high-definition images is achieved.

CN112509016BActive Publication Date: 2024-12-06HANGZHOU XIANGZHENG TECH CO LTD
View PDF 6 Cites 0 Cited by

Patent Information

Application Number
CN202011043801.8
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2020-09-28
Publication Date
2024-12-06
Estimated Expiration
2040-09-28

AI Technical Summary

Technical Problem

When shooting high-definition videos, high-resolution cameras are costly, and it is difficult for the existing technology to effectively synthesize high-resolution images through low-cost cameras.

Method used

By fixing multiple low-cost cameras, their adjacent pictures have overlapping parts, pre-processing, image registration and splicing, and high-definition pictures are generated. The specific steps include preprocessing, image registration and splicing of adjacent cameras, shortening processing time using feature point detection and edge detection, and selecting appropriate splicing algorithms to avoid ghosting.

Benefits of technology

It realizes the generation of high-resolution images through multiple low-cost, low-resolution cameras, significantly reducing the cost of high-definition images.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN112509016B_ABST
    Figure CN112509016B_ABST
Patent Text Reader

Abstract

The present invention provides a method for outputting high-definition images based on shooting with multiple low-cost cameras, which includes: S1, fixing multiple low-cost cameras so that the images shot by two adjacent low-cost cameras have overlapping parts; S2, for the images shot by two adjacent low-cost cameras, processing as follows: S21, preprocessing the images to obtain two preprocessed images; S22, performing image registration on the two preprocessed images to obtain two registered images; S23, splicing the two registered images to obtain a spliced ​​image; S3, performing the processing of S2 on the images shot by all two adjacent low-cost cameras to output high-definition images. The present invention obtains a high-resolution high-definition camera by using multiple low-cost and low-resolution lenses and image splicing, which can greatly reduce the cost of shooting high-definition images.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of shooting, and in particular to a method for shooting and outputting high-definition images based on multiple low-cost cameras. Background Art

[0002] When shooting videos, if you need high-definition images, you often need to increase the resolution of the video. However, high-resolution HD cameras are not cheap. With the development of image processing technology, it is gradually possible to use images taken by multiple low-resolution lenses at a lower cost to synthesize high-resolution HD images. Summary of the invention

[0003] In view of the above problems, the present invention provides a method for shooting and outputting high-definition images based on multiple low-cost cameras to solve the above problems.

[0004] The present application provides a method for shooting and outputting high-definition images based on multiple low-cost cameras, which includes:

[0005] S1, fixing a plurality of low-cost cameras so that the images captured by two adjacent low-cost cameras have overlapping parts;

[0006] S2, for the images taken by two adjacent low-cost cameras, the processing is as follows:

[0007] S21, preprocessing the picture to obtain two preprocessed images;

[0008] S22, performing image registration on the two pre-processed images to obtain two registered images;

[0009] S23, stitching the two registered images to obtain a stitched image;

[0010] S3, processes the images taken by all two adjacent low-cost cameras in S2 and outputs high-definition images.

[0011] The beneficial effects of the present invention are:

[0012] The present invention obtains a high-resolution HD camera by using a plurality of low-cost and low-resolution lenses and an image stitching method, which can significantly reduce the cost of shooting a high-definition picture. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] The present invention is further described using the accompanying drawings, but the embodiments in the accompanying drawings do not constitute any limitation to the present invention. A person skilled in the art can obtain other drawings based on the following drawings without creative work.

[0014] Figure 1, is a diagram showing an exemplary embodiment of a method for outputting high-definition images based on shooting with multiple low-cost cameras according to the present invention. DETAILED DESCRIPTION

[0015] The present invention is further described in conjunction with the following examples.

[0016] See also Figure 1 The present invention provides a method for shooting and outputting high-definition images based on multiple low-cost cameras, which includes:

[0017] S1, fixing a plurality of low-cost cameras so that the images captured by two adjacent low-cost cameras have overlapping parts;

[0018] S2, for the images taken by two adjacent low-cost cameras, the processing is as follows:

[0019] S21, preprocessing the picture to obtain two preprocessed images;

[0020] S22, performing image registration on the two pre-processed images to obtain two registered images;

[0021] S23, stitching the two registered images to obtain a stitched image;

[0022] S3, processes the images taken by all two adjacent low-cost cameras in S2 and outputs high-definition images.

[0023] The images captured by two adjacent low-cost cameras have a set ratio of overlap, which means that in real scenes, part of the scene appears in two adjacent shots at the same time. For example, when an object is located between two adjacent cameras on the left and right, at a suitable distance, the object will appear on the right side of the image captured by the left camera and on the left side of the image captured by the right camera.

[0024] In a preferred embodiment, the area of ​​the overlapping parts accounts for less than 1 / 4 of the captured image.

[0025] In one embodiment, the picture is preprocessed to obtain two preprocessed images, including:

[0026] The images taken by two adjacent low-cost cameras are respectively subjected to distortion correction to obtain two pre-processed images.

[0027] In one embodiment, the following model is used to perform distortion correction on the picture:

[0028]

[0029] In the formula, x and y represent the horizontal coordinate and the vertical coordinate before the image distortion correction, respectively, ax and ay represent the horizontal coordinate and the vertical coordinate after the image distortion correction, respectively, and b and c represent the distortion parameters of the low-cost camera.

[0030] In one embodiment, performing image registration on two preprocessed images to obtain two registered images includes:

[0031] Perform feature point detection on the two preprocessed images respectively;

[0032] Based on the detected feature points, feature point matching is performed.

[0033] In one embodiment, performing feature point detection on the preprocessed image includes:

[0034] The pre-processed image wP1 is divided into two first segmented images with the same area;

[0035] For a first segmented image P1 having an overlapping portion with another pre-processed image wP2 adjacent to wP1, P1 is divided into two second segmented images with the same area;

[0036] For the second segmented image P2 which is closer to wP2 among the two second segmented images, block processing is performed on it to obtain multiple blocks;

[0037] The multiple blocks are screened, edge detection is performed on the screened blocks, and the total number of edge pixels is calculated. If the total number of edge pixels is greater than a set threshold value, feature point detection is performed on the pixels in the multiple blocks; otherwise, feature detection is performed on all the pixels in P2.

[0038] Traditional feature detection algorithms are often global feature detection methods, which are time-consuming to calculate and are not conducive to the rapid output of spliced ​​images. The above embodiment of the present application divides the image into only one-fourth of the image that is closest to the overlapping part of the adjacent image, and then performs feature point detection therein, which can undoubtedly greatly shorten the time for feature point detection. Because generally speaking, the overlapping parts will not exceed 1 / 4, and if the overlapping area is too large, the meaning of image splicing is not great. This can be achieved by adjusting the positional relationship between the cameras.

[0039] In order to further shorten the time for feature point detection, the present application divides P2 into blocks, and then further screens the divided blocks. For the screened blocks, if they meet the conditions, feature point detection is performed; if they do not meet the conditions, the range of feature point detection is expanded. In this way, the time for feature point detection can be greatly shortened.

[0040] Edge detection is creatively used as the basis for screening, because edge pixels carry more information and often become feature points. The edge detection algorithm used needs to have a fast calculation speed, such as the Roberts algorithm, which has low calculation complexity and fast speed.

[0041] In one embodiment, the block processing of P2 includes:

[0042] P2 is divided into blocks using the quadtree block algorithm. For a certain block, whether to divide it into blocks again is determined by the following method:

[0043] Calculate its smoothness, if the smoothness is less than the set smoothness threshold, then no longer divide it into blocks, otherwise, continue to use the quadtree block algorithm to divide it into blocks;

[0044] The smoothness is calculated as:

[0045]

[0046] Where α and β represent the set scale coefficients, |I| represents the total number of pixels in block I, and f i represents the pixel value of pixel i in block I, f ave represents the average pixel value in block I, and S represents the area of ​​block I.

[0047] If the smoothness is less than the smoothness threshold, it means that the difference in pixel values ​​between the pixels in the block is small and the similarity is high, so there is no need to divide it into blocks. This method can filter the background blocks well, and the difference between the background pixels is relatively small. It is meaningless to divide these blocks again. If the block is too small, it is also meaningless and will increase the time for feature point detection. Therefore, this application adds the parameter of the block area when calculating the smoothness, which can control the time required for calculation within a relatively appropriate range and shorten the time for feature point detection.

[0048] In one embodiment, screening the multiple blocks includes:

[0049] Calculate the screening index for each block:

[0050] sIndex=c1×bz1+c2×bz2

[0051] Where bz1 represents the standard deviation of the pixel values ​​in the block, bz2 represents the standard deviation of the gradient amplitude of the pixel points in the block, c1 and c2 represent weight parameters;

[0052] If it is less than the set screening threshold, the block fails the screening; otherwise, the block passes the screening.

[0053] In one embodiment, stitching two registered images to obtain a stitched image includes:

[0054] The two registered images are denoted as pz1 and pz2. For pz1 at time t, the difference is made between it and the previous frame image taken by the same camera, that is, the image ipz1 at time t-1, so as to determine whether there is a moving object in pz1. If so, the stitching algorithm is used to splice pz1 and pz2. If not, the following algorithm is used to splice pz1 and pz2:

[0055]

[0056] In the formula, f(x,y) represents the pixel value of the pixel at position (x,y) after splicing, and f pz1 (x, y) represents the pixel value of the pixel at position (x, y) in pz1 after splicing, f pz2 (x,y) represents the pixel value of the pixel at position (x,y) in pz2 after stitching, pz1∩pz2 represents the set of pixels in the overlapping part of pz1 and pz2 after stitching, u1 and u2 represent the weights of the pixel values ​​of pz1 and pz2 in the overlapping part, respectively.

[0057]

[0058] Where xma and xmi represent the maximum and minimum values ​​of the horizontal coordinates of the overlapping areas in the stitched images, respectively, and f ave (x, y) represents the average pixel value of all pixels in a neighborhood of size kxk centered at the pixel at position (x, y), td(x, y) represents the pixel gradient amplitude at position (x, y), γ represents the control coefficient, and the value of g is as follows:

[0059]

[0060] Wherein, t1 is the set comparison threshold.

[0061] When stitching images, by judging whether there is a moving object in the image to be stitched, that is, the registered image, adaptively selecting different algorithms for image stitching can effectively avoid the problem of ghosting that is prone to occur in traditional stitching algorithms when stitching images with moving objects. When it is judged that there is a moving object, using a stitching line algorithm to stitch the images can solve this problem well, but since the judgment may not be accurate enough, missed judgments may occur. Therefore, when it is judged that there is no moving target, the present application improves the weighted stitching algorithm, and when calculating the weights u1 and u2, factors such as the maximum and minimum values ​​of the horizontal coordinates of the overlapping area, the average pixel value of the neighborhood of the current pixel, and the gradient amplitude of the current pixel are added, which can alleviate the problem of ghosting to a certain extent.

[0062] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention, rather than to limit the scope of protection of the present invention. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solution of the present invention can be modified or replaced by equivalents without departing from the essence and scope of the technical solution of the present invention.

Claims

1. A method for shooting and outputting high-definition images based on multiple low-cost cameras, characterized in that: It includes: S1, fixing a plurality of low-cost cameras so that the images captured by two adjacent low-cost cameras have overlapping parts; S2, for the images taken by two adjacent low-cost cameras, the processing is as follows: S21, preprocessing the picture to obtain two preprocessed images; S22, performing image registration on the two pre-processed images to obtain two registered images; S23, stitching the two registered images to obtain a stitched image; S3, processes the images captured by all two adjacent low-cost cameras in S2 and outputs high-definition images; Perform image registration on the two preprocessed images to obtain two registered images, including: Perform feature point detection on the two preprocessed images respectively; Based on the detected feature points, feature point matching is performed; Perform feature point detection on the preprocessed image, including: The pre-processed image wP1 is divided into two first segmented images with the same area; For a first segmented image P1 having an overlapping portion with another pre-processed image wP2 adjacent to wP1, P1 is divided into two second segmented images with the same area; For the second segmented image P2 which is closer to wP2 among the two second segmented images, block processing is performed on it to obtain multiple blocks; Screening the multiple blocks, performing edge detection on the screened blocks, and calculating the total number of edge pixels. If the total number of edge pixels is greater than a set number threshold, performing feature point detection on the pixels in the multiple blocks; otherwise, performing feature detection on all the pixels in P2; P2 is processed in blocks, including: P2 is divided into blocks using the quadtree block algorithm. For a certain block, whether to divide it into blocks again is determined by the following method: Calculate its smoothness, if the smoothness is less than the set smoothness threshold, then no longer divide it into blocks, otherwise, continue to use the quadtree block algorithm to divide it into blocks; The smoothness is calculated as: In the formula, α and β represent the set scale coefficients, |I| represents the total number of pixels in block I, and f i represents the pixel value of pixel i in block I, f ave represents the average pixel value in block I, and S represents the area of ​​block I.

2. The method for shooting and outputting high-definition images based on multiple low-cost cameras according to claim 1, characterized in that: The picture is preprocessed to obtain two preprocessed images, including: The images taken by two adjacent low-cost cameras are respectively subjected to distortion correction to obtain two pre-processed images.

3. The method for outputting high-definition images based on multiple low-cost cameras according to claim 2, characterized in that: The following model is used to correct the distortion of the image: In the formula, x and y represent the horizontal coordinate and the vertical coordinate before the image distortion correction, respectively, ax and ay represent the horizontal coordinate and the vertical coordinate after the image distortion correction, respectively, and b and c represent the distortion parameters of the low-cost camera.

Citation Information

Patent Citations

  • Image stitching processing system

    CN103279939A

  • Thread bucket interior wall image 360 DEG panorama mosaicing method and system

    CN105957015A

  • Image mosaic method and device

    CN106447602A

  • High resolution remote sensing image registration method based on peripheral point characteristics

    CN107301661A

  • Image splicing method for complex images

    CN110020995A