Image fade area determination method, electronic device and storage medium

Abstract

The present invention provides an image fade area determination method, an electronic device and a storage medium, and relates to the field of video image fade area determination. The method comprises: obtaining an initial image to be spliced ​​C1 and an initial image to be spliced ​​C2 corresponding to a target area; obtaining an intermediate image to be spliced ​​ZC2 corresponding to C2 by a preset image splicing algorithm; adding ZC2 to a first preset rectangular canvas to obtain a target image to be spliced ​​MC2 corresponding to C2; establishing a second preset rectangular canvas and a rectangular coordinate system KT corresponding to the second preset rectangular canvas according to the sizes of C1 and MC2; adding PJT to the second preset rectangular canvas; obtaining a coordinate value x of BJ1 on the X axis in KT; C1 ; and the coordinate value x of BJ2 on the X axis in KT ZC2 ; According to x C1 and x ZC2 , determine the faded area in MC2; the present invention can determine the faded area of ​​the image to be spliced.

Description

Technical Field

[0001] The present invention relates to the field of video image fade area determination, and in particular to an image fade area determination method, electronic equipment and storage medium. Background Art

[0002] In the field of civil aviation, the length of an airport is generally long. When monitoring the airport, it is difficult for one camera to completely cover the entire airport scene; therefore, multiple cameras are usually set up, each camera covers a part of the airport, and then the video images collected by all the cameras are stitched to obtain a panoramic monitoring image of the airport; however, there will be a more obvious stitching area in the stitched image. In order to fade the stitching area, it is necessary to determine the faded area of ​​the image before stitching. Summary of the invention

[0003] In view of the above technical problems, the technical solution adopted by the present invention is:

[0004] According to a first aspect of the present application, a method for determining a faded area of ​​an image is provided, the method comprising the following steps:

[0005] S100, obtaining an initial image to be stitched C1 and an initial image to be stitched C2 corresponding to the target area; wherein C1 and C2 are two initial images to be stitched among a plurality of initial images to be stitched corresponding to the target area, and C1 and C2 have an overlapping area in the target area; C1 and C2 are obtained based on the first frame images generated at the same time of two video streams covering adjacent areas of the target area;

[0006] S200, obtaining an intermediate image ZC2 to be stitched corresponding to C2 through a preset image stitching algorithm; ZC2 is obtained through DH2 and C2, and DH2 is a homography matrix corresponding to C2 when C1 and C2 are stitched;

[0007] S300, adding ZC2 to the first preset rectangular canvas to obtain the target image to be stitched MC2 corresponding to C2; wherein each pixel in ZC2 is within the first preset rectangular canvas; the color of the first preset rectangular canvas is a preset color; and there is an overlapping area between MC2 and C1;

[0008] S400: Establish a second preset rectangular canvas and a rectangular coordinate system KT corresponding to the second preset rectangular canvas according to the sizes of C1 and MC2; wherein L≥L C1 +L MC2 , M ≥ M C1 +M MC2 ; L is the length of the two preset rectangular canvases, M is the width of the two preset rectangular canvases, L C1 is the length of C1, L MC2 is the length of C2, MC1 is the width of C1, M MC2 is the width of MC2;

[0009] S500, adding PJT to a second preset rectangular canvas; wherein PJT is a spliced ​​image generated by splicing C1 and MC2; and a horizontal boundary of C1 is parallel to a horizontal boundary of the second preset rectangular canvas;

[0010] S600: If the target boundary BJ1 of C1 is perpendicular to the X-axis, obtain the X-axis coordinate value x of BJ1 in KT. C1 ; Wherein, BJ1 is the boundary of C1 that is perpendicular to the X-axis and coincides with MC2;

[0011] S700: If the target boundary BJ2 of ZC2 is perpendicular to the X axis, obtain the X axis coordinate value x of BJ2 in KT. ZC2 ; Wherein, BJ2 is the boundary of ZC2 that coincides with C1;

[0012] S800, according to x C1 and x ZC2 , determine the faded area in MC2.

[0013] According to another aspect of the present application, a non-transitory computer-readable storage medium is provided, in which at least one instruction or at least one program is stored, and the at least one instruction or at least one program is loaded and executed by a processor to implement the above-mentioned method for determining the faded area of ​​the spliced ​​image.

[0014] According to another aspect of the present application, an electronic device is provided, including a processor and the above-mentioned non-transitory computer-readable storage medium.

[0015] The present invention has at least the following beneficial effects:

[0016] The method for determining the faded area of ​​a stitched image of the present invention obtains an initial image C1 and an initial image C2 to be stitched corresponding to a target area, obtains an intermediate image ZC2 to be stitched corresponding to C2 through a preset image stitching algorithm, and adds ZC2 to a first preset rectangular canvas to obtain a target image MC2 to be stitched corresponding to C2; establishes a second preset rectangular canvas and a rectangular coordinate system KT corresponding to the second preset rectangular canvas according to the sizes of C1 and MC2; adds PJT to the second preset rectangular canvas, and obtains the coordinate value x of the target boundary BJ1 of C1 on the X-axis in KT. C1 , if the target boundary BJ2 of ZC2 is perpendicular to the X-axis, then obtain the X-axis coordinate value x of BJ2 in KT ZC2 ; According to x C1 and x ZC2 , determine the faded area in MC2, thereby determining the faded area of ​​the image.

[0017] Furthermore, in the process of determining the faded area of ​​the image, the homography matrix DH2 corresponding to C2 when C1 and C2 are spliced ​​can be determined at the same time. When other images to be spliced ​​in the subsequent video stream corresponding to C2 are spliced, DH2 can be directly obtained without recalculating DH2. The time used to directly obtain the homography matrix DH2 is negligible compared to the time used to recalculate the homography matrix DH2. Therefore, the present invention can also improve the efficiency of subsequent video image splicing. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0019] Figure 1 The present invention provides a flowchart of a method for determining a faded area of ​​a spliced ​​image according to an embodiment of the present invention. DETAILED DESCRIPTION

[0020] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0021] It should be noted that various aspects of the embodiments within the scope of the appended claims are described below. It should be apparent that the aspects described herein may be embodied in a wide variety of forms, and any specific structure and / or function described herein is merely illustrative. Based on the present disclosure, it should be understood by those skilled in the art that an aspect described herein may be implemented independently of any other aspect, and two or more of these aspects may be combined in various ways. For example, any number of aspects described herein may be used to implement a device and / or practice a method. In addition, other structures and / or functionalities other than one or more of the aspects described herein may be used to implement this device and / or practice this method.

[0022] The following will refer to Figure 1 The flowchart of the method for determining the image fade area introduces a method for determining the image fade area; the method comprises the following steps:

[0023] S100, obtaining an initial image to be stitched C1 and an initial image to be stitched C2 corresponding to a target area; wherein C1 and C2 are two initial images to be stitched among a plurality of initial images to be stitched corresponding to the target area, and there is an overlapped area between C1 and C2 in the target area; C1 and C2 are obtained based on the first frame images generated at the same time of two video streams covering adjacent areas of the target area.

[0024] Specifically, for example, the target area is an airport, and n cameras with fixed positions and shooting angles are set at the boundary of the airport. Each sub-area of ​​the airport is monitored in real time in turn, and a video stream corresponding to each sub-area can be generated. The number of frames of images contained in the video stream generated by each camera within one second is the same. The images generated by n cameras at the same time point are obtained for the first time, or the first frame of images generated by n cameras at the same time point is obtained, and n initial images to be spliced ​​corresponding to the target area can be obtained; it should be noted that when the target area is divided into sub-areas, there are overlapping areas between two adjacent sub-areas. Therefore, there are also overlapping images in the video streams corresponding to two adjacent sub-areas, that is, C1 and C2 have overlapping areas in the target area.

[0025] S200, obtaining an intermediate image ZC2 to be stitched corresponding to C2 through a preset image stitching algorithm; ZC2 is obtained through DH2 and C2, and DH2 is the homography matrix corresponding to C2 when C1 and C2 are stitched.

[0026] In this embodiment, when C1 and C2 are stitched, the SIFT algorithm is used. The algorithm can automatically calculate the homography matrix DH2 corresponding to C2 when C1 is used as the main image to be stitched and stitched with C2; in the subsequent stitching process, the homography matrices corresponding to other images to be stitched that belong to the same video stream as C2 are all DH2. Therefore, the determined DH2 can be stored in a preset storage space, and when other images to be stitched that belong to the same video stream as C2 are stitched, DH2 can be directly obtained; since the calculation of the homography matrix will take up a lot of computing power, in this embodiment, the homography is calculated only once, therefore, the efficiency of image stitching can be greatly improved, thereby improving the smoothness of the spliced ​​video image.

[0027] S300, adding ZC2 to the first preset rectangular canvas to obtain the target image to be stitched MC2 corresponding to C2; wherein each pixel in ZC2 is in the first preset rectangular canvas; the color of the first preset rectangular canvas is the preset color; and there is an overlapping area between MC2 and C1.

[0028] In this embodiment, after C2 and DH2 are multiplied, the effective image of C2 is distorted so that C2 is no longer a rectangle. The image obtained after multiplying C2 and DH2 can be added to the preset black rectangular canvas corresponding to C2 to obtain the rectangular MC2; that is, there will be a black image in a certain area in MC2.

[0029] S400: Establish a second preset rectangular canvas and a rectangular coordinate system KT corresponding to the second preset rectangular canvas according to the sizes of C1 and MC2; wherein L≥L C1 +L MC2 , M ≥ M C1 +M MC2 ; L is the length of the two preset rectangular canvases, M is the width of the two preset rectangular canvases, L C1 is the length of C1, L MC2 is the length of C2, M C1 is the width of C1, M MC2 is the width of MC2.

[0030] In this embodiment, the color of the second preset rectangular canvas is black; setting L≥L C1 +L MC2 , M ≥ M C1 +M MC2 , so that C1 and ZB2 can be completely added to the preset rectangular canvas, avoiding the situation where the size of the preset rectangular canvas is too small, resulting in the inability to completely add B1 and MC2 to the preset rectangular canvas.

[0031] S500, adding PJT to a second preset rectangular canvas; wherein PJT is a spliced ​​image generated by splicing C1 and MC2; and a horizontal boundary of C1 is parallel to a horizontal boundary of the second preset rectangular canvas.

[0032] In this embodiment, the entire PJT is added to the second preset rectangular canvas so that the PJT is in the KT, so as to facilitate the subsequent determination of the faded area.

[0033] S600: If the target boundary BJ1 of C1 is perpendicular to the X-axis, obtain the X-axis coordinate value x of BJ1 in KT. C1 ; Among them, BJ1 is the boundary of C1 that is perpendicular to the X-axis and coincides with MC2.

[0034] It can be understood that if C1 is located on the left side of MC2, the target boundary of C1 is the right boundary of C1; if C1 is located on the right side of MC2, the target boundary of C1 is the left boundary of C1.

[0035] S700: If the target boundary BJ2 of ZC2 is perpendicular to the X axis, obtain the X axis coordinate value x of BJ2 in KT. ZC2; Among them, BJ2 is the boundary of ZC2 that coincides with C1.

[0036] In this embodiment, if C1 is located on the left side of MC2, the target boundary of ZC2 is the left boundary of ZC2; if C1 is located on the right side of MC2, the target boundary of ZC2 is the right boundary of ZC2; and the target boundary of ZC2 may be perpendicular to the X-axis or may not be perpendicular to the X-axis.

[0037] S800, according to x C1 and x ZC2 , determine the faded area in MC2.

[0038] Specifically, according to x C1 and x ZC2 , determine the faded area in MC2, including the following steps:

[0039] S810, change MC2 to x ZC2 The corresponding column of pixels is determined as the starting column of pixels, and x in MC2 is C1 The corresponding column of pixels is determined as the ending column of pixels.

[0040] S820: Determine an image region formed by pixels in the start column, pixels in the end column, and all pixels between the pixels in the start column and the pixels in the end column in MC2 as a faded region.

[0041] In this embodiment, the faded area is determined according to the position coordinates of the pixels in the rectangular coordinate system, and the range of the faded area can be accurately determined to the pixel level, so that the determined faded area is more accurate. After the faded area is faded, the stitching effect of the stitched image is better.

[0042] Furthermore, after step S600 and before step S700, the method further includes the following steps:

[0043] S610: If the target boundary BJ2 of ZC2 is not perpendicular to the X-axis, obtain the coordinate value x of the first endpoint of BJ2 on the X-axis in KT. D1 And the coordinate value x of the second endpoint of BJ2 on the X axis in KT D2 ; Among them, BJ2 is the boundary of ZC2 in the vertical direction that coincides with C1.

[0044] S620, if |x D1 -x C1 |>|x D2 -x C1 |, then x in MC2 D1 The corresponding column of pixels is determined as the starting column of pixels.

[0045] S630, if |x D1 -xC1 |<|x D2 -x C1 |, then x in MC2 D2 The corresponding column of pixels is determined as the starting column of pixels.

[0046] S640, change MC2 to x C1 The corresponding column of pixels is determined as the ending column of pixels.

[0047] S650: Determine an image region formed by the pixels in the start column, the pixels in the end column, and all pixels between the pixels in the start column and the pixels in the end column in MC2 as a faded region.

[0048] In this embodiment, the target boundary of ZC2 may not be perpendicular to the X-axis. Through the above method, the faded area in MC2 can be determined when the target boundary of ZC2 is not perpendicular to the X-axis.

[0049] Furthermore, after step S500, the method further includes the following steps:

[0050] S510: If the target boundary BJ1 of C1 is not perpendicular to the X-axis and the target boundary BJ2 of ZC2 is not perpendicular to the X-axis, then obtain the coordinate value x of the first endpoint of BJ2 on the X-axis in KT. D1 And the coordinate value x of the second endpoint of BJ2 on the X axis in KT D2 , and the coordinate value x of the first endpoint of BJ1 on the X axis in KT D3 And the coordinate value x of the second endpoint of BJ1 on the X axis in KT D3 ; Among them, BJ2 is the boundary of ZC2 in the vertical direction that coincides with C1.

[0051] S520, if MAX(|x D3 |,|x D4 |)>MAX(|x D1 |,|x D2 |), and |x D1 |>|x D2 |, then x in MC2 D2 The corresponding column of pixels is determined as the starting column of pixels; otherwise, x in MC2 is D1 The corresponding column of pixels is determined as the starting column of pixels.

[0052] S530, MAX(|x D3 |,|x D4 |)>MAX(|x D1 |,|x D2 |), and |x D3 |>|x D4 |, then x in MC2 D3The corresponding column of pixels is determined as the end column of pixels; otherwise, x in MC2 is D4 The corresponding column of pixels is determined as the ending column of pixels.

[0053] S540: Determine an image region formed by pixels in the start column, pixels in the end column, and all pixels between the pixels in the start column and the pixels in the end column in MC2 as a faded region.

[0054] In this embodiment, the target boundary of C1 and the target boundary of ZC2 may not be perpendicular to the X-axis. Through the above method, the faded area in MC2 can be determined when the target boundary of C1 and the target boundary of ZC2 are not perpendicular to the X-axis.

[0055] The image fade area determination method of this embodiment obtains the initial to-be-stitched image C1 and the initial to-be-stitched image C2 corresponding to the target area, obtains the intermediate to-be-stitched image ZC2 corresponding to C2 through a preset image stitching algorithm, and adds ZC2 to the first preset rectangular canvas to obtain the target to-be-stitched image MC2 corresponding to C2; establishes a second preset rectangular canvas and a rectangular coordinate system KT corresponding to the second preset rectangular canvas according to the sizes of C1 and MC2; adds PJT to the second preset rectangular canvas, and obtains the coordinate value x of the target boundary BJ1 of C1 on the X-axis in KT C1 , if the target boundary BJ2 of ZC2 is perpendicular to the X-axis, then obtain the X-axis coordinate value x of BJ2 in KT ZC2 ; According to x C1 and x ZC2 , determine the faded area in MC2, thereby determining the faded area of ​​the image.

[0056] In addition, in the process of determining the faded area of ​​the image, the homography matrix DH2 corresponding to C2 when C1 and C2 are spliced ​​can be determined at the same time. When other images to be spliced ​​in the subsequent video stream corresponding to C2 are spliced, DH2 can be directly obtained without recalculating DH2. The time used to directly obtain the homography matrix DH2 is negligible compared to the time used to recalculate the homography matrix DH2. Therefore, the method in this embodiment can also improve the efficiency of subsequent video image splicing.

[0057] In the above embodiment, the fade area of ​​C2 is determined based on the first frame images C1 and C2 generated at the same time of two video streams covering adjacent areas of the target area, and then based on the determined fade area, for other images belonging to the same video stream as C2, for example, the image to be spliced ​​B2; B2 can directly use the fade area for fade splicing.

[0058] In this embodiment, through the method in the above embodiment, when the program is started, the faded area corresponding to each video stream can be determined according to the first frame image of each video stream obtained or the initial image to be spliced ​​obtained when the program is first started; for the subsequent multiple images to be spliced, after step S800, the method further includes the following steps:

[0059] S900, acquiring each target image to be stitched in the target area to obtain a target image sequence A corresponding to the target area = (A1, A2, ..., A i , …, A n ), i=1, 2,..., n, n≥3; among them, A i is the i-th target image to be stitched corresponding to the target area, n is the number of target images to be stitched corresponding to the target area; A r With A r+1 There are overlapping areas in the target area, r=1, 2, ..., n-1.

[0060] In this embodiment, the target images to be spliced ​​in A are arranged in order. For example, the target area is an airport. n cameras with fixed positions and shooting angles are set at the boundary of the airport. Each sub-area of ​​the airport is monitored in real time in turn, and a video stream corresponding to each sub-area can be generated. The number of frames of the image contained in the video stream generated by each camera within one second is the same. By acquiring the images generated by n cameras at the same time point, A can be obtained. It should be noted that when the sub-areas of the target area are divided, there are overlapping areas between two adjacent sub-areas. Therefore, when the sub-areas of the target area are divided, the adjacent sub-areas are divided into overlapping areas. There are also overlapping images in the images of the video streams corresponding to each other; and the distortion of the target image to be stitched arranged in the middle position in A is minimal or undistorted, for example, the image of the middle area of ​​the airport taken by a camera located in the middle of the airport; in this embodiment, the first frame image of each video stream of the target area at the same time point is used to calculate the homography matrix, and after calculating the homography matrix corresponding to the image to be stitched in the stitching process, the homography matrix can be directly applied in the subsequent image stitching process; it can be understood that in this embodiment, S100 is the stitching step after obtaining the homography matrix corresponding to the image to be stitched.

[0061] S910, if n is an odd number, obtain a first target number m=(n+1) / 2.

[0062] In this embodiment, when n is an odd number, the images to be spliced ​​in A cannot be evenly divided, and the sequence number m of the image to be spliced ​​arranged in the middle is determined.

[0063] S920, according to m, split A into a first sequence of images to be stitched AP = (A1, A2, ..., A j , …, A m), j = 1, 2, ..., m; and the second image sequence to be stitched AQ = (A m , A m+1 , …, A k , …, A n ), k=m, m+1,…,n.

[0064] Specifically, for example, n=5, that is, A includes 5 images to be stitched, and after differencing A, AP=(A1, A2, A3), AQ=(A3, A4, A5) are obtained; thus, all target images to be stitched in A are split into a first sequence of images to be stitched and a second sequence of images to be stitched.

[0065] S930, rotating each target image to be stitched in AP by 180 degrees to obtain a third sequence of images to be stitched TA=(TA1, TA2, . . . , TA j ,…,TA m ), where TA j To A j The image to be stitched is obtained after rotating 180 degrees.

[0066] In this embodiment, A m is the target image to be stitched in the middle position in A. m The distortion is minimal or no distortion; the existing image stitching algorithm can be used for stitching, for example, SIFT algorithm; when the algorithm is used to stitch two images, the algorithm will determine the first input image as the main image to be stitched, and the second input image as the slave image to be stitched, and the slave image to be stitched will be converted according to the main image to be stitched; based on this, it is necessary to m As the first image, each image to be stitched in AP is first rotated 180 degrees.

[0067] S940, invert the arrangement order of all the images to be stitched in TA to obtain a fourth image sequence to be stitched TA′=(TA m , T.A. m-1 ,…,TA j ,…,TA1).

[0068] In this embodiment, when n is an odd number, m may be an odd number or an even number; reversing the arrangement order of all images to be spliced ​​in TA includes the following steps:

[0069] S941, if m is an odd number, then replace TA r With TA m-r+1 The arrangement positions are interchanged, r = 1, 2, ..., m; TA b The position remains unchanged, b = (m+1) / 2; to obtain TA'.

[0070] S942, if m is an even number, then replace TA r With TA m-r+1 The positions are interchanged to obtain TA'.

[0071] TA' is obtained by rotating the images to be stitched in AP and reversing their arrangement order, and the two images with overlapping areas in TA' are still arranged adjacent to each other, and the image with the smallest distortion or no distortion is arranged in the first place. In the subsequent stitching process, it is only necessary to stitch the images to be stitched in the arrangement order of TA', which makes the subsequent stitching more convenient.

[0072] S950, obtaining a first designated image to be stitched QT=TA m , the second designated image to be stitched QY = A m And the first target value N=1.

[0073] S960, using SIFT algorithm, QT is determined as the main image to be stitched and TA m-N Stitching is performed, and QY is determined as the main image to be stitched and A m+N Splice to get QT and TA m-N The stitched image HA after stitching N As well as QY and A m+N The stitched image after GA N .

[0074] S970, obtain QT = HA N and QY=GA N .

[0075] S980, if N<m-1, obtain N=N+1 and go to step S960; otherwise, HA N Rotate 180 degrees to get HA N The corresponding rotated image XA N .

[0076] In this embodiment, TA in TA' m Determine the main image to be stitched and TA m-1 The obtained stitching image HA1 is determined as the main stitching image and then combined with TA m-2 The obtained stitching image HA2 is determined as the main stitching image and then combined with TA m-3 The stitching is repeated until N≥m-1; the stitching image obtained by the last stitching is determined as the stitching image corresponding to TA'.

[0077] In this embodiment, it should be noted that when stitching the images to be stitched in TA' and AQ, two threads are called at the same time, one thread is responsible for the stitching processing of the images to be stitched in TA', and the other thread is responsible for the stitching processing of the images to be stitched in AQ; therefore, compared with the sequential stitching processing of the images to be stitched in A, the method implemented in this embodiment can reduce the stitching time by half; thereby improving the stitching efficiency when stitching images.

[0078] Further, after step S940 and before step S950, the method further includes the following steps:

[0079] S901, obtain the preset homography matrix list DL corresponding to TA', the preset homography matrix list DQ corresponding to AQ, and GA N The corresponding preset homography matrix DGA N ; Where, DL = (DL m-1 , DL m-2 ,…,DL p ,..., DL1), P=1, 2,..., m-1; DL p For TA p The corresponding homography matrix; DQ = (DQ m+1 , DQ m+2 , …, DQ q , …, DQ n ), q=m+1, m+2,…,n; DQ q A q The corresponding homography matrix; DL and DQ are obtained by splicing the first frame image of each video stream of the target area at the same time.

[0080] In this embodiment, the whole image stitching process is divided into two stages. The first stage includes the process of obtaining the homography matrix corresponding to the image to be stitched. The first stage can be understood as when the program is started, including the following steps:

[0081] S902, obtaining the first frame of image generated at the same time point of each video stream of the target area, and determining it as the initial image to be spliced, so as to obtain the initial image sequence to be spliced ​​CS = (CS1, CS2, ..., CS i ,…,CS n );where CS i A i The first frame image of the corresponding video stream.

[0082] S903, if n is an odd number and n>1, obtain the target number m=(n+1) / 2.

[0083] S904, according to m, split CS into a ninth sequence of images to be spliced ​​CSP = (CS1, CS2, ..., CSj ,…,CS m ); and the tenth image sequence to be stitched CSQ = (CS m , C.S. m+1 ,…,CS k ,…,CS n ).

[0084] S905, rotating each of the initial images to be stitched in the CSP by 180 degrees to obtain an eleventh sequence of images to be stitched TCS=(TCS1, TCS2, . . . , TCS j ,…,TCS m );TCS j For CS j The image to be stitched after being rotated 180 degrees.

[0085] S906, invert the arrangement order of all the images to be spliced ​​in TCS to obtain a twelfth image sequence to be spliced ​​TCS'=(TCS m , TCS m-1 ,…,TCS j ,…,TCS1).

[0086] S907, obtaining the fifth designated image to be stitched QT'=TCS m , the sixth designated image to be stitched QY'=CS m and the third target value N1=1;

[0087] S908, using SIFT algorithm, determine QT' as the main image to be stitched and TCS m-1 Stitching is performed, and QY' is determined as the main image to be stitched and CS m+1 Splice to obtain QT' and TCS m-1 The stitched image KA after stitching N As well as QY' and CS m+1 The stitched image VA after stitching N ; and QT' and TCS m-1 TCS during splicing m-1 The corresponding homography matrix is ​​determined as DL m-1 And QY' and CS m+1 CS during splicing m+1 The corresponding homography matrix is ​​determined as DQ m+1 ;

[0088] In this embodiment, the SIFT algorithm is used for each stitching, and the algorithm can automatically calculate the homography matrix of the image to be stitched corresponding to the main image to be stitched. Therefore, two homography matrices can be obtained each time step S017 is executed.

[0089] S909, obtain QT'=KAN and QY'=VA N ;

[0090] S9091, if N1<m-1, obtain N1=N1+1 and go to step S017; otherwise, set KA N Rotate 180 degrees to get KA N The corresponding rotated image XKA N ;

[0091] S9092, XKA N and VA N Stitching is performed to obtain a panoramic image of the target area and the VA N The corresponding homography matrix is ​​determined as DGA N .

[0092] In this embodiment, after the homography matrix list DL and the homography matrix list DQ are determined by the SIFT algorithm and CS, DL and DQ are stored in a preset storage space. When image stitching is performed on the next acquired image sequence to be stitched, there is no need to calculate the homography matrix of each image to be stitched again, and the corresponding homography matrix can be directly obtained from the preset storage space. Since the calculation of the homography matrix will take up a lot of computing power, the homography is calculated only once in this embodiment. Therefore, the efficiency of image stitching can be greatly improved, thereby improving the smoothness of the stitched video image.

[0093] Specifically, stitching the images to be stitched in TA' and AQ includes the following steps:

[0094] S961, get TA from DL m-N The corresponding homography matrix DL m-N , and get A from DQ m+N The corresponding homography matrix DQ m+N .

[0095] S962, TA m-N With DL m-N Multiply and add A m+N With DQ m+N Multiply to get TA m-N The corresponding intermediate image to be stitched ZA m-N and A m+N The corresponding intermediate image to be stitched Z m+N .

[0096] In this embodiment, it is possible to m-N Corresponding to the number mN, get the homography matrix DL numbered mN from DL m-N ; Similarly, A can be obtained from DQ m+NThe corresponding homography matrix DQ m+N .

[0097] The time taken to directly obtain the homography matrix is ​​negligible compared to the time taken to recalculate the homography matrix. Therefore, directly obtaining the homography matrix in this embodiment can greatly improve the efficiency of image stitching.

[0098] S963, using SIFT algorithm, QT is determined as the main image to be stitched and ZA m-N Stitching is performed, and QY is determined as the main image to be stitched and Z m+N Splicing to obtain HA N and GA N .

[0099] Understandably, DL m-N It is to determine QT' as the main image to be stitched and TCS m- The homography matrix obtained during stitching, TA m-N and TCS m-N belong to the same video stream, so TA m-N With DL m-N The multiplication results in ZA m-N is matched with QT'; similarly, Z m+N It matches QY'.

[0100] It should be noted that when the SIFT algorithm is used for stitching, the SIFT algorithm does not perform the process of calculating the homography matrix, but only uses the stitching algorithm in the SIFT algorithm; therefore, the stitching efficiency can be greatly improved.

[0101] S990, XA N and GA N Stitching is performed to obtain a panoramic image of the target area.

[0102] Specifically, XA N and GA N The stitching process includes the following steps:

[0103] S9910, will GA N Multiply it by DGA to get GA N The corresponding intermediate image to be stitched ZGA N .

[0104] In this embodiment, ZGA N That is, the DGA determined in the above embodiment N ; You can directly obtain GA N The corresponding intermediate image to be stitched ZGA N, there is no need to calculate the homography matrix of each image to be stitched again, therefore, the stitching efficiency of video images can be improved.

[0105] S9920, through SIFT algorithm, XA N Determine the main image to be stitched and ZGA N Stitching to obtain a panoramic image of the target area.

[0106] The video image stitching method of this embodiment obtains each target image to be stitched in the target area to obtain an initial image sequence A to be stitched corresponding to the target area. If the number of images to be stitched in A is an odd number, A is split into a first image sequence AP and a second image sequence AQ to be stitched. Then, after each image to be stitched in AP is rotated 180 degrees, the arrangement order is inverted to obtain TA'. In this way, the image with the smallest distortion in AP and arranged at the end can be adjusted to the first position, and the image is used as the main image to be stitched and stitched with the adjacent images to be stitched, so that the adjacent images to be stitched of the main image to be stitched can be The transformation is performed based on the main image to be stitched, so as to obtain a transformed image to be stitched that is adapted to the main image to be stitched and has a smaller distortion, so that the stitched image obtained by stitching all the images to be stitched in AP has the smallest distortion; and the image to be stitched with the smallest distortion in AQ is ranked first, and there is no need to adjust the arrangement order of the images to be stitched in AQ. The same stitching method as AP is adopted, and then the stitched image with the smallest distortion corresponding to AQ is obtained; the panoramic image of the target area obtained by stitching the stitched images with the smallest distortion also has the smallest distortion; therefore, the present invention can reduce the distortion of the panoramic image after stitching.

[0107] In addition, the present invention splits A into AP and AQ. Therefore, in the subsequent stitching process, the images to be stitched in AP and the images to be stitched in AP can be stitched simultaneously. Compared with the sequential stitching of the images to be stitched in A, the present invention can reduce the stitching time by half; therefore, the present invention can also improve the stitching efficiency when stitching images.

[0108] Furthermore, after step S900, the method further includes the following steps:

[0109] S921, if n is an even number, obtain the second target number m1=n / 2.

[0110] S922, according to m1, split A into a fifth sequence of images to be stitched BP = (A1, A2, ..., A j1 , …, A m1 ), j1=1, 2, ..., m1; and the sixth image sequence to be stitched BQ=(A m1+1 , A m1+2 , …, A k1, …, A n ), k1=m+1, m+2,...,n.

[0111] S923, rotating each image to be stitched in BP by 180 degrees to obtain a seventh image sequence to be stitched TB=(TB1, TB2, . . . , TB j1 , …, TB m1 ), where TB j1 To A j1 The image to be stitched is obtained after rotating 180 degrees.

[0112] S924, invert the arrangement order of all the images to be spliced ​​in TB to obtain an eighth image sequence to be spliced ​​TB′=(TB m1 , TB m1-1 , …, TB j1 ,…,TB1).

[0113] S925, obtaining the third designated image to be stitched UT=TB m1 , the fourth designated image to be stitched UY = A m1+1 And the second target value Q=1.

[0114] S926, using SIFT algorithm, determine UT as the main image to be stitched and TB m1-Q Stitching is performed, and UY is determined as the main image to be stitched and A m1+Q+1 Splice to get UT and TB m1-Q The stitched image HB after stitching Q As well as UY and A m1+N+1 The stitched image GB after stitching Q .

[0115] S927, if Q<m1-1, obtain Q=Q+1 and go to step S260; otherwise, set HB Q Rotate 180 degrees to get HB Q The corresponding rotated image XB Q .

[0116] S928, XB Q and GB Q Stitching is performed to obtain a panoramic image of the target area.

[0117] In this embodiment, when n is an even number, the two target images to be stitched arranged in the middle position in A have minimal distortion or no distortion, A is split into BP and BQ, and the processing process of the images to be stitched in BP is the same as the processing process of the images to be stitched in AP and the subsequent stitching process, which are not repeated here; thereby, the stitching of an even number of images to be stitched can be achieved, and the obtained panoramic image of the target area has less distortion.

[0118] Further, step S926 includes the following steps:

[0119] S9261, Get TB m1-Q The corresponding homography matrix DL' m1-Q and A m1+Q+1 The corresponding homography matrix DQ' m1+Q+1 .

[0120] S9262, TB m1-Q With DL' m1-Q Multiply and add A m1+Q+1 With DQ' m1+Q+1 Multiply to get TB m1-Q The corresponding intermediate image to be stitched ZB m1-Q and A m1+Q+1 The corresponding intermediate image to be stitched Z m1+Q+1 .

[0121] S9263, through the SIFT algorithm, UT is determined as the main image to be stitched and ZB m1-Q Stitching is performed, and UY is determined as the main image to be stitched and Z m1+Q+1 Splice to get HB Q and GB Q .

[0122] In this embodiment, when n is an even number, the same method as that for obtaining the homography matrix when n is an odd number can be used to obtain the corresponding homography matrix list. For the stitching of the images to be stitched in BP and BQ, the homography matrix corresponding to the images to be stitched is also obtained first; therefore, when n is an even number, there is no need to calculate the homography matrix of each image to be stitched again, and the corresponding homography matrix can be directly obtained from the preset storage space; since the calculation of the homography matrix will take up a lot of computing power, the homography is calculated only once in this embodiment, and therefore, the efficiency of image stitching can be greatly improved, thereby improving the smoothness of the stitched video image.

[0123] Further, step S928 includes the following steps:

[0124] S9281, Get GB Q The corresponding homography matrix DGB Q .

[0125] S9282, GB Q With DGB Q Multiply to get GB Q The corresponding intermediate image to be stitched ZGB Q .

[0126] S9283, through SIFT algorithm, XB QDetermine the main image to be stitched and ZGB Q Stitching to obtain a panoramic image of the target area.

[0127] In this embodiment, GB Q The corresponding homography matrix DGB Q DGA N , you can directly get DGA from the preset storage space N , so there is no need to calculate GB again Q The corresponding homography matrix is ​​obtained, thereby improving the splicing efficiency of video images when n is an even number.

[0128] In an exemplary embodiment, a method for fading and splicing video images is provided, the method comprising the following steps:

[0129] S1, obtain two adjacent target images to be stitched from A, and determine them as target image to be stitched B1 and target image to be stitched B2 respectively; wherein B1 and B2 are two target images to be stitched among n target images to be stitched corresponding to the target area, and there is an overlapping area between B1 and B2 in the target area; B1 and B2 are obtained based on non-first frame images generated at the same time of two video streams covering adjacent areas of the target area.

[0130] In this embodiment, for example, the target area is an airport, and n cameras with fixed positions and shooting angles are set at the boundary of the airport. Each sub-area of ​​the airport is monitored in real time in turn, and a video stream corresponding to each sub-area can be generated. The number of frames of the image contained in the video stream generated by each camera within one second is the same. The images generated by n cameras at the same time point are obtained, and n target images to be stitched corresponding to the target area are obtained; it should be noted that when the sub-areas of the target area are divided, there are overlapping areas between two adjacent sub-areas. Therefore, there are also overlapping images in the images of the video streams corresponding to the two adjacent sub-areas, that is, B1 and B2 have overlapping areas in the target area; in this embodiment, the first frame image of each video stream of the target area at the same time point is used to calculate the homography matrix. After calculating the homography matrix corresponding to the image to be stitched in the stitching process, the homography matrix can be directly applied in the subsequent image stitching process; it can be understood that, in this embodiment, S1 is the stitching step after obtaining the homography matrix corresponding to the image to be stitched.

[0131] S2, obtain the homography matrix DH2 corresponding to B2 from the preset storage space.

[0132] In this embodiment, the whole image stitching process is divided into two stages. The first stage includes the process of obtaining the homography matrix corresponding to the image to be stitched. The first stage can be understood as when the program is started, the generation of the homography matrix DH2 includes the following steps:

[0133] S21, obtaining initial images C1 and C2 to be stitched corresponding to the target area; wherein C1 and C2 have overlapping areas in the target area; C1 and C2 are obtained according to the first frame images generated at the same time of two video streams covering adjacent areas of the target area; B1 and C1 belong to the same video stream, and B2 and C2 belong to the same video stream.

[0134] S22, stitching C1 and C2 using a SIFT algorithm to obtain a stitched image PJT after stitching C1 and C2.

[0135] S23, in the process of splicing C1 and C2 to obtain PJT, the homography matrix corresponding to C2 is determined as DH2.

[0136] In this embodiment, C1 and C2 are the initial images to be stitched that are acquired for the first time. When stitching C1 and C2, the SIFT algorithm is used. The algorithm can automatically calculate the homography matrix DH2 corresponding to C2 with C1 as the main image to be stitched. In the subsequent stitching process, the homography matrices corresponding to other images to be stitched that belong to the same video stream as C2 are all DH2. Therefore, the determined DH2 can be stored in a preset storage space. When stitching other images to be stitched that belong to the same video stream as C2, DH2 can be directly obtained. Since the calculation of the homography matrix will take up a lot of computing power, the homography is calculated only once in this embodiment. Therefore, the efficiency of image stitching can be greatly improved, thereby improving the smoothness of the stitched video image.

[0137] S3, multiply B2 and DH2 to obtain the intermediate image to be stitched ZB2 corresponding to B2.

[0138] In this embodiment, an existing image stitching algorithm is used to stitch two images to be stitched; for example, the SIFT algorithm; when the algorithm is used to stitch the two images to be stitched, the algorithm determines the image to be stitched that is input first as the main image to be stitched, and the image to be stitched that is input later as the slave image to be stitched, and the slave image to be stitched will be converted according to the main image to be stitched, that is, B2 is multiplied by DH2 to adapt to B1; it should be noted that the distortion of B1 is smaller than the distortion of B2, thereby making the distortion of the stitched image obtained by stitching B1 and B2 smaller; after B2 is multiplied by DH2, the effective image of B2 is distorted, so that B2 is no longer a rectangle, and the image obtained by multiplying B2 by DH2 can be added to the preset black rectangular canvas to obtain ZB2; that is, there will be a certain area of ​​black image in ZB2.

[0139] S4, obtain the sub-area Y in ZB2 that overlaps with B1 B2 .

[0140] In this embodiment, it should be noted that those skilled in the art can use existing methods for determining overlapping regions of images to determine the sub-region Y in ZB2 that overlaps with B1 as needed. B2 , I will not elaborate here; it is understandable that Y B2 This is also an area that needs to be desalinated.

[0141] S5, obtaining a preset matrix FG from a preset storage space; wherein FG is obtained according to the first frame image generated at the same time by two video streams covering adjacent areas of the target area.

[0142] In this embodiment, FG is used to B2 The matrix for fading each pixel in FG, the number of rows in FG and Y B2 The number of rows of pixels in FG is the same as that in Y B2 The number of columns of pixels inside is the same, and the values ​​of each row in each column in FG are the same; and FG is obtained based on the first frame image generated at the same time of two video streams covering adjacent areas of the target area, that is, FG is determined in the first stage and is also stored in the preset storage space, and can be directly obtained later.

[0143] S6, according to FG, for Y B2 Each pixel in ZB2 is faded to obtain the target image K to be stitched corresponding to ZB2. B2 .

[0144] Specifically, Y B2 The RGB value of each pixel in is multiplied by the corresponding value in FG to obtain the target image K corresponding to ZB2 B2 .

[0145] S7, adding B1 to the first preset position in the preset rectangular canvas; wherein L≥L B1 +L B2 , M ≥ M B1 +M B2 ; L is the length of the rectangular canvas, M is the width of the rectangular canvas, L B1 is the length of B1, L B2 is the length of ZB2, M B1 is the width of B1, M B2 is the width of ZB2; the color of the rectangular canvas is the preset color.

[0146] In this embodiment, the preset color is black; set L≥L B1 +L B2 , M ≥ M B1 +M B2, so that B1 and ZB2 can be completely added to the preset rectangular canvas, avoiding the situation where the size of the preset rectangular canvas is too small, resulting in the inability to completely add B1 and ZB2 to the preset rectangular canvas.

[0147] S8, K B2 Adds a second preset position to the rectangular canvas.

[0148] S9, move the Y B2 The two overlapping pixel points in are superimposed respectively to obtain the stitching image after B1 and B2 are stitched together.

[0149] The video image fading and splicing method of this embodiment obtains the images B1 and B2 to be spliced ​​corresponding to the target area, and adds B1 to the first preset position in the preset rectangular canvas; obtains the homography matrix DH2 corresponding to B2, multiplies B2 by DH2, and obtains the intermediate image ZB2 to be spliced ​​corresponding to B2; at this time, obtains the sub-area Y in ZB2 that overlaps with B1 B2 And the preset matrix FG, for Y B2 Each pixel in ZB2 is faded to obtain the target image K to be stitched corresponding to ZB2. B2 ; K B2 Add to the second preset position of the rectangular canvas; superimpose the two overlapping pixel points in the rectangular canvas respectively to obtain the stitching image after B1 and B2 are stitched together; due to Y B2 Each pixel in has been faded, so the Y B2 The region has a fading effect, thereby weakening the more obvious stitching area in the panoramic image of the target area.

[0150] In addition, in this embodiment, the homography matrix and the preset matrix FG are both predetermined. When the homography matrix or FG is needed in the step, it can be directly obtained from the preset storage space without recalculating the homography matrix and FG. The time used to directly obtain the homography matrix and FG is negligible compared to the time used to recalculate the homography matrix and FG; therefore, the present invention can also improve the efficiency of video image fading and splicing; in this embodiment, the fading of a sub-area of ​​one of the two images to be spliced ​​is completed before the images are spliced; while in the prior art, the fading process is performed after the image splicing is completed, and it is necessary to comprehensively calculate the brightness of the spliced ​​image and the fading range; therefore, the present invention occupies less computing power when fading and splicing video images.

[0151] Further, the first preset position and the second preset position are determined by the following steps:

[0152] S020, add PJT to the preset rectangular canvas;

[0153] S021, obtaining the position coordinate F1 of the first preset pixel of C1 in the coordinate system corresponding to the rectangular canvas C1 =(x F1 ,y F1 ) and the position coordinate F2 of the second preset pixel in the coordinate system corresponding to the rectangular canvas C1 =(x F2 ,y F2 ), where x F1 is the coordinate value of the first preset pixel of C1 on the x-axis in the coordinate system corresponding to the rectangular canvas, y F1 is the coordinate value of the first preset pixel of C1 on the Y axis in the coordinate system corresponding to the rectangular canvas, x F2 is the coordinate value of the second preset pixel of C1 on the x-axis in the coordinate system corresponding to the rectangular canvas, y F2 It is the coordinate value of the second preset pixel of C1 on the Y-axis in the coordinate system corresponding to the rectangular canvas; the first preset pixel of C1 is any pixel among the pixels corresponding to the four vertices of C1, and the second preset pixel of C1 is any pixel among the pixels corresponding to the four vertices of C1 except the first preset pixel.

[0154] S022, obtaining the position coordinate F3 of the third preset pixel of C3 in the coordinate system corresponding to the rectangular canvas C3 =(x F3 ,y F3 ) and the position coordinate F4 of the fourth preset pixel in the coordinate system corresponding to the rectangular canvas C3 =(x F4 ,y F4 ), where C3 is the image to be stitched after C2 is converted by DH2; x F3 is the coordinate value of the third preset pixel of C3 on the x-axis in the coordinate system corresponding to the rectangular canvas, y F3 is the coordinate value of the third preset pixel of C3 on the Y axis in the coordinate system corresponding to the rectangular canvas, x F4 is the coordinate value of the fourth preset pixel of C3 on the x-axis in the coordinate system corresponding to the rectangular canvas, y F4 It is the coordinate value of the fourth preset pixel of C3 on the Y-axis in the coordinate system corresponding to the rectangular canvas; the third preset pixel of C3 is any pixel among the pixels corresponding to the four vertices of C3, and the fourth preset pixel of C3 is any pixel among the pixels corresponding to the four vertices of C3 except the third preset pixel.

[0155] S023, according to F1 C1 and F2 C1 , determine the first preset position WZ1 in the rectangular canvas = (F1 C1 , F2 C1 ), and according to F3 C3and F4 C3 , determine the second preset position WZ2 in the rectangular canvas = (F3 C3 , F4 C3 ).

[0156] In this embodiment, after obtaining PJT, the entire PJT is added to the preset rectangular canvas. The position of C1 in the rectangular canvas can be accurately determined by WZ1, that is, the position of C1 is determined by two fixed pixels in C1, and the other pixels in C1 are related to F1. C1 and F2 C1 They have a fixed relative position relationship, so the position of each pixel in C1 in the rectangular canvas can be determined; C3 is the same as C1 and will not be elaborated here.

[0157] Furthermore, the preset matrix FG is generated by the following steps:

[0158] S51, get Y C3 The number of rows of pixels in Y, the number of pixel columns num1 corresponding to at least one pixel point located at the left vertical boundary, and the number of pixel columns num2 corresponding to at least one pixel point located at the right vertical boundary; C3 It is the sub-region in C3 that overlaps with C1.

[0159] S52, obtain the target row pixel number QW=|num1-num2|+1.

[0160] In this embodiment, Y C3 May be irregular shape, FG needs to cover Y C3 Therefore, it is necessary to determine Y C3 The maximum number of columns occupied by the inner pixel; determine the Y C3 The left boundary and the right boundary can determine the number of columns corresponding to the pixels on the left boundary and the number of columns corresponding to the pixels on the right boundary. For example, the number of columns corresponding to the pixels on the left boundary is 1, the number of columns corresponding to the pixels on the right boundary is 100, and the target row pixel number QW is 100.

[0161] S53, according to b and QW, establish FG; wherein FG includes b rows and QW columns, and any two values ​​in each column are equal; the value α corresponding to the j1th column in FG j1 =j1 / QW, j1=1, 2,..., QW.

[0162] In this embodiment, the value corresponding to each column in FG is set to decrease in sequence, thereby obtaining FG.

[0163] Furthermore, Y B2 The number of rows of pixels in Y C2 The number of rows of pixels in the same B2 The number of columns of pixels in YC2 The number of columns of pixels in the image is the same, step S6 comprises the following steps:

[0164] S61, get Y B2 The RGB value of each pixel in the B2 The corresponding initial RGB value list set RY B2 =(R B2 1，1 , R B2 1，2 , …, R B2 1，j1 , …, R B2 1，QW , …, R B2 a，1 , R B2 a，2 , …, R B2 a，j1 , …, R B2 a，QW , …, R B2 b，1 , R B2 b，2 , …, R B2 b，j1 , …, R B2 b，QW ), where R B2 a，j1 Y B2 The initial RGB value list corresponding to the pixel in row a and column j1 in R B2 a，j1 =(R1 a，j1 , G1 a，j1 , B1 a，j1 ); R1 a，j1 Y B2 The value corresponding to the red color of the pixel in the ath row and j1th column in G1 a，j1 Y B2 The green value of the pixel in the ath row and j1th column in B1 a，j1 Y B2 The value corresponding to the blue color of the pixel at the ath row and j1th column in .

[0165] S62, according to FG, Y B2 The value corresponding to the red color of the pixel in the ath row and j1th column is adjusted to Y B2 The green value of the pixel in the ath row and j1th column is adjusted to Y B2 The value corresponding to the blue color of the pixel in the ath row and j1th column is adjusted to To get K B2 ; is the default floor function.

[0166] In this embodiment, the value corresponding to each column in FG decreases in the range of 0-1, so that Y B2 The RGB values ​​of the pixels in each column decrease in turn, so that Y B2 Gradient transparency effect.

[0167] Further, step S9 includes the following steps:

[0168] S91, in the rectangular canvas, traverse all pixels in B1 and convert any pixel in B1 that belongs to K B2 The pixels are determined as pixels to be superimposed.

[0169] S92, obtaining the initial RGB value of each pixel to be superimposed, so as to obtain a list set TY of initial RGB values ​​of the pixels to be superimposed corresponding to B1 B1 =(R B1 1，1 , R B1 1，2 , …, R B1 1，j1 , …, R B1 1，QW , …, R B1 a，1 , R B1 a，2 , …, R B1 a，j1 , …, R B1 a，QW , …, R B1 b，1 , R B1 b，2 , …, R B1 b，j1 , …, R B1 b，QW ), where R B1 a，j1 is the initial RGB value list corresponding to the pixel in row a and column j1 in B1; R B1 a，j1 =(R2 a，j1 , G2 a，j1 , B2 a，j1 ); R2 a，j1 For B1 and K B2 The pixel XS at row a and column j1 in the overlapping area a，j1 The red color corresponds to the initial value, G2 a，j1 For XS a，j1 The green color corresponds to the initial value, B2 a，j1 For XS a，j1 The blue color corresponds to the initial value.

[0170] S93, according to TY B1 And RY B2 , determine the intermediate RGB value of each pixel to be superimposed to obtain the intermediate RGB value list set ZY corresponding to B1 B1 =(ZR B1 1，1 , ZR B1 1，2 , …, ZR B1 1，j1 , …, ZR B1 1，QW , …, ZR B1 a，1 , ZR B1 a，2 , …, ZR B1 a，j1 , …, ZR B1 a，QW , …, ZR B1 b，1 , ZR B1 b，2 , …, ZR B1 b，j1 , …, ZR B1 b，QW ), where ZR B1 a，j1 For R B1 a，j1 The corresponding intermediate RGB value list; ZR B1 a，j1 =(ZR2 a，j1 , ZG2 a，j1 , ZB2 a，j1 ); ZR2 a，j1 For XS a，j1 The red color corresponds to the middle value, ZG2 a，j1 For XS a，j1 The green color corresponds to the middle value, ZB2 a，j1 For XS a，j1 The blue color corresponds to the middle value;

[0171] S94, traverse ZY B1 , if ZR2 a，j1 >255, then MR2 a，j1 =255; otherwise, MR2 a，j1 =ZR2 a，j1 ; If ZG2 a，j1 >255, then MG2 a，j1 =255; otherwise, MG2 a，j1 =ZG2 a，j1 ; if ZB2 a，j1>255, then MB2 a，j1 =255; otherwise, MB2 a，j1 =ZB2 a，j1 ; To obtain the target RGB value list set MY corresponding to B1 B1 =(MR B1 1，1 , MR B1 1，2 ,…,MR B1 1，j1 ,…,MR B1 1，QW ,…,MR B1 a，1 , MR B1 a，2 ,…,MR B1 a，j1 ,…,MR B1 a，QW ,…,MR B1 b，1 , MR B1 b，2 ,…,MR B1 b，j1 ,…,MR B1 b，QW ), where MR B1 a，j1 For R B1 a，j1 The corresponding target RGB value list; MR B1 a，j1 =(MR2 a，j1 , MG2 a，j1 , MB2 a，j1 );MR2 a，j1 For XS a，j1 The red color corresponds to the target value, MG2 a，j1 For XS a，j1 The green color corresponds to the target value, MB2 a，j1 For XS a，j1 The blue color corresponds to the target value.

[0172] In this embodiment, ZR2 a，j1 、ZG2 a，j1 and ZB2 a，j1 The maximum value is limited to 255 to avoid errors in the superimposed pixels.

[0173] S95, traverse all pixels in B1 and update R2 a，j1 =MR2 a，j1 , G2 a，j1 =MG2 a，j1 , B2 a，j1 =MB2 a，j1 .

[0174] In this embodiment, all pixels in B1 that exist in Y B2 After the RGB value of each pixel in is updated, the fading transition of the stitched image can be achieved, thereby weakening the more obvious stitching area in the panoramic image.

[0175] Further, step S7 includes the following steps:

[0176] S71, adding B1 to the preset rectangular canvas.

[0177] S72, adjust the position of B1 in the rectangular canvas so that the first preset pixel of B1 is at F1 C1 The corresponding position and the second preset pixel of B1 is at F2 C1 Corresponding positions; wherein, the relative position of the first preset pixel of B1 relative to the central pixel of B1 is the same as the relative position of the first preset pixel of C1 relative to the central pixel of C1, and the relative position of the second preset pixel of B1 relative to the central pixel of B1 is the same as the relative position of the second preset pixel of C1 relative to the central pixel of C1.

[0178] In this embodiment, the first preset pixel and the second preset pixel are set so that the position of B1 in the rectangular canvas is the same as the position of C1 in the rectangular canvas, thereby ensuring the accuracy of subsequent stitching.

[0179] Further, step S8 includes the following steps:

[0180] S81, K B2 Add to the preset rectangular canvas;

[0181] S82, adjust K B2 The position within the rectangular canvas is such that K B2 The third preset pixel is at F3 C3 The corresponding position and K B2 The fourth preset pixel is at F4 C3 The corresponding position; among them, K B2 The third preset pixel is relative to K B2 The relative position of the central pixel of C3 is the same as the relative position of the third preset pixel of C3 relative to the central pixel of C3. B2 The relative position of the fourth preset pixel of C3 relative to the central pixel of C3 is the same as the relative position of the fourth preset pixel of C3 relative to the central pixel of C3.

[0182] In this embodiment, the first preset pixel and the second preset pixel are set so that K B2 The position of C3 in the rectangular canvas is the same as that of K. B2Accuracy of splicing with B1.

[0183] In this embodiment, all the moving standard aircraft information is acquired, and when the first target information is matched with the moving aircraft image, it is only necessary to match within the moving standard aircraft information list; on the one hand, it can reduce the amount of calculation during matching, and on the other hand, it can avoid matching the static standard aircraft information to the moving aircraft image, thereby improving the accuracy of the moving aircraft image information matching.

[0184] In addition, although the steps of the method in the present disclosure are described in a specific order in the drawings, this does not require or imply that the steps must be performed in this specific order, or that all the steps shown must be performed to achieve the desired results. Additionally or alternatively, some steps may be omitted, multiple steps may be combined into one step, and / or one step may be decomposed into multiple steps, etc.

[0185] An embodiment of the present invention also provides a non-transitory computer-readable storage medium, which can be set in an electronic device to store at least one instruction or at least one program related to implementing a method in a method embodiment. The at least one instruction or the at least one program is loaded and executed by the processor to implement the method provided in the above embodiment.

[0186] The program product may use any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. The readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples (non-exhaustive list) of readable storage media include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0187] Computer readable signal media may include data signals propagated in baseband or as part of a carrier wave, in which readable program code is carried. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Readable signal media may also be any readable medium other than a readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0188] The program code embodied on the readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wired, optical cable, RF, etc., or any suitable combination of the foregoing.

[0189] Program code for performing the operations of the present application may be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java, C++, etc., and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as a separate software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server. In the case of a remote computing device, the remote computing device may be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computing device (e.g., using an Internet service provider to connect through the Internet).

[0190] An embodiment of the present invention further provides an electronic device, comprising a processor and the aforementioned non-transitory computer-readable storage medium.

[0191] The electronic device according to this embodiment of the present application is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0192] The electronic device is presented in the form of a general-purpose computing device. The components of the electronic device may include, but are not limited to: the at least one processor mentioned above, the at least one storage device mentioned above, and a bus connecting different system components (including storage devices and processors).

[0193] The storage stores program codes, which can be executed by the processor, so that the processor executes the steps described in the above “exemplary method” section of this specification according to various exemplary embodiments of the present application.

[0194] The memory may include readable media in the form of volatile memory, such as random access memory (RAM) and / or cache memory, and may further include read only memory (ROM).

[0195] The storage may also include a program / utility having a set (at least one) of program modules, such program modules including but not limited to: an operating system, one or more application programs, other program modules and program data, each of which or some combination may include the implementation of a network environment.

[0196] The bus may represent one or more of several types of bus structures including a memory bus or memory controller, a peripheral bus, an accelerated graphics port, a processor, or a local bus using any of a variety of bus architectures.

[0197] The electronic device may also communicate with one or more external devices (e.g., keyboards, pointing devices, Bluetooth devices, etc.), may communicate with one or more devices that enable a user to interact with the electronic device, and / or may communicate with any device (e.g., routers, modems, etc.) that enables the electronic device to communicate with one or more other computing devices. Such communication may be performed through an input / output (I / O) interface. Furthermore, the electronic device may also communicate with one or more networks (e.g., local area networks (LANs), wide area networks (WANs), and / or public networks, such as the Internet) through a network adapter. The network adapter communicates with other modules of the electronic device through a bus. It should be understood that, although not shown in the figure, other hardware and / or software modules may be used in conjunction with the electronic device, including but not limited to: microcode, device drivers, redundant processors, external disk drive arrays, RAID systems, tape drives, and data backup storage systems, etc.

[0198] Through the description of the above implementation, it is easy for those skilled in the art to understand that the example implementation described here can be implemented by software, or by software combined with necessary hardware. Therefore, the technical solution according to the implementation of the present disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, including several instructions to enable a computing device (which can be a personal computer, a server, a terminal device, or a network device, etc.) to execute the method according to the implementation of the present disclosure.

[0199] An embodiment of the present invention further provides a computer program product, which includes program code. When the program product is run on an electronic device, the program code is used to enable the electronic device to execute the steps of the method according to various exemplary embodiments of the present invention described above in this specification.

[0200] Although some specific embodiments of the present invention have been described in detail by way of example, it will be appreciated by those skilled in the art that the above examples are for illustration only and are not intended to limit the scope of the present invention. It will also be appreciated by those skilled in the art that various modifications may be made to the embodiments without departing from the scope and spirit of the present invention. The scope of the present invention is defined by the appended claims.

Claims

1. A method for determining a faded area of ​​a spliced ​​image, characterized in that: The method comprises the following steps: S100, obtaining an initial image to be stitched C1 and an initial image to be stitched C2 corresponding to the target area; wherein C1 and C2 are two initial images to be stitched among a plurality of initial images to be stitched corresponding to the target area, and C1 and C2 have an overlapping area in the target area; C1 and C2 are obtained based on the first frame images generated at the same time of two video streams covering adjacent areas of the target area; S200, obtaining an intermediate image ZC2 to be stitched corresponding to C2 through a preset image stitching algorithm; ZC2 is obtained through DH2 and C2, and DH2 is a homography matrix corresponding to C2 when C1 and C2 are stitched; S300, adding ZC2 to the first preset rectangular canvas to obtain the target image to be stitched MC2 corresponding to C2; wherein each pixel in ZC2 is within the first preset rectangular canvas; the color of the first preset rectangular canvas is a preset color; and there is an overlapping area between MC2 and C1; S400: Establish a second preset rectangular canvas and a rectangular coordinate system KT corresponding to the second preset rectangular canvas according to the sizes of C1 and MC2; wherein L≥L C1 +L MC2 , M ≥ M C1 +M MC2 ; L is the length of the two preset rectangular canvases, M is the width of the two preset rectangular canvases, L C1 is the length of C1, L MC2 is the length of C2, M C1 is the width of C1, M MC2 is the width of MC2; S500, adding PJT to a second preset rectangular canvas; wherein PJT is a spliced ​​image generated by splicing C1 and MC2; and a horizontal boundary of C1 is parallel to a horizontal boundary of the second preset rectangular canvas; S600: If the target boundary BJ1 of C1 is perpendicular to the X-axis, obtain the X-axis coordinate value x of BJ1 in KT. C1 ; Wherein, BJ1 is the boundary of C1 that is perpendicular to the X-axis and coincides with MC2; S700: If the target boundary BJ2 of ZC2 is perpendicular to the X axis, obtain the X axis coordinate value x of BJ2 in KT. ZC2 ; Wherein, BJ2 is the boundary of ZC2 that coincides with C1; S800, according to x C1 and x ZC2 , determine the faded area in MC2.

2. The method for determining the faded area of ​​a spliced ​​image according to claim 1, characterized in that: Step S800 includes the following steps: S810, change MC2 to x ZC2 The corresponding column of pixels is determined as the starting column of pixels, and x in MC2 is C1 The corresponding column of pixels is determined as the ending column of pixels; S820: Determine an image region formed by pixels in the start column, pixels in the end column, and all pixels between the pixels in the start column and the pixels in the end column in MC2 as a faded region.

3. The method for determining the faded area of ​​a spliced ​​image according to claim 1, characterized in that: After step S600 and before step S700, the method further includes the following steps: S610: If the target boundary BJ2 of ZC2 is not perpendicular to the X-axis, obtain the coordinate value x of the first endpoint of BJ2 on the X-axis in KT. D1 And the coordinate value x of the second endpoint of BJ2 on the X axis in KT D2 ; Wherein, BJ2 is the boundary of ZC2 in the vertical direction that coincides with C1; S620, if |x D1 -x C1 |>|x D2 -x C1 |, then x in MC2 D1 A corresponding column of pixels is determined as a starting column of pixels; S630, if |x D1 -x C1 |<|x D2 -x C1 |, then x in MC2 D2 A corresponding column of pixels is determined as a starting column of pixels; S640, change MC2 to x C1 The corresponding column of pixels is determined as the ending column of pixels; S650: Determine an image region formed by the pixels in the start column, the pixels in the end column, and all pixels between the pixels in the start column and the pixels in the end column in MC2 as a faded region.

4. The method for determining the faded area of ​​a spliced ​​image according to claim 1, characterized in that: After step S500, the method further comprises the following steps: S510: If the target boundary BJ1 of C1 is not perpendicular to the X-axis and the target boundary BJ2 of ZC2 is not perpendicular to the X-axis, then obtain the coordinate value x of the first endpoint of BJ2 on the X-axis in KT. D1 And the coordinate value x of the second endpoint of BJ2 on the X axis in KT D2 , and the coordinate value x of the first endpoint of BJ1 on the X axis in KT D3 And the coordinate value x of the second endpoint of BJ1 on the X axis in KT D3 ; Wherein, BJ2 is the boundary of ZC2 in the vertical direction that coincides with C1; S520, if MAX(|x D3 |,|x D4 |)>MAX(|x D1 |,|x D2 |), and |x D1 |>|x D2 |, then x in MC2 D2 The corresponding column of pixels is determined as the starting column of pixels; otherwise, x in MC2 is D1 A corresponding column of pixels is determined as a starting column of pixels; S530, MAX(|x D3 |,|x D4 |)>MAX(|x D1 |,|x D2 |), and |x D3 |>|x D4 |, then x in MC2 D3 The corresponding column of pixels is determined as the end column of pixels; otherwise, x in MC2 is D4 A corresponding column of pixels is determined as a termination column of pixels; S540: Determine an image region formed by the pixels in the start column, the pixels in the end column, and all pixels between the pixels in the start column and the pixels in the end column in MC2 as a faded region.

5. The method for determining the faded area of ​​a spliced ​​image according to claim 1, characterized in that: After step S800, the method further includes the following steps: S910, acquiring each target image to be stitched in the target area to obtain a target image sequence A corresponding to the target area = (A1, A2, ..., A i , …, A n ), i=1, 2,..., n, n≥3; among them, A i is the i-th target image to be stitched corresponding to the target area, n is the number of target images to be stitched corresponding to the target area; A r With A r+1 There is an overlapping area in the target area, r=1, 2, ..., n-1; the target image to be stitched is obtained according to the non-first frame image generated at the same time point of each video stream of the target area; S920, if n is an odd number and n>1, obtain a first target number m=(n+1) / 2; S930, split A into a first sequence of images to be stitched A according to m 1 =(A1, A2, ..., A j , …, A m ), j = 1, 2, ..., m; and a second sequence of images to be stitched A 2 =(A m , A m+1 , …, A k , …, A n ), k=m, m+1, ..., n; S940, A 1 Each target image to be stitched in is rotated 180 degrees to obtain a third image sequence to be stitched TA=(TA1, TA2, ..., TA j ,…,TA m ), where TA j A j The image to be stitched after being rotated 180 degrees; S950, invert the arrangement order of all the images to be stitched in TA to obtain a fourth image sequence to be stitched TA'=(TA m ,TA m-1 ,…,TA j , …, TA1); S960, if n≥5, obtain N=1; wherein N is a preset first variable; S970, get TA m-N Corresponding faded area UTA m-N and A m+N Corresponding faded area UA m+N ; S980, for UTA m-N and UA m+N Each pixel in the image is faded to obtain TA m-N The corresponding faded image to be stitched ETA m-N and A m+N The corresponding faded image to be stitched EA m+N ; S990, BA N Determine the main image to be stitched and ETA m-N-1 Splice and connect CA N Determine the main image to be stitched and EA m+N+1 Splice to get BA N and ETA m-N-1 The stitched image BA after stitching N+1 and CA N and EA m+N+1 The stitched image CA after stitching N+1 BA1 is TA m Determine the main image to be stitched and ETA m-1 The stitched image obtained by stitching, CA1 is A m Determine the main image to be stitched and EA m+1 The stitched image obtained by stitching; BA N1 To BA N1-1 With ETA m-N1 The stitched image obtained by stitching, CA N1 To N1-1 With EA m+N1 A stitched image obtained by stitching; N1=2, 3, ..., N; S1000, if N+1<m-1, then update N=N+1 and execute step S970; otherwise, set BA N+1 Rotate 180 degrees to get BA N+1 The corresponding rotated image X'A N+1 ; S1100, X'A N+1 and CA N+1 Stitching is performed to obtain a faded stitched panoramic image corresponding to the target area.

6. The method for determining the faded area of ​​a spliced ​​image according to claim 5, characterized in that: After step S950 and before step S960, the method further includes the following steps: S951, if n=3, obtain the faded area UTA1 corresponding to TA1 and the faded area UA3 corresponding to A3; S952, performing a fade process on each pixel in UTA1 and UA3 to obtain a faded image to be spliced ​​ETA1 corresponding to TA1 and a faded image to be spliced ​​EA3 corresponding to A3; S953, determining TA2 as the main image to be stitched and stitching it with ETA1, and determining A2 as the main image to be stitched and stitching it with EA3, so as to obtain a stitched image BA1 stitched from TA2 and ETA1 and a stitched image CA1 stitched from A2 and A3; S954, rotating BA1 by 180 degrees to obtain a rotated image X'A1 corresponding to BA1; S955, X'A1 and CA1 are spliced ​​to obtain a faded spliced ​​panoramic image corresponding to the target area.

7. The method for determining the faded area of ​​a spliced ​​image according to claim 5, characterized in that: Step S950 includes the following steps: S21, if m is an odd number, replace TA in TA r With TA m-r+1 The arrangement positions are interchanged, r = 1, 2, ..., m; TA b The position remains unchanged, b=(m+1) / 2; to obtain TA'; S22, if m is an even number, then replace TA in TA r With TA m-r+1 The positions are interchanged to obtain TA'.

8. The method for determining the faded area of ​​a spliced ​​image according to claim 1, characterized in that: The preset image stitching algorithm includes a SIFT algorithm.

9. A non-transitory computer-readable storage medium, wherein at least one instruction or at least one program is stored in the storage medium, characterized in that: The at least one instruction or the at least one program is loaded and executed by the processor to implement the method for determining the faded area of ​​the spliced ​​image according to any one of claims 1 to 8.

10. An electronic device, characterized in that: The method comprises a processor and the non-transitory computer-readable storage medium of claim 9.

Images