Image processing method and device for long-focus squint area-array camera

Abstract

The invention provides an image processing method and device for a long-focus squint area-array camera. The method comprises the following steps: acquiring an image distribution sequence diagram of a plurality of to-be-spliced images; for each relation sub-graph, rough matching homonymy point pairs among the images are calculated; taking the reference image as an image plane of the panoramic image, calculating a geometric transformation matrix between each neighborhood image and the reference image according to the roughly matched homonymy point pairs, and calculating coordinate values of each neighborhood image in the image plane according to the geometric transformation matrix; and splicing the plurality of images to be spliced in the blank panoramic layer according to the image planes and the coordinate values corresponding to the relation sub-images to obtain a panoramic spliced image. According to the method provided by the invention, the precision of completing panoramic image splicing by the long-focus squint area array image is improved, and the requirement of obtaining a large-range panoramic image in a long-distance area in a long-distance reconnaissance and emergency target search scene is met.

Description

Technical Field

[0001] The present invention relates to the technical field of aerial remote sensing and image stitching, and in particular to an image processing method and device for a telephoto oblique array camera. Background Art

[0002] The telephoto oblique camera can obtain target image information at a long distance without overhead, and has important application value in long-distance reconnaissance, emergency target search, etc.

[0003] Due to the large tilt angle of the telephoto oblique array image, the objects in the image are severely deformed, and some objects block the distant view. If orthorectification is performed, it will cause more serious geometric stretching distortion. At the same time, due to the limitation of POS (Position and Orientation System) accuracy, direct geolocation has positioning errors, which leads to misalignment of adjacent image stitching, thus affecting the identification of objects.

[0004] In the related technology, traditional aerial remote sensing image stitching usually obtains accurate exterior orientation elements after regional network adjustment and obtains DEM (Digital Elevation Model) data, and then orthorectifies the images to obtain true orthogonal images with complete geographic coordinate information, and then performs panoramic image stitching based on the geographic location, using a long focal length camera for side-view scanning imaging. At this time, the images have a large tilt angle. Since the image overlap is generally low, stereo relativity cannot be formed, DEM data cannot be generated, and the scale of the objects is inconsistent. If orthorectification is performed based on POS data and historically known DEM data, severe geometric stretching deformation will result. In addition, image stitching and positioning based on the orthorectified images will result in large stitching errors and low positioning accuracy, which cannot meet the needs of obtaining large-scale panoramic images of long-distance areas in long-distance reconnaissance and emergency target search scenarios. Summary of the invention

[0005] The present invention provides a method and device for processing images of a telephoto oblique area array camera, which are used to solve the problem that when the images of the telephoto oblique area array camera are processed based on orthorectification technology in the prior art, geometric stretching and deformation of the images are caused, resulting in large stitching errors and low positioning accuracy in image processing tasks such as image stitching and positioning, thereby improving the accuracy of panoramic image stitching completed by telephoto oblique area array images.

[0006] The present invention provides a method for processing an image of a telephoto oblique area array camera, comprising: Acquire an image distribution time sequence diagram of a plurality of images to be stitched, wherein the image distribution time sequence diagram includes a plurality of relationship sub-graphs, each relationship sub-graph includes a reference image and at least one neighborhood image, the reference image and the at least one neighborhood image belong to the plurality of images to be stitched, and the plurality of images to be stitched are collected by a telephoto oblique area array camera; For each relationship subgraph, the coarse matching same-name point pairs between each image are calculated; the reference image is used as the image plane of the panoramic image, the geometric transformation matrix between each neighborhood image and the reference image is calculated according to the coarse matching same-name point pairs, and the coordinate value of each neighborhood image in the image plane is calculated according to the geometric transformation matrix, and the multiple images to be stitched are stitched on a blank panoramic layer according to the image plane corresponding to each relationship subgraph and the coordinate value to obtain a panoramic stitching image.

[0007] According to a method for processing an image of a telephoto oblique area array camera provided by the present invention, after obtaining the panoramic stitching image, the method further includes: Calculating the coordinates of the principal points of each image to be stitched in the panoramic stitching image to obtain a set of principal point coordinates; The perspective transformation matrix is ​​determined according to the mapping relationship between each coordinate in the image principal point coordinate set and the real geographical coordinates of each image principal point, and the geographical coordinates of each image in the panoramic mosaic image are obtained according to the perspective transformation matrix.

[0008] According to a method for processing images of a telephoto oblique area array camera provided by the present invention, the steps of stitching the multiple images to be stitched together on a blank panoramic layer according to the image planes corresponding to the relationship subgraphs and the coordinate values ​​to obtain a panoramic stitching image include: Determining a cross stitching framework diagram according to the number of the plurality of images to be stitched and the expected size of the blank panoramic layer, wherein the expected size includes an expected length and an expected width; Determine the placement position of the target reference image on the cross-stitching framework diagram, and obtain the translation matrix between the target reference image and the blank panoramic layer according to the coordinate values; wherein the target reference image is one of the multiple images to be stitched; The geometric transformation matrix between each image to be stitched and the blank panoramic layer is determined according to the translation matrix, and the placement positions of other images to be stitched except the target reference image in the blank panoramic layer are determined according to the geometric transformation matrix to obtain the panoramic stitching image.

[0009] According to a method for processing an image of a telephoto oblique area array camera provided by the present invention, before obtaining the panoramic stitching image, the method further includes: The plurality of images to be stitched are traversed, and image preprocessing and overlapping area mosaicking operations are performed on each image to be stitched, so as to complete the panoramic image stitching.

[0010] According to a method for processing images of a telephoto oblique area array camera provided by the present invention, the step of obtaining a time sequence diagram of image distribution of a plurality of images to be stitched comprises: Arranging the images to be spliced ​​in time sequence to obtain a one-dimensional array related to time; A two-dimensional image array is constructed with the number of images in the strip as the height and the number of strips as the width to obtain the image distribution time series diagram; The temporal relationship of each image in the image distribution timing diagram is expressed by the following formula: ; in, F is the image distribution timing diagram, i is the number of rows in the two-dimensional image array, j is the number of columns of the 2D image array.

[0011] The present invention also provides a telephoto oblique area array camera image processing device, comprising: A timing diagram acquisition module is used to acquire an image distribution timing diagram of a plurality of images to be stitched, wherein the image distribution timing diagram includes a plurality of relationship sub-graphs, each relationship sub-graph includes a reference image and at least one neighborhood image, the reference image and the at least one neighborhood image belong to the plurality of images to be stitched, and the plurality of images to be stitched are collected by a telephoto oblique area array camera; The panoramic stitching module is used to calculate the coarse matching same-name point pairs between each image for each relationship sub-graph; taking the reference image as the image plane of the panoramic image, calculating the geometric transformation matrix between each neighborhood image and the reference image according to the coarse matching same-name point pairs, and calculating the coordinate value of each neighborhood image in the image plane according to the geometric transformation matrix; stitching the multiple images to be stitched on the blank panoramic layer according to the image plane corresponding to each relationship sub-graph and the coordinate value to obtain a panoramic stitching image.

[0012] According to the present invention, a telephoto oblique area array camera image processing device is provided, the device also includes: A positioning module is used to calculate the coordinates of the principal points of each image to be stitched in the panoramic stitching image after the panoramic stitching image is obtained, and obtain a set of principal point coordinates; determine a perspective transformation matrix according to a mapping relationship between each coordinate in the principal point coordinate set and the real geographic coordinates of each principal point, and obtain the geographic coordinates of the panoramic stitching image according to the perspective transformation matrix.

[0013] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the image processing method for a telephoto oblique area array camera as described above is implemented.

[0014] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the image processing method for a telephoto oblique area array camera as described in any one of the above is implemented.

[0015] The present invention also provides a computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the computer program implements any of the above-mentioned methods for processing images of a telephoto oblique area array camera.

[0016] The telephoto oblique area array camera image processing method and device provided by the present invention obtain an image distribution time sequence diagram of a plurality of images to be stitched, and for each relationship subgraph, calculate the coarse matching same-name point pairs between the images; take the reference image as the image plane of the panoramic image, calculate the geometric transformation matrix between each neighborhood image and the reference image according to the coarse matching same-name point pairs, and calculate the coordinate value of each neighborhood image in the image plane according to the geometric transformation matrix; stitch the plurality of images to be stitched on a blank panoramic layer according to the image plane and the coordinate value corresponding to each relationship subgraph, and obtain a panoramic stitching image, thereby improving the precision of completing the panoramic image stitching by the telephoto oblique area array images, and further meeting the demand for obtaining a large-scale panoramic image of a long-distance area in long-distance reconnaissance and emergency target search scenes. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0018] Figure 1 It is a schematic flow chart of the image processing method of the telephoto oblique area array camera provided by the present invention.

[0019] Figure 2 This is the cross splicing logic diagram provided by the present invention.

[0020] Figure 3 It is the image distribution arrangement relationship and the same-name point relationship diagram provided by the present invention.

[0021] Figure 4 It is a structural schematic diagram of the image processing device of the telephoto oblique area array camera provided by the present invention.

[0022] Figure 5 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0023] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0024] Combine the following Figure 1-Figure 4 The invention describes a method and device for processing images of a telephoto oblique area array camera.

[0025] Figure 1 FIG. 1 is a flow chart of the image processing method of the telephoto oblique area array camera provided by the present invention, such as Figure 1 As shown, the method comprises the following steps: Step 110: obtaining an image distribution time sequence diagram of a plurality of images to be stitched, wherein the image distribution time sequence diagram includes a plurality of relationship sub-graphs, each relationship sub-graph includes a reference image and at least one neighborhood image, the reference image and the at least one neighborhood image belong to a plurality of images to be stitched, and the plurality of images to be stitched are collected by a telephoto oblique area array camera.

[0026] In this step, a plurality of images to be stitched can be captured by a telephoto oblique area array camera under a specific climate environment, for example, on a clear day with sufficient light, to obtain a telephoto oblique image with better color and detail expression.

[0027] In this embodiment, a telephoto oblique area array camera is used to achieve long-distance high-resolution shooting through a telephoto lens, and oblique swing scanning is used to obtain ground information in a larger range. During the shooting process, the camera follows a preset cyclic swing scanning shooting mode, and shoots an image of a strip area in each cycle. At the same time, since the flying platform carrying the camera flies forward, continuous multiple strip image data are obtained to achieve continuous imaging coverage of the target area.

[0028] Specifically, obtaining the image distribution time sequence diagram of the plurality of images to be stitched includes: arranging the images to be stitched in time sequence to obtain a one-dimensional array about time; constructing a two-dimensional image array with the number of images in the strip as the height and the number of strips as the width to obtain the image distribution time sequence diagram; the time sequence relationship of each image in the image distribution time sequence diagram is expressed by the following formula; ; in, F is the image distribution time series diagram, i is the number of rows in the two-dimensional image array, j is the number of columns of the 2D image array.

[0029] In this embodiment, the images and strips in each strip may be arranged in the order of time in which they were taken, from left to right and from top to bottom.

[0030] In this embodiment, the central image of the stitching area is used as the reference image of the panoramic image, and cross stitching and expansion stitching are performed based on the reference image to obtain a stitched panoramic image.

[0031] In this embodiment, the adjacent area may include one or more images to be stitched.

[0032] Step 120: For each relationship subgraph, calculate the coarse matching same-name point pairs between each image; take the reference image as the image plane of the panoramic image, calculate the geometric transformation matrix between each neighborhood image and the reference image according to the coarse matching same-name point pairs, and calculate the coordinate value of each neighborhood image in the image plane according to the geometric transformation matrix; stitch multiple images to be stitched on a blank panoramic layer according to the image plane and coordinate values ​​corresponding to each relationship subgraph to obtain a panoramic stitching image.

[0033] In this step, the reference image in the first relationship subgraph is used as the starting point of the panoramic image, and all other neighborhood images are transformed into the coordinate system of the reference image; this is specifically achieved through the following steps: (1) In the reference image and the adjacent image to be registered, feature point detection algorithms (such as SIFT, SURF, or ORB algorithms) are used to extract feature points. Then, feature matching algorithms (such as FLANN or BFMatcher algorithms) are used to find matching feature point pairs between the reference image and the adjacent image, that is, roughly matching the same-name point pairs, which serve as the basis for the subsequent calculation of the geometric transformation matrix.

[0034] exist Figure 2 In the illustrated embodiment, multiple types of symbols (circle, diamond, triangle and five-pointed star respectively represent feature points) in each image, and symbols of the same type in different images constitute coarse matching pairs of points with the same name.

[0035] (2) Use the random sampling consensus algorithm (RANSAC) to randomly select a small number of matching point pairs to calculate an initial geometric transformation matrix, and then use this matrix to test all matching point pairs and calculate the number of inliers (i.e., point pairs that conform to the matrix). Repeat this process multiple times and finally select the geometric transformation matrix with the largest number of inliers as the geometric transformation matrix for the neighboring image to the reference image. At the same time, perform coordinate transformation on all feature points of the neighboring image and calculate the coordinates of the feature points of the neighboring image in the reference image plane.

[0036] Next, the reference image of the next relational sub-image, that is, the neighboring image of the previous reference image, is used as the new reference image. The geometric transformation matrix between each neighboring image in the current relational sub-image and the reference image is solved according to the above steps (1)-(2). At the same time, the coordinates of all feature points of the neighboring image are transformed, and the coordinates of the feature points of the neighboring image in the reference image plane are calculated. By sequentially processing, the corresponding coordinates of the feature points of each image to be stitched can be converted to the image plane coordinate system of the panoramic image.

[0037] In this embodiment, a blank panoramic layer is established, and the blank panoramic layer is assumed to have theoretical length and width values ​​of adjacent stitching with no overlap between images, so that the translation matrix of the reference image relative to the blank layer can be obtained, and the geometric transformation matrix of all images relative to the blank panoramic layer is finally obtained according to the translation matrix, and then the stitching position of each image compared to the blank panoramic image is determined according to the geometric transformation matrix to obtain a panoramic stitching image.

[0038] The telephoto oblique area array camera image processing method provided by the embodiment of the present invention obtains an image distribution time sequence diagram of multiple images to be stitched, and for each relationship subgraph, calculates the coarse matching same-name point pairs between each image; takes the reference image as the image plane of the panoramic image, calculates the geometric transformation matrix between each neighborhood image and the reference image according to the coarse matching same-name point pairs, and calculates the coordinate value of each neighborhood image in the image plane according to the geometric transformation matrix; stitches the multiple images to be stitched on a blank panoramic layer according to the image plane and the coordinate value corresponding to each relationship subgraph to obtain a panoramic stitching image, thereby improving the accuracy of panoramic image stitching completed by the telephoto oblique area array images, and further meeting the demand for obtaining a large-scale panoramic image of a long-distance area in long-distance reconnaissance and emergency target search scenarios.

[0039] In some embodiments, after obtaining the panoramic stitching image, the telephoto oblique area array camera image processing method further includes: calculating the coordinates of the image principal points of each image to be stitched in the panoramic stitching image to obtain an image principal point coordinate set; determining a perspective transformation matrix based on a mapping relationship between each coordinate in the image principal point coordinate set and the real geographic coordinates of each image principal point, and obtaining the geographic coordinates of the panoramic stitching image according to the perspective transformation matrix.

[0040] In this embodiment, the coordinates of the principal point of each original image to be stitched in the frame coordinate system are defined as (0,0), and the center point of the image in the image coordinate system is (m,n). According to the processing method similar to the above steps (1)-(2), the coordinates of the principal point of the current image are transformed to the image coordinate system of the reference image. After the traversal is completed, the affine transformation matrix of all images relative to the panoramic image is calculated. The coordinates of the principal point of each image in the panoramic image can be calculated by the affine transformation matrix. o i ( x,y ).

[0041] In direct geolocation, the real geographic longitude and latitude coordinates of the main point are solved using the direct geolocation method. O k ( lon,lat ), then the image principal points in the two coordinate systems can be mapped by the image name to form a set of mapping point pairs, namely: in, k The one-dimensional array of images representing the order of the imaging time series.

[0042] In this embodiment, after obtaining the coordinate mapping from the image coordinate system to the geographic coordinate system, the coordinate mapping relationship is assumed to be a perspective relationship, thereby solving the perspective transformation matrix and solving the geographic coordinates of any point in the image coordinate system.

[0043] Specifically, by mapping the geographic coordinates of the image principal point, a set of mapping point pairs for panoramic image geolocation is obtained, and the mapping relationship is determined as a perspective relationship. The same principle as that of solving the image geometric transformation matrix in the above embodiment is used to calculate the optimal perspective transformation matrix using the RANSAC algorithm. T , in order to establish the mapping relationship between the image coordinates and geographic coordinates of the panoramic image, the mapping relationship is expressed by the following formula: ; In this embodiment, the actual geographic coordinates of any pixel in the panoramic image can be calculated using the above mapping relationship.

[0044] For example, by recording the perspective transformation matrix, the real geographic coordinates of any pixel in the panoramic image can be dynamically calculated, and the geographic positioning of the panoramic image can be performed according to the method of writing the geographic coordinates in the TIFF image format.

[0045] Specifically, first use the perspective transformation matrix T Calculate the true geographic coordinates of the upper left, upper right, and lower left corners of the panoramic image O 左上 ( lon,lat ) 、O 右上 ( lon,lat ) 、O 左下 ( lon,lat ), and then use the width of the panoramic image wide and high hight Computational imaging x Directional resolution dlon , y Directional resolution dlat ; dlon and dlaThey are respectively expressed by the following formulas: ; ; The affine transformation matrix of the six parameters required for the TIFF image is calculated by the following formula: ; Finally, write the above six parameter affine matrices into the TIFF image and set the geographic coordinate system to the WGS_1984 coordinate system to complete the geographic coordinate positioning of the panoramic image.

[0046] The telephoto oblique area array camera image processing method and device provided by the present invention obtain an image principal point coordinate set by calculating the coordinates of the image principal point of each image to be stitched in a panoramic stitching image; determine a perspective transformation matrix according to a mapping relationship between each coordinate in the image principal point coordinate set and the real geographic coordinates of each image principal point, and obtain the geographic coordinates of the panoramic stitching image according to the perspective transformation matrix, thereby realizing the construction of a mapping relationship from the image space to the object space coordinates based on the direct geographic positioning coordinate constraints of the image principal point of each image, and improving the positioning accuracy of the panoramic image.

[0047] In some embodiments, multiple images to be stitched are stitched on a blank panoramic layer according to image planes and coordinate values ​​corresponding to each relationship sub-graph to obtain a panoramic stitching image, including: determining a cross-stitching framework diagram according to the number of multiple images to be stitched and an expected size of the blank panoramic layer, the expected size including an expected length and an expected width; determining a placement position of a target reference image on the cross-stitching framework diagram, and obtaining a translation matrix between the target reference image and the blank panoramic layer according to the coordinate value; wherein the target reference image is one of the multiple images to be stitched; determining a geometric transformation matrix between each image to be stitched and the blank panoramic layer according to the translation matrix, and determining placement positions of other images to be stitched except the target reference image in the blank panoramic layer according to the geometric transformation matrix to obtain a panoramic stitching image.

[0048] Figure 2 is the cross splicing logic diagram provided by the present invention, in Figure 2In the embodiment shown, it is assumed that there are multiple images (pictures), and the width and height of each image are known. Then, the width of the blank panoramic layer is the width of a single image multiplied by the number of strips (i.e., the number of images in the flight direction), and the height of the base map is the height of a single image multiplied by the number of images in the strip (i.e., the number of images in the vertical flight direction; image stitching is performed through the following steps: S1, selecting a reference sub-image from all images as an intermediate reference image; the reference image will be used as the starting point of the panoramic base map, and other images will be transformed relative to this reference image; for the blank panoramic layer, the selection of the intermediate reference image only involves To a translation transformation, the position of the reference image in the base map is known (usually in the center of the stitching area); S2, taking the horizontal and vertical directions of the reference image as the reference (forming a "cross" shape), calculate the geometric transformation matrix of other images (Li, Ti, Ri, Bi, representing the images in the left, upper, right, and lower directions, respectively, where i=1,2,...) relative to the reference image, and finally, for the images in each direction (L1, T1, R1, B1), calculate the coordinates of the transformed feature points on the base map; repeat the above steps S1 and S2 until all the images to be stitched are filled in the blank panoramic layer to obtain the entire panoramic stitching image.

[0049] In this embodiment, a central or most representative image is selected as a reference image (e.g., Figure 2 The central image in the image is taken as the reference sub-image), the placement of the reference image is determined on the blank panoramic layer, and the translation matrix from the reference image to the panoramic layer is calculated according to the resolution of the reference image and the resolution of the panoramic layer.

[0050] In this embodiment, for each image to be stitched, a feature point matching algorithm (such as SIFT, SURF or ORB) is used to find the overlapping area with the reference image, and the transformation matrix (including translation, rotation and scaling) of each image to the reference image is calculated based on the matched feature points, and then these local transformation matrices are converted into a global transformation matrix relative to the panoramic layer; then, the calculated global transformation matrix is ​​used to transform each image and place it in the correct position on the panoramic layer, and finally all the transformed images are merged to generate a panoramic stitching image.

[0051] In this embodiment, after the panoramic stitching image is obtained, any obvious stitching errors or defects need to be checked and possibly repaired.

[0052] Specifically, before obtaining the panoramic stitching image, the method further includes: traversing a plurality of images to be stitched, and performing image preprocessing and overlapping area mosaic operations on each image to be stitched, so as to complete the panoramic image stitching.

[0053] In this embodiment, for the overlapping area, weighted averaging or other fusion algorithms may be used to smooth the transition and reduce stitching traces.

[0054] The telephoto oblique area array camera image processing method and device provided by the present invention realizes splicing of multiple images into a seamless panoramic image by constructing a cross-stitching framework diagram, and performs outward expansion splicing based on the cross-stitching framework diagram, thereby reducing image splicing distortion.

[0055] Figure 3 The image distribution arrangement relationship and the same-name point relationship diagram provided by the present invention are Figure 3 In the embodiment shown, the reference image (reference sub-image) includes four adjacent areas: upper, lower, left, and right. Each time the horizontal and vertical directions of the current reference sub-image are used as references (forming a "cross" shape) for splicing and calculating the coordinates of the neighboring images, the coordinates are calculated according to Figure 2 The embodiment shown performs multiple iterations of the corresponding stitching method (the current reference image is the neighborhood image during the last cross stitching), and finally obtains a panoramic image and the distribution relationship of each image on the panoramic image and the relationship between the corresponding feature points.

[0056] The following is a description of the telephoto oblique view area array camera image processing device provided by the present invention. The telephoto oblique view area array camera image processing device described below and the telephoto oblique view area array camera image processing method described above can correspond to each other.

[0057] Figure 4 is a schematic structural diagram of a telephoto oblique area array camera image processing device provided by the present invention, Figure 4 In the embodiment shown, the telephoto oblique area array camera image processing device includes: a timing diagram acquisition module 410 and a panoramic stitching module 420 A timing diagram acquisition module 410 is used to acquire an image distribution timing diagram of a plurality of images to be stitched, wherein the image distribution timing diagram includes a plurality of relationship sub-graphs, each relationship sub-graph includes a reference image and at least one neighboring image, the reference image and the at least one neighboring image belong to a plurality of images to be stitched, and the plurality of images to be stitched are collected by a telephoto oblique area array camera; The panoramic stitching module 420 is used to calculate the coarse matching same-name point pairs between each image for each relationship sub-graph; taking the reference image as the image plane of the panoramic image, calculating the geometric transformation matrix between each neighborhood image and the reference image according to the coarse matching same-name point pairs, and calculating the coordinate value of each neighborhood image in the image plane according to the geometric transformation matrix; stitching multiple images to be stitched on a blank panoramic layer according to the image plane and coordinate values ​​corresponding to each relationship sub-graph, to obtain a panoramic stitching image.

[0058] The image processing device for a telephoto oblique area array camera provided by an embodiment of the present invention obtains an image distribution time sequence diagram of a plurality of images to be stitched, and for each relationship subgraph, calculates the coarse matching same-name point pairs between the images; takes the reference image as the image plane of the panoramic image, calculates the geometric transformation matrix between each neighborhood image and the reference image according to the coarse matching same-name point pairs, and calculates the coordinate value of each neighborhood image in the image plane according to the geometric transformation matrix; stitches the plurality of images to be stitched on a blank panoramic layer according to the image plane and the coordinate value corresponding to each relationship subgraph, and obtains a panoramic stitching image, thereby improving the stitching accuracy of the panoramic image stitching completed by the telephoto oblique area array images, and thus meets the demand for obtaining a large-scale panoramic image of a long-distance area in long-distance reconnaissance and emergency target search scenarios. In some embodiments, the telephoto oblique area array camera image processing device further includes: a positioning module 430 .

[0059] The positioning module 430 is used to calculate the coordinates of the principal points of each image to be stitched in the panoramic stitching image after obtaining the panoramic stitching image, and obtain a set of principal point coordinates; determine the perspective transformation matrix according to the mapping relationship between each coordinate in the principal point coordinate set and the real geographic coordinates of each principal point, and obtain the geographic coordinates of the panoramic stitching image according to the perspective transformation matrix.

[0060] The telephoto oblique area array camera image processing method and device provided by the present invention obtain an image principal point coordinate set by calculating the coordinates of the image principal point of each image to be stitched in a panoramic stitching image; determine a perspective transformation matrix according to a mapping relationship between each coordinate in the image principal point coordinate set and the real geographic coordinates of each image principal point, and obtain the geographic coordinates of the panoramic stitching image according to the perspective transformation matrix, thereby realizing the construction of a mapping relationship from the image space to the object space coordinates based on the direct geographic positioning coordinate constraints of the image principal point of each image, and improving the positioning accuracy of the panoramic image.

[0061] Figure 5 is a schematic diagram of the structure of the electronic device provided by the present invention, such as Figure 5As shown, the electronic device may include: a processor (processor) 510, a communication interface (Communications Interface) 520, a memory (memory) 530 and a communication bus 540, wherein the processor 510, the communication interface 520, and the memory 530 communicate with each other through the communication bus 540. The processor 510 can call the logic instructions in the memory 530 to execute the telephoto oblique area array camera image processing method, which includes: obtaining an image distribution time sequence diagram of multiple images to be stitched, the image distribution time sequence diagram includes multiple relationship sub-graphs, each relationship sub-graph includes a reference image and at least one neighborhood image, the reference image and the at least one neighborhood image belong to the multiple images to be stitched, and the multiple images to be stitched are collected by the telephoto oblique area array camera; for each relationship sub-graph, calculating the coarse matching same-name point pairs between each image; taking the reference image as the image plane of the panoramic image, calculating the geometric transformation matrix between each neighborhood image and the reference image according to the coarse matching same-name point pairs, and calculating the coordinate value of each neighborhood image in the image plane according to the geometric transformation matrix, and stitching the multiple images to be stitched on the blank panoramic layer according to the image plane and the coordinate value corresponding to each relationship sub-graph to obtain a panoramic stitching image.

[0062] In addition, the logic instructions in the above-mentioned memory 530 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

[0063] On the other hand, the present invention also provides a computer program product, which includes a computer program. The computer program can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the telephoto oblique area array camera image processing method provided by the above methods, the method including: obtaining an image distribution time sequence diagram of multiple images to be stitched, the image distribution time sequence diagram including multiple relationship sub-graphs, each relationship sub-graph including a reference image and at least one neighborhood image, the reference image and the at least one neighborhood image belonging to multiple images to be stitched, and the multiple images to be stitched are collected by a telephoto oblique area array camera; for each relationship sub-graph, calculating the coarse matching same-name point pairs between each image; taking the reference image as the image plane of the panoramic image, calculating the geometric transformation matrix between each neighborhood image and the reference image according to the coarse matching same-name point pairs, and calculating the coordinate value of each neighborhood image in the image plane according to the geometric transformation matrix, and stitching the multiple images to be stitched on a blank panoramic layer according to the image plane and coordinate values ​​corresponding to each relationship sub-graph to obtain a panoramic stitching image.

[0064] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which is implemented when the computer program is executed by a processor to execute the telephoto oblique area array camera image processing method provided by the above methods, the method comprising: obtaining an image distribution time sequence diagram of multiple images to be stitched, the image distribution time sequence diagram comprising multiple relationship subgraphs, each relationship subgraph comprising a reference image and at least one neighborhood image, the reference image and the at least one neighborhood image belonging to multiple images to be stitched, and the multiple images to be stitched are collected by a telephoto oblique area array camera; for each relationship subgraph, calculating coarse matching same-name point pairs between each image; taking the reference image as the image plane of the panoramic image, calculating the geometric transformation matrix between each neighborhood image and the reference image according to the coarse matching same-name point pairs, and calculating the coordinate value of each neighborhood image in the image plane according to the geometric transformation matrix, and stitching the multiple images to be stitched on a blank panoramic layer according to the image plane and the coordinate value corresponding to each relationship subgraph to obtain a panoramic stitching image.

[0065] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without creative work.

[0066] Through the description of the above implementation methods, technicians in this field can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiment.

[0067] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, ordinary technicians in this field should understand that they can still modify the technical solutions recorded in the aforementioned embodiments, or replace some of the technical features therein with equivalents. However, these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for processing images of a telephoto oblique area array camera, characterized in that: include: Acquire an image distribution time sequence diagram of a plurality of images to be stitched, wherein the image distribution time sequence diagram includes a plurality of relationship sub-graphs, each relationship sub-graph includes a reference image and at least one neighborhood image, the reference image and the at least one neighborhood image belong to the plurality of images to be stitched, and the plurality of images to be stitched are collected by a telephoto oblique area array camera; For each relationship subgraph, the coarse matching same-name point pairs between each image are calculated; the reference image is used as the image plane of the panoramic image, the geometric transformation matrix between each neighborhood image and the reference image is calculated according to the coarse matching same-name point pairs, and the coordinate value of each neighborhood image in the image plane is calculated according to the geometric transformation matrix, and the multiple images to be stitched are stitched on a blank panoramic layer according to the image plane corresponding to each relationship subgraph and the coordinate value to obtain a panoramic stitching image.

2. The method for processing images of a telephoto oblique area array camera according to claim 1, characterized in that: After obtaining the panoramic mosaic image, the method further includes: Calculating the coordinates of the principal points of each image to be stitched in the panoramic stitching image to obtain a set of principal point coordinates; A perspective transformation matrix is ​​determined according to a mapping relationship between each coordinate in the image principal point coordinate set and the real geographic coordinates of each image principal point, and the geographic coordinates of the panoramic stitching image are obtained according to the perspective transformation matrix.

3. The method for processing images of a telephoto oblique area array camera according to claim 1, characterized in that: The step of stitching the plurality of images to be stitched together on a blank panoramic layer according to the image planes corresponding to the relationship subgraphs and the coordinate values ​​to obtain a panoramic stitching image comprises: Determining a cross stitching framework diagram according to the number of the plurality of images to be stitched and the expected size of the blank panoramic layer, wherein the expected size includes an expected length and an expected width; Determine the placement position of the target reference image on the cross-stitching framework diagram, and obtain the translation matrix between the target reference image and the blank panoramic layer according to the coordinate values; wherein the target reference image is one of the multiple images to be stitched; The geometric transformation matrix between each image to be stitched and the blank panoramic layer is determined according to the translation matrix, and the placement positions of other images to be stitched except the target reference image in the blank panoramic layer are determined according to the geometric transformation matrix to obtain the panoramic stitching image.

4. The method for processing images of a telephoto oblique area array camera according to claim 3, characterized in that: Before obtaining the panoramic mosaic image, the method further includes: The plurality of images to be stitched are traversed, and image preprocessing and overlapping area mosaicking operations are performed on each image to be stitched, so as to complete the panoramic image stitching.

5. The method for processing images of a telephoto oblique area array camera according to claim 1, characterized in that: The step of obtaining a time sequence diagram of image distribution of a plurality of images to be stitched comprises: Arranging the images to be spliced ​​in time sequence to obtain a one-dimensional array related to time; A two-dimensional image array is constructed with the number of images in the strip as the height and the number of strips as the width to obtain the image distribution time series diagram; The temporal relationship of each image in the image distribution timing diagram is expressed by the following formula: ； in, F is the image distribution timing diagram, i is the number of rows in the two-dimensional image array, j is the number of columns of the 2D image array.

6. A telephoto oblique area array camera image processing device, characterized in that: include: A timing diagram acquisition module is used to acquire an image distribution timing diagram of a plurality of images to be stitched, wherein the image distribution timing diagram includes a plurality of relationship sub-graphs, each relationship sub-graph includes a reference image and at least one neighborhood image, the reference image and the at least one neighborhood image belong to the plurality of images to be stitched, and the plurality of images to be stitched are collected by a telephoto oblique area array camera; The panoramic stitching module is used to calculate the coarse matching same-name point pairs between each image for each relationship sub-graph; taking the reference image as the image plane of the panoramic image, calculating the geometric transformation matrix between each neighborhood image and the reference image according to the coarse matching same-name point pairs, and calculating the coordinate value of each neighborhood image in the image plane according to the geometric transformation matrix; stitching the multiple images to be stitched on the blank panoramic layer according to the image plane corresponding to each relationship sub-graph and the coordinate value to obtain a panoramic stitching image.

7. The image processing device for a telephoto oblique area array camera according to claim 6, characterized in that: The device also includes: A positioning module is used to calculate the coordinates of the principal points of each image to be stitched in the panoramic stitching image after the panoramic stitching image is obtained, and obtain a set of principal point coordinates; determine a perspective transformation matrix according to a mapping relationship between each coordinate in the principal point coordinate set and the real geographic coordinates of each principal point, and obtain the geographic coordinates of the panoramic stitching image according to the perspective transformation matrix.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the telephoto oblique area array camera image processing method according to any one of claims 1 to 5 is implemented.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the image processing method for a telephoto oblique area array camera as claimed in any one of claims 1 to 5 is implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the image processing method for a telephoto oblique area array camera as claimed in any one of claims 1 to 5 is implemented.

Images