Aerial strip image fusion method and device based on homonymy points and electronic equipment

Abstract

The invention belongs to the technical field of image processing, and discloses a homonymy point-based air strip image fusion method and device and electronic equipment, and the method comprises the steps: obtaining a first air strip image and a second air strip image; performing feature matching on the first air strip image and the second air strip image to obtain a plurality of feature point matching pairs as initial homonymy point pairs; screening the initial homonymy point pairs based on the attributes of the plurality of homonymy point connecting lines to obtain first homonymy point pairs, and responding to the screening operation of the target user for the first homonymy point pairs to obtain second homonymy point pairs; partitioning the first air strip image and the second air strip image to obtain a first partitioned image and a second partitioned image; carrying out resampling on the second block image; and carrying out image fusion on the second block image and the first block image based on the transverse offset of each row of pixel points. Accurate homonymy point pairs are obtained through secondary screening, and the image fusion method which does not depend on the quality and content of the air strip image is realized through resampling.

Description

Technical Field

[0001] The invention belongs to the technical field of image processing, and in particular relates to a method, a device and an electronic device for fusing flight strip images based on homonymous points. Background Art

[0002] Image stitching technology refers to the technology of stitching two or more images with overlapping parts into a high-resolution, wide-viewing angle image containing the information of each image through image preprocessing, image registration and image fusion technology.

[0003] A flight strip image is a sequence of overlapping images obtained by aerial photography along a certain direction. These image sequences have important applications in surveying and mapping and geographic information collection. In related technologies, when fusing flight strip images, the image quality and image content of the flight strip image are usually relied on. If the image quality of the flight strip image is low or there are no feature points in the flight strip image, the image fusion will fail. Summary of the invention

[0004] The purpose of the present invention is to obtain accurate pairs of points with the same name in adjacent flight strip images using a secondary screening method, and to cope with possible squeezing or stretching in the flight strip images by resampling, thereby realizing an image fusion method that is independent of the quality and content of the flight strip images.

[0005] In a first aspect, an embodiment of the present invention provides a method for fusion of flight strip images based on homonymous points, the method comprising:

[0006] Acquire a first flight strip image and a second flight strip image, wherein the first flight strip image and the second flight strip image are images corresponding to adjacent flight strips during the UAV's bow-shaped flight;

[0007] Performing feature matching on the first flight strip image and the second flight strip image to obtain a plurality of feature point matching pairs as initial same-name point pairs, and connecting two same-name points included in each initial same-name point pair to obtain a plurality of same-name point connecting lines;

[0008] Based on the obtained properties of the plurality of same-name point lines, the initial same-name point pairs are screened to obtain first same-name point pairs, and in response to a target user's screening operation on the first same-name point pairs, the first same-name point pairs are added and / or deleted to obtain second same-name point pairs;

[0009] For each second same-name point pair, the first flight strip image is divided into blocks based on the row where the first same-name point included in the second same-name point pair is located to obtain a plurality of first block images, and the second flight strip image is divided into blocks based on the row where the second same-name point included in the second same-name point pair is located to obtain a plurality of second block images;

[0010] Taking each first block image as a reference, resampling the second block image corresponding to the first block image to obtain a resampled second block image, wherein the resampled second block image has the same size as the first block image;

[0011] For each second block image, based on the second horizontal coordinate of the second same-name point in the second block image and the first horizontal coordinate of the first same-name point in the first block image corresponding to the second block image, the horizontal offset of each row of pixel points in the second block image is calculated, and image fusion is performed on the second block image and the first block image based on the horizontal offset of each row of pixel points.

[0012] Optionally, the filtering of the initial pairs of points of the same name based on the properties of the obtained multiple lines of points of the same name to obtain the first pairs of points of the same name includes:

[0013] Respectively obtain the mapping line segments in the longitudinal direction and the mapping line segments in the transverse direction of the multiple lines connecting the points of the same name;

[0014] Compare the mapped line segment in the vertical direction with the preset segment length. If the mapped line segment in the vertical direction is greater than the preset segment length, filter out the corresponding initial same-name point pairs.

[0015] Compare the horizontal mapping line segment with the preset ratio of the image width, and if the horizontal mapping line segment is smaller than the preset ratio of the image width, filter out the corresponding initial same-name point pairs;

[0016] The lengths of the multiple lines connecting the same name points are compared with the image width respectively. If the length of the lines connecting the same name points is greater than the image width, the corresponding initial same name point pairs are filtered out. The image width is the first width of the first flight strip image or the second width of the second flight strip image, and the first width is the same as the second width.

[0017] Optionally, the method further includes:

[0018] Determine whether there are multiple first points with the same name within a preset number of pixel points in the longitudinal direction of the first flight strip image; if there are multiple first points with the same name, retain only one target first point with the same name and delete the other first points with the same name;

[0019] The second point of the same name in the second flight strip image that matches the first point of the same name of the target is deleted.

[0020] Optionally, the method further includes:

[0021] Displaying the first flight strip image, the second flight strip image, and the first pair of points with the same name in a visual display interface;

[0022] Correspondingly, in response to the target user's screening operation on the first same-name point pairs, the first same-name point pairs are added and / or deleted to obtain second same-name point pairs, including:

[0023] receiving a screening operation input by a target user through a visual interface, wherein the screening operation includes deleting incorrect same-name point pairs in the first same-name point pairs, and / or adding correct same-name point pairs in the first same-name point pairs;

[0024] If the screening operation is to delete incorrect same-name point pairs in the first same-name point pairs, in response to the screening operation, delete incorrect same-name point pairs in the first same-name point pairs;

[0025] If the screening operation is to add a correct pair of same-named points to the first pair of same-named points, in response to the screening operation, the correct pair of same-named points is added to the first pair of same-named points.

[0026] Optionally, for each second block image, calculating the lateral offset of each row of pixel points in the second block image based on the second horizontal coordinate of the second same-name point in the second block image and the first horizontal coordinate of the first same-name point in the first block image corresponding to the second block image includes:

[0027] For each second block image, based on the second horizontal coordinates of the second like-named points in the second block image, the first horizontal coordinates of the first like-named points corresponding to the second like-named points in the first block image, and the image width, calculate the horizontal offset of the row pixel points of the row where the second like-named points are located;

[0028] For any row of pixels between two second vertically adjacent points of the same name in the second block image, the lateral offset of the row of pixels is calculated based on the lateral offset of the row of pixels in which the two vertically adjacent points of the same name are located, the row number of the row of pixels, and the height of the image between the two vertically adjacent points of the same name.

[0029] Optionally, the performing image fusion on the second block image and the first block image based on the lateral offset of each row of pixel points includes:

[0030] Based on the lateral offset of each row of pixel points, determining, in the first block image, first pixel points that overlap with each second pixel point in the second block image;

[0031] For each second pixel point of the second block image, a linear weighted sum is performed on a pixel value of the second pixel point and a pixel value of a first pixel point overlapping with the second pixel point.

[0032] In a second aspect, an embodiment of the present invention provides a flight path image fusion device based on homonymous points, the device comprising:

[0033] A flight strip image acquisition module, used to acquire a first flight strip image and a second flight strip image, wherein the first flight strip image and the second flight strip image are images corresponding to adjacent flight strips during the UAV's bow-shaped flight;

[0034] A same-name point pair determination module is used to perform feature matching on the first flight strip image and the second flight strip image to obtain a plurality of feature point matching pairs as initial same-name point pairs, and connect the two same-name points included in each initial same-name point pair to obtain a plurality of same-name point connection lines;

[0035] a homonymous point pair screening module, configured to screen the initial homonymous point pairs based on the properties of the obtained plurality of homonymous point lines to obtain first homonymous point pairs, and in response to a target user's screening operation on the first homonymous point pairs, add and / or delete the first homonymous point pairs to obtain second homonymous point pairs;

[0036] A flight strip image blocking module is used for, for each second same-name point pair, blocking the first flight strip image based on the row where the first same-name point included in the second same-name point pair is located to obtain a plurality of first blocked images, and blocking the second flight strip image based on the row where the second same-name point included in the second same-name point pair is located to obtain a plurality of second blocked images;

[0037] An image block resampling module, used for resampling a second block image corresponding to each first block image based on the first block image to obtain a resampled second block image, wherein the resampled second block image has the same size as the first block image;

[0038] The flight strip image fusion module is used to calculate the lateral offset of each row of pixel points in each second block image based on the second horizontal coordinate of the second same-name point in the second block image and the first horizontal coordinate of the first same-name point in the first block image corresponding to the second block image, and perform image fusion on the second block image with the first block image based on the lateral offset of each row of pixel points.

[0039] In a third aspect, an embodiment of the present invention provides an electronic device, including:

[0040] at least one processor;

[0041] a memory for storing the at least one processor-executable instruction;

[0042] The at least one processor is configured to execute the instructions to implement the method described in the first aspect.

[0043] In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium. When instructions in the computer-readable storage medium are executed by a processor of an electronic device, the electronic device is enabled to execute the method described in the first aspect.

[0044] In a fifth aspect, an embodiment of the present invention provides a computer program product, including a computer program, and when the computer program is executed by a processor, the method described in the first aspect is implemented.

[0045] The technical solution provided by the embodiment of the present invention can screen the initial same-name point pairs through a secondary screening method, and then accurately obtain the same-name point pairs of the flight strip image to be stitched, and by resampling the image to be stitched, it can cope with the possible squeezing or stretching in the flight strip image, thereby realizing the flight strip image fusion that is independent of the quality and content of the flight strip image. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 A schematic diagram of a flow chart of a flight strip image fusion method based on same-name points provided in an embodiment of the present invention;

[0047] Figure 2 A schematic diagram of the structure of a flight path image fusion device based on homonymous points provided by an embodiment of the present invention;

[0048] Figure 3 A schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0049] The present invention will be described in detail below through examples.

[0050] Image stitching technology refers to the technology of stitching two or more images with overlapping parts into a high-resolution, wide-viewing angle image containing the information of each image through image preprocessing, image registration and image fusion technology.

[0051] A flight strip image is a sequence of overlapping images obtained by aerial photography along a certain direction. These image sequences have important applications in surveying and mapping and geographic information collection. In related technologies, when fusing flight strip images, the image quality and image content of the flight strip image are usually relied on. If the image quality of the flight strip image is low or there are no feature points in the flight strip image, the image fusion will fail.

[0052] In order to solve the above technical problems, the embodiment of the present invention provides a universal image fusion method that does not depend on the quality and content of the flight strip image. The flight strip image fusion method provided by the embodiment of the present invention can be applied to the application scenario of UAV bow flight, and the flight strip image is linear array push-scan imaging data.

[0053] The overall technical solution of the embodiment of the present invention may include the following steps:

[0054] 1. Acquire adjacent flight strip images, and use segmented feature detection and matching methods between adjacent flight strip images to obtain multiple feature point matching pairs as initial same-name point pairs, where the widths of adjacent flight strip images are the same.

[0055] 2. For each initial pair of points with the same name, connect the two points with the same name included in the initial pair of points with the same name to obtain a line of points with the same name, perform a preliminary screening on the initial pair of points with the same name according to the properties of the line of points with the same name, such as position, length, angle, etc., and obtain the first pair of points with the same name after the preliminary screening. The adjacent flight strip images and the line of points with the same name of the first pair of points with the same name after the preliminary screening can be displayed in the visual display interface.

[0056] 3. The user can input a screening operation for the first pair of homonymous points in the visual display interface to obtain a second pair of homonymous points after secondary screening; wherein the screening operation includes deleting incorrect pair of homonymous points in the first pair of homonymous points and adding correct pair of homonymous points to the first pair of homonymous points.

[0057] 4. After obtaining the second pair of points with the same name after secondary screening, the points with the same name included in the first flight strip image can be called first points with the same name, and the points with the same name included in the second flight strip image can be called second points with the same name. The first flight strip image can be divided into blocks according to the rows where the first points with the same name are located to obtain multiple first image blocks, and the second flight strip image can be divided into blocks according to the rows where the second points with the same name are located to obtain multiple second image blocks.

[0058] It can be understood that, since the first like-named points and the second like-named points have a one-to-one correspondence, the first image blocks and the second image blocks have a one-to-one correspondence.

[0059] 5. Taking each first image block as a reference, resample the second image block corresponding to the first image block so that the size of the resampled second image block is the same as the size of the first image block, that is, the resampled second image block is aligned with the first image block.

[0060] 6. After aligning each second image block with the corresponding first image block through resampling, next, calculate the lateral offset of each row of pixel points in each second image block, and based on the calculated lateral offset of each row of pixel points, perform image fusion on the overlapping part of the second image block and the first image block.

[0061] It can be seen that the present invention uses a secondary screening method to obtain accurate pairs of points with the same name in adjacent flight strip images, and copes with possible squeezing or stretching in the flight strip images through resampling, thereby realizing an image fusion method that is independent of the quality and content of the flight strip images.

[0062] A method for fusion of flight strip images based on same-name points provided by an embodiment of the present invention will be described in detail below.

[0063] like Figure 1 As shown, a method for fusion of flight strip images based on homonymous points provided by an embodiment of the present invention may include the following steps:

[0064] S110, acquiring a first flight strip image and a second flight strip image.

[0065] The first flight strip image and the second flight strip image are images corresponding to adjacent flight strips during the UAV's bow-shaped flight.

[0066] Specifically, the start and end line numbers of each flight strip can be obtained by manual designation or by analyzing the GPS track, and the flight strip image can be intercepted. The original image data collected by the UAV linear array push sweep is composed of rows of images arranged vertically, so when collecting in a bow-shaped flight, it is necessary to intercept the flight strip image according to the start and end line numbers of each flight strip. Among them, the selection of the start and end line numbers can be manually specified, or if real-time GPS information is stored during the flight strip image collection process, the start and end line numbers can be obtained by analyzing the flight strip track.

[0067] In practical applications, there are usually multiple flight strip images. When fusing flight strip images, adjacent flight strip images are usually fused step by step. For example, assuming there are three flight strip images, the first flight strip image can be fused with the second flight strip image to obtain a fused flight strip image, and then the fused flight strip image can be fused with the third flight strip image in the same way.

[0068] The embodiment of the present invention can be applied to the application scenario of the UAV bow-shaped flight, by acquiring the first flight strip image and the second flight strip image corresponding to any adjacent flight strips during the UAV bow-shaped flight, so as to perform image fusion on the first flight strip image and the second flight strip image.

[0069] S120, feature matching is performed on the first flight strip image and the second flight strip image to obtain a plurality of feature point matching pairs as initial same-name point pairs, and two same-name points included in each initial same-name point pair are connected to obtain a plurality of same-name point connection lines.

[0070] Specifically, after the first flight strip image and the second flight strip image are acquired, feature matching is performed on the first flight strip image and the second flight strip image to obtain a plurality of feature point matching pairs as initial same-name point pairs.

[0071] Since there may be erroneous feature point matching pairs among the multiple feature point matching pairs, there may also be erroneous same-name point pairs among the initial same-name point pairs. Therefore, it is necessary to connect the two same-name points included in each initial same-name point pair to obtain multiple same-name point connection lines, so that in the following steps, the initial same-name point pairs can be screened by the attributes of the multiple same-name point connection lines.

[0072] S130, based on the properties of the obtained multiple points-of-the-same-name lines, the initial points-of-the-same-name pairs are screened to obtain first points-of-the-same-name pairs, and in response to the target user's screening operation on the first points-of-the-same-name pairs, the first points-of-the-same-name pairs are added and / or deleted to obtain second points-of-the-same-name pairs.

[0073] Specifically, the attributes of the same-name point connection line may include: the position, length, and connection angle of the same-name point connection line. In actual situations, the lengths of multiple same-name point connections are basically the same, and the angles of multiple same-name point connections are also basically the same. Therefore, the same-name point pairs corresponding to the same-name point connections with basically the same length and basically the same angle will not be filtered out, and the same-name point pairs with obviously different lengths from other same-name point connections will be filtered out, and the same-name point pairs with obviously different angles from other same-name point connections will also be filtered out.

[0074] In addition, the same-name point pairs can be screened based on the mapped line segments of the same-name point connection lines in the longitudinal and transverse directions, and the same-name point pairs can be screened based on the density of the same-name points in the longitudinal direction in the flight strip image. In order to describe the solution clearly, it will be described in detail in the following embodiments.

[0075] In order to improve the accuracy of the same-name point pairs, the initial same-name point pairs are screened according to the attributes of the same-name point connection lines. After the first same-name point pairs are obtained, the first same-name point pairs can be screened again.

[0076] As an implementation of the embodiment of the present invention, the method may further include: displaying the first flight strip image, the second flight strip image and the first pair of points with the same name in a visual display interface.

[0077] Accordingly, in response to the target user's screening operation on the first same-name point pairs, adding and / or deleting the first same-name point pairs to obtain second same-name point pairs may include the following steps:

[0078] A screening operation input by a target user through a visual interface is received, where the screening operation includes deleting incorrect same-name point pairs in the first same-name point pairs, and / or adding correct same-name point pairs to the first same-name point pairs.

[0079] If the screening operation is to delete the incorrect same-name point pairs in the first same-name point pairs, in response to the screening operation, the incorrect same-name point pairs in the first same-name point pairs are deleted.

[0080] If the screening operation is to add a correct pair of same-named points to the first pair of same-named points, in response to the screening operation, the correct pair of same-named points is added to the first pair of same-named points.

[0081] Specifically, the first flight strip image, the second flight strip image and the first pair of points with the same name can be displayed in a visual display interface. From the visual display interface, it can be seen that the correct line connecting the points with the same name has similar properties such as length and angle, and the wrong pair of points with the same name can be clearly identified. The user can input the operation of deleting the wrong pair of points with the same name in the visual display interface, thereby deleting the wrong pair of points with the same name. In addition, for the part where the pairs of points with the same name are sparse, the user can manually specify the correct pair of points with the same name and add the pair of points with the same name. The pair of points with the same name that is screened by the user for the second time is called the second pair of points with the same name. By performing a secondary screening on the initial pair of points with the same name, the accuracy of the final pair of points with the same name can be guaranteed.

[0082] S140, for each second pair of points with the same name, the first flight strip image is divided into blocks based on the row where the first point with the same name included in the second pair of points with the same name is located, to obtain a plurality of first block images, and the second flight strip image is divided into blocks based on the row where the second point with the same name included in the second pair of points with the same name is located, to obtain a plurality of second block images.

[0083] Specifically, after obtaining the final second same-name point pair, each second same-name point pair includes two same-name points, and the same-name points located in the first flight strip image can be called first same-name points, and the same-name points located in the second flight strip image can be called second same-name points. Assume that the vertical coordinates of the first same-name points are Ay1, Ay2, Ay3, ..., AyN, N is the total number of same-name points in the first flight strip image, and N is a positive integer. Assume that the vertical coordinates of the second same-name points are By1, By2, By3, ..., ByN. After obtaining the positions of the N first same-name points of the first flight strip image, the first flight strip image can be divided into blocks according to the rows where the vertical coordinates of the first same-name points are located, and multiple first block images are obtained. After obtaining the positions of the N second same-name points of the second flight strip image, the second flight strip image can be divided into blocks according to the rows where the vertical coordinates of the second same-name points are located, and multiple second block images are obtained.

[0084] It can be understood that the first block image and the second block image have a one-to-one correspondence. For example, the first block image obtained by dividing the first flight strip image by the row where Ay1 is located and the row where Ay2 is located corresponds to the second block image obtained by dividing the second flight strip image by the row where By1 is located and the row where By2 is located.

[0085] S150 , taking each first block image as a reference, resampling the second block image corresponding to the first block image to obtain a resampled second block image.

[0086] The resampled second block image has the same size as the first block image.

[0087] Since linear array push-scan imaging may be squeezed and stretched, the difference between [Ay1, Ay2, Ay3, ...AyN] and the difference between [By1, By2, By3, ...ByN] are not completely equal. Therefore, the first block image of the first flight strip image can be used as a reference to resample the second block image corresponding to the second flight strip image to ensure that the resampled second block image is the same size as the first block image. For example, assuming that there are 100 pixels in the vertical direction of the first block image and 90 pixels in the vertical direction of the second block image, then the second block image can be resampled so that the number of pixels in the vertical direction of the second block image is also changed to 100, thereby ensuring that the resampled second block image is the same size as the first block image. It can be understood that the vertical coordinates of each second same-name point in the resampled second block image have also changed.

[0088] S160, for each second block image, based on the second horizontal coordinate of the second same-name point in the second block image and the first horizontal coordinate of the first same-name point in the first block image corresponding to the second block image, calculate the horizontal offset of each row of pixel points in the second block image, and perform image fusion on the second block image and the first block image based on the horizontal offset of each row of pixel points.

[0089] As an implementation of the embodiment of the present invention, S160, for each second block image, based on the second horizontal coordinate of the second same-name point in the second block image and the first horizontal coordinate of the first same-name point in the first block image corresponding to the second block image, calculating the horizontal offset of each row of pixel points in the second block image may include the following two steps:

[0090] The first step is to calculate, for each second block image, the lateral offset of the row pixel points of the row where each second like-name point is located based on the second horizontal coordinates of each second like-name point in the second block image, the first horizontal coordinates of the first like-name point corresponding to each second like-name point in the first block image, and the image width.

[0091] In the second step, for any row of pixels between two second vertically adjacent points of the same name in the second block image, the lateral offset of the row of pixels is calculated based on the lateral offset of the row of pixels in the row where the two vertically adjacent points of the same name are located, the row number of the row of pixels, and the height of the image between the two vertically adjacent points of the same name.

[0092] Specifically, the horizontal coordinates of the second like-named points of the resampled second block image, Bx1', Bx2', Bx3', ...BxN', correspond to the horizontal coordinates of the first like-named points of the first block image, Ax1, Ax2, Ax3, ..., assuming that the splicing direction of the resampled second block image is the right side of the first block image, for the row pixels of the row where Bx1' of the second block image is located, the lateral offset for alignment with the first block image is w+Bx1'-Ax1, denoted as offset_1, where w is the width of the first flight strip image or the second flight strip image. For the row pixels of the row where Bx2' is located, the lateral offset is w+Bx2'-Ax2, denoted as offset_2. Similarly, offset_1 and offset_2 are not equal when the flight strip image quality is generally good. Therefore, for the row pixels between Bx1' and Bx2', the calculation formula for the lateral offset offset_i is:

[0093] offset_i=round(offset_1+(offset_2-offset_1) / h*i).

[0094] i is the row index between Bx1' and Bx2', h is the image height between Bx1' and Bx2', and round() is rounded to an integer.

[0095] After calculating the lateral offset of the row pixel points in each second block image, the overlapping part of each second block image and its corresponding first block image can be fused based on the lateral offset of the row pixel points in each row. The fusion of adjacent flight strip images corresponding to all flight strips can achieve the fusion of all flight strip images.

[0096] As an implementation of the embodiment of the present invention, performing image fusion on the second block image and the first block image based on the lateral offset of each row of pixel points may include the following steps:

[0097] Based on the lateral offset of each row of pixel points, first pixel points overlapping with each second pixel point in the second block image are determined in the first block image.

[0098] For each second pixel point of the second block image, a linear weighted sum is performed on the pixel value of the second pixel point and the pixel value of the first pixel point overlapped with the second pixel point.

[0099] Specifically, after determining the horizontal and vertical offsets of each row of pixels in the second flight strip image, a linear weighted fusion is performed on the overlapping portion of the first flight strip image and the second flight strip image, and the weight of the overlapping portion decreases linearly from 1 to 0 from the center of the row to the edge. Of course, the weight of the overlapping portion can also be determined according to actual conditions, and the embodiment of the present invention does not specifically limit this.

[0100] The technical solution provided by the embodiment of the present invention can screen the initial same-name point pairs through a secondary screening method, and then accurately obtain the same-name point pairs of the flight strip image to be stitched, and by resampling the image to be stitched, it can cope with the possible squeezing or stretching in the flight strip image, thereby realizing the flight strip image fusion that is independent of the quality and content of the flight strip image.

[0101] In order to describe the scheme clearly, the screening process of the initial same-name points will be described in detail below.

[0102] As an implementation of an embodiment of the present invention, based on the properties of the obtained multiple points-of-the-same-name lines, the initial points-of-the-same-name pairs are screened to obtain the first points-of-the-same-name pairs, which may include the following steps:

[0103] Respectively obtain the mapping line segments of the multiple lines connecting the points of the same name in the vertical direction and the mapping line segments in the horizontal direction;

[0104] Compare the mapped line segment in the vertical direction with the preset segment length. If the mapped line segment in the vertical direction is greater than the preset segment length, filter out the corresponding initial same-name point pairs.

[0105] Compare the horizontal mapping line segment with the preset image width. If the horizontal mapping line segment is smaller than the preset image width, filter out the corresponding initial same-name point pairs.

[0106] The lengths of multiple lines connecting points of the same name are compared with the image width. If the length of the lines connecting points of the same name is greater than the image width, the corresponding initial pairs of points of the same name are filtered out.

[0107] The image width is the width of the first flight strip image or the second width of the second flight strip image, and the first width is the same as the second width.

[0108] Specifically, after obtaining the lines connecting the points of the same name, the mapped line segments of the lines connecting the points of the same name in the vertical direction can be obtained, and the mapped line segments in the vertical direction can be compared with the preset segment length. If the mapped line segments in the vertical direction are greater than the preset segment length, it means that the probability of an erroneous pair of points of the same name is high. Therefore, the erroneous pair of points of the same name will be screened out. The preset segment length can be determined based on the actual situation and is not specifically limited here.

[0109] After obtaining the same-name point connection line, the mapping line segment of the same-name point connection line in the horizontal direction can be obtained, and the mapping line segment in the horizontal direction is compared with the image width of the preset ratio, wherein the preset ratio can be 1 / 10. If the mapping line segment in the horizontal direction is smaller than the image width of the preset ratio, it means that the probability of an erroneous same-name point pair is high, and therefore, the erroneous same-name point pair is screened out. It can be seen that through this implementation, the erroneous same-name point pair can be deleted, so that the finally obtained same-name point pair is more accurate.

[0110] Based on the above embodiment, the method may further include the following steps:

[0111] Determine whether there are multiple first points with the same name within a preset number of pixel points in the longitudinal direction of the first flight strip image. If there are multiple first points with the same name, only one target first point with the same name is retained, and the other first points with the same name are deleted;

[0112] The second point of the same name in the second flight strip image that matches the first point of the same name of the target is deleted.

[0113] Specifically, in order to prevent the same-name point pairs in the flight path image from being too dense, the same-name point pairs can be screened based on the density of the same-name points in the vertical direction of the flight path image. For example, only one pair of same-name point pairs is retained every 10 pixel units in the vertical direction. This screening method is to prevent the detected same-name point pairs from being too dense.

[0114] In order to describe the solution clearly, the technical solution of the embodiment of the present invention will be described in detail below with reference to a specific example.

[0115] This solution is suitable for the stitching and fusion of UAV hyperspectral bow-shaped flight images. The specific examples are as follows:

[0116] 1. Crop the flight strip image based on the time record of the original image and the corresponding GPS track.

[0117] 2. Extract the i-th and i+1-th flight strip images, with width and height of [w, h1] and [w, h2] respectively. Divide the i+1-th flight strip image into image blocks with width and height of w (blocks with height less than w are merged with the previous block). Perform the following operations on each block image:

[0118] First, detect the feature matching points with the i-th flight strip as the initial points of the same name.

[0119] Second, the same-name point pairs are selected according to the position attribute, the length attribute, the angle attribute and the longitudinal position attribute of the line connecting the same-name points, the same-name point pairs selected for the first time are obtained, and the coordinates of the same-name point pairs are recorded.

[0120] 3. Manually screen the same-name point pairs after the initial screening. If there are incorrect same-name point pairs, delete them and add correct same-name point pairs in places where the same-name point pairs are sparse to ensure the image fusion and stitching effect. Finally, sort the same-name point pairs after the secondary screening in ascending order of the vertical coordinate.

[0121] 4. Traverse the ordinates of the same-name points, record the difference between the ordinates of the j-th and j+1-th same-name points in the i+1-th flight strip as the height h21, the height of the corresponding image block in the i-th flight strip is h11, resample the image surrounded by the j-th and j+1-th same-name points in the i+1-th flight strip with a width and height of [w, h21] to a width and height of [w, h11], and update the ordinates of the same-name points in the i+1-th flight strip.

[0122] 5. Traverse the horizontal coordinates of the same-name point pairs and calculate the offset of each row of pixels in the i+1th flight strip image according to the offset calculation formula.

[0123] 6. Adjust the position of the i+1th flight strip image row by row according to the calculated offset, and perform linear weighted fusion on the overlapping parts of the flight strip images;

[0124] 7. Repeat steps 2 to 6 for the remaining flight strips to complete the stitching and fusion of all flight strip images.

[0125] In a second aspect, an embodiment of the present invention provides a flight path image fusion device 20 based on homonymous points, such as Figure 2 As shown, the device comprises:

[0126] The flight path image acquisition module 210 is used to acquire a first flight path image and a second flight path image, wherein the first flight path image and the second flight path image are images corresponding to adjacent flight paths during the UAV's bow-shaped flight;

[0127] The same-name point pair determination module 220 is used to perform feature matching on the first flight strip image and the second flight strip image to obtain a plurality of feature point matching pairs as initial same-name point pairs, and connect the two same-name points included in each initial same-name point pair to obtain a plurality of same-name point connection lines;

[0128] A homonymous point pair screening module 230 is used to screen the initial homonymous point pairs based on the properties of the obtained multiple homonymous point lines to obtain first homonymous point pairs, and in response to a target user's screening operation on the first homonymous point pairs, add and / or delete the first homonymous point pairs to obtain second homonymous point pairs;

[0129] The flight strip image blocking module 240 is used for, for each second same-name point pair, blocking the first flight strip image based on the row where the first same-name point included in the second same-name point pair is located to obtain a plurality of first blocked images, and blocking the second flight strip image based on the row where the second same-name point included in the second same-name point pair is located to obtain a plurality of second blocked images;

[0130] An image block resampling module 250 is used to resample the second block images corresponding to each first block image based on the first block image to obtain a resampled second block image, wherein the resampled second block image has the same size as the first block image;

[0131] The flight strip image fusion module 260 is used to calculate the lateral offset of each row of pixel points in each second block image based on the second horizontal coordinate of the second same-name point in the second block image and the first horizontal coordinate of the first same-name point in the first block image corresponding to the second block image, and perform image fusion of the second block image with the first block image based on the lateral offset of each row of pixel points.

[0132] The technical solution provided by the embodiment of the present invention can screen the initial same-name point pairs through a secondary screening method, and then accurately obtain the same-name point pairs of the flight strip image to be stitched, and by resampling the image to be stitched, it can cope with the possible squeezing or stretching in the flight strip image, thereby realizing the flight strip image fusion that is independent of the quality and content of the flight strip image.

[0133] In a third aspect, an embodiment of the present invention provides an electronic device 300, such as Figure 3 As shown, including:

[0134] at least one processor 301;

[0135] a memory 302 for storing the at least one processor executable instruction;

[0136] The at least one processor is configured to execute the instructions to implement the method described in the embodiment of the present invention.

[0137] The technical solution provided by the embodiment of the present invention can screen the initial same-name point pairs through a secondary screening method, and then accurately obtain the same-name point pairs of the flight strip image to be stitched, and by resampling the image to be stitched, it can cope with the possible squeezing or stretching in the flight strip image, thereby realizing the flight strip image fusion that is independent of the quality and content of the flight strip image.

[0138] In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium. When instructions in the computer-readable storage medium are executed by a processor of an electronic device, the electronic device is enabled to execute the method described in the embodiment of the present invention.

[0139] The technical solution provided by the embodiment of the present invention can screen the initial same-name point pairs through a secondary screening method, and then accurately obtain the same-name point pairs of the flight strip image to be stitched, and by resampling the image to be stitched, it can cope with the possible squeezing or stretching in the flight strip image, thereby realizing the flight strip image fusion that is independent of the quality and content of the flight strip image.

[0140] In a fifth aspect, an embodiment of the present invention provides a computer program product, including a computer program, which implements the method described in the embodiment of the present invention when executed by a processor.

[0141] The technical solution provided by the embodiment of the present invention can screen the initial same-name point pairs through a secondary screening method, and then accurately obtain the same-name point pairs of the flight strip image to be stitched, and by resampling the image to be stitched, it can cope with the possible squeezing or stretching in the flight strip image, thereby realizing the flight strip image fusion that is independent of the quality and content of the flight strip image.

[0142] Although the embodiments of the present invention have been shown and described above, it is to be understood that the above embodiments are exemplary and are not to be construed as limitations on the present invention. A person skilled in the art may change, modify, replace and modify the above embodiments within the scope of the present invention without departing from the principles and intent of the present invention.

Claims

1. A flight strip image fusion method based on homonymous points, characterized in that: The method comprises: Acquire a first flight strip image and a second flight strip image, wherein the first flight strip image and the second flight strip image are images corresponding to adjacent flight strips during the UAV's bow-shaped flight; Performing feature matching on the first flight strip image and the second flight strip image to obtain a plurality of feature point matching pairs as initial same-name point pairs, and connecting two same-name points included in each initial same-name point pair to obtain a plurality of same-name point connecting lines; Based on the obtained properties of the plurality of same-name point lines, the initial same-name point pairs are screened to obtain first same-name point pairs, and in response to a target user's screening operation on the first same-name point pairs, the first same-name point pairs are added and / or deleted to obtain second same-name point pairs; For each second same-name point pair, the first flight strip image is divided into blocks based on the row where the first same-name point included in the second same-name point pair is located to obtain a plurality of first block images, and the second flight strip image is divided into blocks based on the row where the second same-name point included in the second same-name point pair is located to obtain a plurality of second block images; Taking each first block image as a reference, resampling the second block image corresponding to the first block image to obtain a resampled second block image, wherein the resampled second block image has the same size as the first block image; For each second block image, based on the second horizontal coordinate of the second same-name point in the second block image and the first horizontal coordinate of the first same-name point in the first block image corresponding to the second block image, the horizontal offset of each row of pixel points in the second block image is calculated, and image fusion is performed on the second block image and the first block image based on the horizontal offset of each row of pixel points.

2. The method according to claim 1, characterized in that The method of screening the initial pairs of points of the same name based on the properties of the obtained multiple lines of points of the same name to obtain the first pairs of points of the same name includes: Respectively obtain the mapping line segments in the longitudinal direction and the mapping line segments in the transverse direction of the multiple lines connecting the points of the same name; Compare the mapped line segment in the vertical direction with the preset segment length. If the mapped line segment in the vertical direction is greater than the preset segment length, filter out the corresponding initial same-name point pairs. Compare the horizontal mapping line segment with the preset ratio of the image width, and if the horizontal mapping line segment is smaller than the preset ratio of the image width, filter out the corresponding initial same-name point pairs; The lengths of the multiple lines connecting the same name points are compared with the image width respectively. If the length of the lines connecting the same name points is greater than the image width, the corresponding initial same name point pairs are filtered out. The image width is the first width of the first flight strip image or the second width of the second flight strip image, and the first width is the same as the second width.

3. The method according to claim 1, characterized in that The method further comprises: Determine whether there are multiple first points with the same name within a preset number of pixel points in the longitudinal direction of the first flight strip image; if there are multiple first points with the same name, retain only one target first point with the same name and delete the other first points with the same name; The second point of the same name in the second flight strip image that matches the first point of the same name of the target is deleted.

4. The method according to claim 1, characterized in that: The method further comprises: Displaying the first flight strip image, the second flight strip image, and the first pair of points with the same name in a visual display interface; Correspondingly, in response to the target user's screening operation on the first same-name point pairs, the first same-name point pairs are added and / or deleted to obtain second same-name point pairs, including: receiving a screening operation input by a target user through a visual interface, wherein the screening operation includes deleting incorrect same-name point pairs in the first same-name point pairs, and / or adding correct same-name point pairs in the first same-name point pairs; If the screening operation is to delete incorrect same-name point pairs in the first same-name point pairs, in response to the screening operation, delete incorrect same-name point pairs in the first same-name point pairs; If the screening operation is to add a correct pair of same-named points to the first pair of same-named points, in response to the screening operation, the correct pair of same-named points is added to the first pair of same-named points.

5. The method according to any one of claims 1 to 4, characterized in that: The step of calculating, for each second block image, a lateral offset of each row of pixel points in the second block image based on a second horizontal coordinate of a second same-name point in the second block image and a first horizontal coordinate of a first same-name point in a first block image corresponding to the second block image, comprises: For each second block image, based on the second horizontal coordinates of the second like-named points in the second block image, the first horizontal coordinates of the first like-named points corresponding to the second like-named points in the first block image, and the image width, calculate the horizontal offset of the row pixel points of the row where the second like-named points are located; For any row of pixels between two second vertically adjacent points of the same name in the second block image, the lateral offset of the row of pixels is calculated based on the lateral offset of the row of pixels in which the two vertically adjacent points of the same name are located, the row number of the row of pixels, and the height of the image between the two vertically adjacent points of the same name.

6. The method according to any one of claims 1 to 4, characterized in that: The performing image fusion on the second block image and the first block image based on the lateral offset of each row of pixel points includes: Based on the lateral offset of each row of pixel points, determining, in the first block image, first pixel points that overlap with each second pixel point in the second block image; For each second pixel point of the second block image, a linear weighted sum is performed on a pixel value of the second pixel point and a pixel value of a first pixel point overlapping with the second pixel point.

7. A flight strip image fusion device based on homonymous points, characterized in that: The device comprises: A flight strip image acquisition module, used to acquire a first flight strip image and a second flight strip image, wherein the first flight strip image and the second flight strip image are images corresponding to adjacent flight strips during the UAV's bow-shaped flight; A same-name point pair determination module is used to perform feature matching on the first flight strip image and the second flight strip image to obtain a plurality of feature point matching pairs as initial same-name point pairs, and connect the two same-name points included in each initial same-name point pair to obtain a plurality of same-name point connection lines; a homonymous point pair screening module, configured to screen the initial homonymous point pairs based on the properties of the obtained plurality of homonymous point lines to obtain first homonymous point pairs, and in response to a target user's screening operation on the first homonymous point pairs, add and / or delete the first homonymous point pairs to obtain second homonymous point pairs; A flight strip image blocking module is used for, for each second same-name point pair, blocking the first flight strip image based on the row where the first same-name point included in the second same-name point pair is located to obtain a plurality of first blocked images, and blocking the second flight strip image based on the row where the second same-name point included in the second same-name point pair is located to obtain a plurality of second blocked images; An image block resampling module, used for resampling a second block image corresponding to each first block image based on the first block image to obtain a resampled second block image, wherein the resampled second block image has the same size as the first block image; The flight strip image fusion module is used to calculate the lateral offset of each row of pixel points in each second block image based on the second horizontal coordinate of the second same-name point in the second block image and the first horizontal coordinate of the first same-name point in the first block image corresponding to the second block image, and perform image fusion of the second block image with the first block image based on the lateral offset of each row of pixel points.

8. An electronic device, characterized in that: include: at least one processor; a memory for storing the at least one processor-executable instruction; The at least one processor is configured to execute the instructions to implement the method according to any one of claims 1-6.

9. A computer-readable storage medium, characterized in that: When the instructions in the computer-readable storage medium are executed by a processor of an electronic device, the electronic device is enabled to execute the method according to any one of claims 1 to 6.

10. A computer program product, characterized in that The method comprises a computer program, which implements the method according to any one of claims 1 to 6 when being executed by a processor.

Images