Image stitching method, device, electronic device and storage medium

Abstract

The present application provides an image stitching method, device, electronic device and storage medium, and relates to the field of image processing technology. The method comprises: obtaining a target image to be stitched; determining the target POS data of the target image according to the same-name point data of each reference image in the benchmark stitched image obtained by stitching the images before the target image; stitching the target image into the benchmark stitched image according to the target POS data of the target image to obtain a new benchmark stitched image. In this scheme, since the stitching of the target images is based on the previously stitched benchmark stitched images, the integrity is fully considered, and on this basis, the image stitching is achieved by local optimization. In this way, it can be ensured that the increase in stitching error will be relatively small, and as the number of images increases, the efficiency of calculating the target POS data remains basically unchanged, thereby significantly improving the stitching effect between flight routes.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to an image stitching method, device, electronic device and storage medium. Background Art

[0002] As a new remote sensing platform, UAVs are flexible, low-cost, high-resolution, high-efficiency, simple to operate and maintain. Compared with conventional aerial photogrammetry methods, they are more adaptable in densely built areas, dangerous work areas, hilly areas with complex terrain, and foggy areas. Their application scope is also continuously expanding, and they are playing an increasingly important role in national defense, environmental protection, and disaster relief. Due to the large number of UAV remote sensing images, large and irregular inclination angles, irregular heading overlap, and large coordinate position errors, UAV image matching is difficult, slow, and low-precision, which affects a series of problems in the subsequent processing of images. It is crucial to develop an accurate and efficient method for stitching, processing, and mapping UAV remote sensing images.

[0003] Existing image stitching algorithms mainly perform image stitching based on the relationship between adjacent images by matching features between adjacent images.

[0004] However, using the existing methods, as the number of images increases, the accumulated error will also increase, and the stitching effect will be poor. Summary of the invention

[0005] In order to solve the problems existing in the prior art, the present application provides an image stitching method, device, electronic device and storage medium.

[0006] To achieve the above purpose, the technical solution adopted in this application is:

[0007] The first aspect of the present application provides an image stitching method, comprising:

[0008] Acquire a target image to be stitched, where the target image is an image after the Nth frame of the image captured by the shooting device, where N is an integer greater than 1;

[0009] Determine the target POS data of the target image according to the same-name point data of each reference image in the reference stitched image obtained by stitching the images before the target image, wherein the reference image is an image in the reference stitched image having the same-name point number as the target image greater than a preset threshold;

[0010] According to the target POS data of the target image, the target image is spliced ​​into the reference spliced ​​image to obtain a new reference spliced ​​image.

[0011] Optionally, determining the target POS data of the target image according to the same-name point data of each reference image in the reference stitched image obtained by stitching each image before the target image comprises:

[0012] Determine the initial POS data of the target image according to the same-name point data of each reference image;

[0013] The initial POS data of the target image is optimized using a preset optimization algorithm to obtain target POS data of the target image.

[0014] Optionally, determining the initial POS data of the target image according to the same-name point data of each reference image includes:

[0015] The initial POS data of the target image is determined according to the same-name point data of each reference image and the model coordinates of each same-name point in each reference image.

[0016] Optionally, before determining the target POS data of the target image according to the same-name point data of each reference image in the reference spliced ​​image obtained by splicing each image before the target image, the method further includes:

[0017] Matching the first N frames of images among the images captured by the shooting device;

[0018] According to the matching result, the first N frames of images are spliced ​​to form an initial reference spliced ​​image.

[0019] Optionally, the step of stitching the first N frames of images to form an initial reference stitched image according to the matching result includes:

[0020] Selecting a reference image pair from the first N frames of images, wherein the two frames of images in the reference image pair are the two frames of images with the largest number of points with the same name among the first N frames of images;

[0021] Obtaining relative POS data of each remaining image in the first N frames of images according to the same-name point data of the reference image pair and the model coordinates of the same-name points;

[0022] Determine the POS data of each image in the first N frames of images according to the relative POS data of each image in the first N frames of images and the position information of each image when the shooting device shoots the image;

[0023] The first N frames of images are spliced ​​according to the POS data of each image in the first N frames of images to obtain an initial reference spliced ​​image.

[0024] Optionally, determining the POS data of each image in the first N frames of images according to the relative POS data of each image in the first N frames of images and the position information of each image when the shooting device shoots the image includes:

[0025] The POS data of each image in the first N frames of images are determined according to the POS data of each remaining image and the global positioning system data when the photographing device photographs the remaining images.

[0026] Optionally, before obtaining the relative POS data of each remaining image in the first N frames of images according to the same-name point data and the model coordinates of the same-name points of the reference image pair, the method further includes:

[0027] Based on each image in the reference image pair, relative orientation is performed to obtain relative POS data and model coordinates of the same-name points of each image in the reference image pair.

[0028] A second aspect of the present application provides an image stitching device, comprising: an acquisition unit, a determination unit, and a stitching unit;

[0029] The acquisition unit is used to acquire a target image to be stitched, where the target image is an image after the Nth frame of the image captured by the shooting device, where N is an integer greater than 1;

[0030] The determination unit is used to determine the target POS data of the target image according to the same-name point data of each reference image in the reference stitched image obtained by stitching each image before the target image, wherein the reference image is an image in the reference stitched image having the same-name point number as the target image greater than a preset threshold;

[0031] The stitching unit is used to stitch the target image into the reference stitching image according to the target POS data of the target image to obtain a new reference stitching image.

[0032] Optionally, the determination unit is used to determine the initial POS data of the target image based on the same-name point data of each reference image; and use a preset optimization algorithm to optimize the initial POS data of the target image to obtain the target POS data of the target image.

[0033] Optionally, the determining unit is used to determine the initial POS data of the target image according to the same-name point data of each reference image and the model coordinates of each same-name point in each reference image.

[0034] Optionally, the determining unit is used to match first N frames of images among the images captured by the capturing device; and according to the matching result, splice the first N frames of images to form an initial reference spliced ​​image.

[0035] Optionally, the determining unit is used to select a reference image pair from the first N frames of images, where the two frames of images in the reference image pair are the two frames of images with the largest number of points with the same name among the first N frames of images;

[0036] Obtaining relative POS data of each remaining image in the first N frames of images according to the same-name point data of the reference image pair and the model coordinates of the same-name points;

[0037] Determine the POS data of each image in the first N frames of images according to the relative POS data of each image in the first N frames of images and the position information of each image when the shooting device shoots the image;

[0038] The first N frames of images are spliced ​​according to the POS data of each image in the first N frames of images to obtain an initial reference spliced ​​image.

[0039] Optionally, the determining unit is used to determine the POS data of each image in the first N frames of images according to the POS data of each remaining image and the global positioning system data when the shooting device shoots the remaining images.

[0040] Optionally, the acquisition unit is further used to perform relative orientation based on each image in the reference image pair to obtain relative POS data and model coordinates of the same-name points of each image in the reference image pair.

[0041] The third aspect of the present application provides an electronic device, comprising: a processor, a storage medium and a bus, wherein the storage medium stores machine-readable instructions executable by the processor, and when the electronic device is running, the processor and the storage medium communicate through the bus, and the processor executes the machine-readable instructions to perform the steps of the method described in the first aspect above.

[0042] A fourth aspect of the present application provides a storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the method described in the first aspect are executed.

[0043] The present application provides an image stitching method, device, electronic device and storage medium. The image stitching method includes: obtaining a target image to be stitched, wherein the target image is an image after the Nth frame of the image captured by the shooting device, where N is an integer greater than 1; determining the target POS data of the target image according to the same-name point data of each reference image in the benchmark stitched image obtained by stitching the images before the target image, wherein the reference image is an image in the benchmark stitched image whose number of same-name points as the target image is greater than a preset threshold; stitching the target image into the benchmark stitched image according to the target POS data of the target image to obtain a new benchmark stitched image. In this solution, by obtaining the same-name point data of each reference image before the target image to be stitched, the same-name point data of the reference image is used to determine the target POS data of the target image, so as to stitch the target image into the benchmark stitched image to obtain a new benchmark stitched image. Since the stitching of target images is based on the previously stitched reference stitching images, the integrity is fully considered, and on this basis, the image stitching is achieved through local optimization. In this way, the increase in stitching error can be guaranteed to be relatively small, and as the number of images increases, the efficiency of calculating the target POS data remains basically unchanged, thereby significantly improving the stitching effect between flight routes. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0045] Figure 1 A schematic diagram of the flow of an existing image stitching method provided in an embodiment of the present application;

[0046] Figure 2 A schematic diagram of a flow chart of an image stitching method provided in one embodiment of the present application;

[0047] Figure 3 A schematic diagram of a flow chart of an image stitching method provided in another embodiment of the present application;

[0048] Figure 4 A schematic diagram of a flow chart of an image stitching method provided in another embodiment of the present application;

[0049] Figure 5 A schematic diagram of a flow chart of an image stitching method provided in another embodiment of the present application;

[0050] Figure 6A schematic diagram of an image stitching device provided in one embodiment of the present application;

[0051] Figure 7 A schematic diagram of the structure of an electronic device provided in one embodiment of the present application. DETAILED DESCRIPTION

[0052] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. The components of the embodiments of the present application described and shown in the drawings here can be arranged and designed in various different configurations.

[0053] Therefore, the following detailed description of the embodiments of the present application provided in the accompanying drawings is not intended to limit the scope of the present application for which protection is sought, but merely represents selected embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without creative work are within the scope of protection of the present application.

[0054] It should be noted that similar reference numerals and letters denote similar items in the following drawings, and therefore, once an item is defined in one drawing, further definition and explanation thereof is not required in subsequent drawings.

[0055] In the description of the present application, it should be noted that if the terms "upper", "lower", "inside", "outside", etc. appear to indicate an orientation or position relationship, they are based on the orientation or position relationship shown in the drawings, or are the orientation or position relationship in which the product of the application is usually placed when used. This is only for the convenience of describing the present application and simplifying the description, and does not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on the present application.

[0056] In addition, the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0057] It should be noted that, in the absence of conflict, the features in the embodiments of the present application may be combined with each other.

[0058] Existing image stitching algorithms mainly stitch images based on the relationship between adjacent images. Figure 1 A schematic diagram of the flow of an existing image stitching method provided in an embodiment of the present application is shown in FIG. Figure 1 As shown, in the existing processing method, firstly, step S10 is executed to input the image sequence, and after the image sequence is obtained, step S11 is executed to match the features between adjacent images, and according to the matching relationship between adjacent images, step S12 is executed to calculate the relative relationship between images, and the image stitching result is determined by using the relative relationship between adjacent images, and step S13 is executed to stitch the images. In the above-mentioned image stitching scheme, only the relative relationship between adjacent images is calculated each time, which lacks integrity. Therefore, as the number of images increases, the accumulated error of image stitching will also increase, resulting in poor stitching effect.

[0059] In order to solve the technical problems existing in the prior art, the present application provides an inventive concept: based on the reference image with the largest number of points of the same name as the target image in the reference stitched image that has been stitched before the target image to be stitched, the target image is stitched into the already stitched reference stitched image. Since the stitching of each frame of the image is based on the previously stitched reference stitched image, the integrity is fully considered, and on this basis, the image stitching is achieved by local optimization. In this way, the increase in stitching error can be guaranteed to be relatively small, and as the number of images increases, the efficiency of calculating the target POS data remains basically unchanged, thereby significantly improving the stitching effect between flight routes.

[0060] The specific technical solution provided in this application is described below through possible implementation methods.

[0061] The implementation principle of the image stitching method provided in the present application and the corresponding beneficial effects will be explained through multiple specific embodiments as follows.

[0062] Optionally, the present application can be applied to the stitching scene of images taken by drones in real time. For example, the drone takes images in real time, and the electronic device communicating with the drone obtains these images in real time accordingly, and uses the solution of the present application to perform image stitching in real time. For another example, after the drone completes taking images, the electronic device obtains the images taken this time in batches, and uses the solution of the present application to complete the stitching of these images. It should be understood that the solution of the present application is not limited to the application in the above-mentioned drone image stitching scene, but can also be applied to other image stitching scenes.

[0063] Figure 2This is a flow chart of an image stitching method provided in an embodiment of the present application. The execution subject of the method may be a computer, a server or other electronic device. Figure 2 As shown, the method may include:

[0064] S101, obtaining a target image to be stitched.

[0065] For example, in the embodiment of the present application, the target image to be stitched may be the current image data acquired by the aerial photography function of the drone, or the current image shot by the camera on the rotating bracket. It may also be the current image that needs to be processed when stitching all the shooting data after shooting is completed.

[0066] The present application does not limit the type of target images to be stitched. For ease of description, the following embodiments are described using drone aerial images as examples.

[0067] Of course, the above are only several exemplary application scenarios, and actual applications are not limited to the above application scenarios.

[0068] It should be noted that the present application is used to complete the stitching of multiple frames of images, and the present embodiment describes the stitching process of any frame of images after the Nth frame of image, that is, the present embodiment is applicable to all images after the Nth frame of image. The stitching process of the Nth frame and images before it will be described in the following embodiments.

[0069] In an embodiment, the target image is an image after the Nth frame of the image captured by the capturing device, where N is an integer greater than 1.

[0070] Specifically, the number N can be determined according to the hardware processing level, the shooting time, and the storage size of the image data. For example, the value of N can be 5. In the embodiment of the present application, the value of N is not limited.

[0071] S102 , determining target POS data of the target image according to the same-name point data of each reference image in the reference spliced ​​image obtained by splicing each image before the target image.

[0072] In an embodiment of the present application, the target POS data of the target image is determined using the data of the same-name points of each reference image. It should be noted that the reference image is obtained in a preset manner from a reference stitched image obtained by stitching together the images before the target image. Optionally, the reference image may be image data in the reference stitched image before the target image, whose correlation with the target image exceeds a set threshold. Among them, POS data refers to the data of the Position and Orientation System, and the POS data includes position information and posture information. Optionally, in an embodiment of the present application, the POS data is the position information when the image is shot.

[0073] In one implementation, the reference image may be an image in the reference stitched image in which the number of points with the same name as the target image is greater than a preset threshold. Points with the same name are also called corresponding points, and refer to the image points of the target points in the images taken by any drone on different images. The number of points with the same name is generally determined by performing a scale-invariant feature transform (Sift) on the image before the current image to obtain feature points. The acquired feature points are matched between the two images, and the number of feature points that can be matched is the number of points with the same name.

[0074] In the embodiment of the present application, the target POS data of the target image is obtained by using a reference image, thereby avoiding the problem of error accumulation caused by calculating the relative relationship between adjacent images and reducing the influence of error accumulation on the stitching effect.

[0075] S103 . According to the target POS data of the target image, stitch the target image into the reference stitched image to obtain a new reference stitched image.

[0076] Optionally, the target POS data of the target image can be used to calculate the stitching matrix H of the target image, and the image can be mosaicked and cropped using the stitching matrix H, and the target image can be stitched into the reference stitching image to obtain a new reference stitching image. The target POS data is 3*4 matrix data.

[0077] Specifically, the calculation process of the splicing matrix H is as follows:

[0078] H=K -1 [K(R-RTV T )K -1 ] T (1-1)

[0079] Among them, K is the camera intrinsic parameter matrix, R is the image corresponding rotation matrix (corresponding to the first column of its matrix of the target POS data), T is the image exterior orientation line element (corresponding to the last three columns of its matrix of the target POS data), and V is the ground normal vector.

[0080] The embodiment of the present application provides an image stitching method, including: obtaining a target image to be stitched, the target image being an image after the Nth frame of the image captured by the shooting device, where N is an integer greater than 1; determining target POS data of the target image based on the same-name point data of each reference image in a benchmark stitched image obtained by stitching images before the target image, the reference image being an image in the benchmark stitched image whose number of same-name points as the target image is greater than a preset threshold; stitching the target image into the benchmark stitched image based on the target POS data of the target image to obtain a new benchmark stitched image. In this solution, by obtaining the same-name point data of each reference image before the target image to be stitched, the same-name point data of the reference image is used to determine the target POS data of the target image, so as to stitch the target image into the benchmark stitched image to obtain a new benchmark stitched image. Since the stitching of target images is based on the previously stitched reference stitching images, the integrity is fully considered, and on this basis, the image stitching is achieved through local optimization. In this way, the increase in stitching error can be guaranteed to be relatively small, and as the number of images increases, the efficiency of calculating the target POS data remains basically unchanged, thereby significantly improving the stitching effect between flight routes.

[0081] Figure 3 A schematic diagram of a flow chart of an image stitching method provided in another embodiment of the present application is shown in FIG. Figure 3 As shown, step S102 may specifically include:

[0082] S201 : Determine initial POS data of a target image based on the same-name point data of each reference image.

[0083] S202: Use a preset optimization algorithm to optimize the initial POS data of the target image to obtain target POS data of the target image.

[0084] In the embodiment of the present application, the POS data of the target image before optimization, ie, the initial POS data, is determined by the same-name point data of each reference image related to the target image.

[0085] The initial POS data is optimized by using an optimization algorithm to obtain target POS data. Specifically, the initial POS data can be optimized by using a bundle adjustment algorithm (BA for short).

[0086] Optionally, determining the initial POS data of the target image according to the same-name point data of each reference image includes:

[0087] The initial POS data of the target image is determined according to the same-name point data of each reference image and the model coordinates of each same-name point in each reference image.

[0088] It should be noted that, in the embodiment of the present application, the model coordinates are relative coordinates determined by the current image based on its previous frame image.

[0089] Optionally, in an embodiment of the present application, images before the target image whose number of points with the same name as the current image exceeds a preset number are grouped together to form a reference image.

[0090] The initial POS data p of the target image is calculated by applying formula 1-2 through the same-name point data of each reference image and the model coordinates of each same-name point in each reference image. Formula 1-2 is as follows:

[0091]

[0092] Among them, (x, y) is the coordinates of the same-name points of the reference image pair in the reference image, (X, Y, Z) is the model coordinates of each same-name point in the reference image, and P is the initial POS data of the target image to be obtained.

[0093] The following is an explanation of the stitching process of the first N frames of images.

[0094] Figure 4 A schematic diagram of a flow chart of an image stitching method provided in another embodiment of the present application is shown in FIG. Figure 4 As shown, before step S102, the method further includes:

[0095] S301, matching first N frames of images captured by a shooting device.

[0096] S302 : According to the matching result, the first N frames of images are stitched together to form an initial reference stitched image.

[0097] In the embodiment of the present application, in order to obtain an image stitching reference, after obtaining the N+1 frame image, the previous N frames of image are stitched to obtain a reference image.

[0098] For example, in the embodiment of the present application, the result obtained by stitching the first five frames of images can be used as a reference stitching image. It should be noted that in this implementation, when the first five frames of images are used as reference stitching images, it must be ensured that there are 3 degree matching points on each image. In addition, the above limitation of N is only exemplary, and this embodiment does not specifically limit the setting of the number of reference stitching images N.

[0099] The determined first N frames of images are subjected to feature extraction and feature matching, and the first N frames of images are spliced ​​according to the acquired feature matching relationship to obtain a reference image.

[0100] Figure 5 A schematic diagram of a flow chart of an image stitching method provided in another embodiment of the present application is shown in FIG. Figure 5 As shown, step S302 may specifically include:

[0101] S401: Select a reference image pair from the previous N frames of images.

[0102] It should be noted that, in the embodiment of the present application, the two frames of images in the reference image pair are the two frames of images having the largest number of points with the same name among the first N frames of images.

[0103] Optionally, in an embodiment of the present application, feature extraction is performed on all images of the first N frames, and based on the extracted image features, feature matching is performed on all images of the first N frames. For example, taking N as 5, after extracting the features of the first 5 frames of images, matching relationships of 1-2, 1-3, 1-4, 1-5, 2-3, 2-4, 2-5, 3-4, 3-5, and 4-5 are formed, and a pair of matching relationships with the most points of the same name in the matching relationships is obtained as the reference image pair.

[0104] S402: Obtain relative POS data of each remaining image in the first N frames of images according to the same-name point data of the reference image pair and the model coordinates of the same-name points.

[0105] In the embodiment of the present application, the relative POS data of each remaining image in the first N frames of images are calculated using the model coordinates of the reference image pair, which may specifically be the model coordinates corresponding to the same-name points of the reference image pair, using preset formulas 1-3.

[0106] The preset formula 1-3 is as follows:

[0107]

[0108] (x, y) is the coordinates corresponding to the same-name points of the reference image pair, (X, Y, Z) is the model coordinates corresponding to the same-name points of the reference image pair, and P is the relative POS data of the remaining images.

[0109] It should be noted that P is a continuously updated process. For example, after the relative POS data of the first image in the remaining images is calculated through the reference image pair, the POS data can be used to update the same-name points of the reference image pair to obtain the relative POS data of the next remaining image, and the updating process is repeated until the relative POS data of all the remaining images are obtained.

[0110] S403 , determining the POS data of each image in the first N frames of images according to the relative POS data of each image in the first N frames of images and the position information of each image when the shooting device shoots each image.

[0111] In the embodiment of the present application, the POS data of each image in the first N frames is obtained by using the auxiliary aerotriangulation algorithm according to the relative POS data of all images in the first N frames and the position information of the camera when taking each image. The position information of the camera when taking each image includes (X, Y, Z) information, X and Y are used together to represent the position information of the camera on the horizontal plane, and Z is used to represent the height information of the camera, and the position information of the camera when taking each image is known data.

[0112] S404 , stitching the first N frames of images according to the POS data of each image in the first N frames of images to obtain an initial reference stitching image.

[0113] Using the POS data of each image in the first N frames, the stitching matrix H of each image is calculated by the above formula 1-1. The coordinates of each pixel in the image are calculated into the same coordinate system through the stitching matrix H, and then the image is mosaicked and cropped to obtain the initial reference stitching image.

[0114] Optionally, determining the POS data of each image in the first N frames of images according to the relative POS data of each image in the first N frames of images and the position information of each image when the shooting device shoots the image includes:

[0115] The POS data of each image in the first N frames of images are determined according to the POS data of each remaining image and the Global Positioning System (GPS) data when the photographing device photographs the remaining images.

[0116] In the embodiment of the present application, the POS data of the remaining images calculated using formulas 1-3 and the GPS data when the shooting device shoots the remaining images are used to determine the POS data of each image in the first N frames of images through an auxiliary aerial triangulation algorithm.

[0117] Optionally, before obtaining the relative POS data of each remaining image in the first N frames of images according to the same-name point data of the reference image pair and the model coordinates of the same-name point, the method further includes:

[0118] Based on each image in the reference image pair, relative orientation is performed to obtain relative POS data of each image in the reference image pair and model coordinates of the same-name points.

[0119] In the embodiment of the present application, since the relative POS data of each image in the reference image pair and the model coordinates of the same-name points are required when calculating the relative POS data of each remaining image in the first N frames of images, before obtaining the relative POS data of each remaining image in the first N frames of images, the relative POS data of each image in the reference image pair and the model coordinates of the same-name points are also required to be calculated.

[0120] Specifically, the acquired reference image pair may be subjected to relative orientation processing to obtain relative POS data of each image in the reference image pair and model coordinates of the same-name points.

[0121] The following describes the device and storage medium corresponding to the image stitching method provided in this application. The specific implementation process and technical effects are as mentioned above and will not be repeated below.

[0122] Figure 6 A schematic diagram of an image stitching device provided in an embodiment of the present application, such as Figure 6 As shown, the device may include: an acquisition unit 501, a determination unit 502, and a splicing unit 503;

[0123] An acquisition unit 501 is used to acquire a target image to be stitched, where the target image is an image after the Nth frame of the image captured by the shooting device, where N is an integer greater than 1;

[0124] A determination unit 502 is used to determine target POS data of a target image based on the same-name point data of each reference image in a reference stitched image obtained by stitching images before the target image, wherein the reference image is an image in the reference stitched image having the same-name point number as the target image greater than a preset threshold;

[0125] The stitching unit 503 is used to stitch the target image into the reference stitching image according to the target POS data of the target image to obtain a new reference stitching image.

[0126] Optionally, the determination unit 502 is used to determine the initial POS data of the target image according to the same-name point data of each reference image; and optimize the initial POS data of the target image using a preset optimization algorithm to obtain the target POS data of the target image.

[0127] Optionally, the determining unit 502 is configured to determine the initial POS data of the target image according to the same-name point data of each reference image and the model coordinates of each same-name point in each reference image.

[0128] Optionally, the determination unit 502 is configured to match first N frames of images among the images captured by the capturing device; and according to the matching result, splice the first N frames of images to form an initial reference spliced ​​image.

[0129] Optionally, the determination unit 502 is configured to select a reference image pair from the first N image frames, wherein the two image frames in the reference image pair are the two image frames having the largest number of points with the same name in the first N image frames;

[0130] According to the same-name point data of the reference image pair and the model coordinates of the same-name point, the relative POS data of each remaining image in the first N frames of images are obtained;

[0131] Determine the POS data of each image in the previous N frames of images according to the relative POS data of each image in the previous N frames of images and the position information of each image when the shooting device shoots the image;

[0132] According to the POS data of each image in the first N frames of images, the first N frames of images are spliced ​​to obtain an initial reference spliced ​​image.

[0133] Optionally, the determination unit 502 is used to determine the POS data of each image in the first N frames of images according to the POS data of each remaining image and the GPS data when the shooting device shoots the remaining images.

[0134] Optionally, the acquisition unit 501 is further configured to perform relative orientation based on each image in the reference image pair, and obtain relative POS data of each image in the reference image pair and model coordinates of the same-name points.

[0135] Figure 7 The structural diagram of the electronic device provided in the embodiment of the present application includes: a processor 710, a storage medium 720 and a bus 730, the storage medium 720 stores machine-readable instructions executable by the processor 710, when the electronic device is running, the processor 710 and the storage medium 720 communicate through the bus 730, and the processor 710 executes the machine-readable instructions to perform the steps of the above method embodiment. The specific implementation method and technical effect are similar and will not be repeated here.

[0136] An embodiment of the present application provides a storage medium having a computer program stored thereon. When the computer program is executed by a processor, the above method is executed.

[0137] In the several embodiments provided by the present invention, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0138] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0139] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of hardware plus software functional units.

[0140] The above-mentioned integrated unit implemented in the form of a software functional unit can be stored in a computer-readable storage medium. The above-mentioned software functional unit is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor (English: processor) to perform some steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (English: Read-Only Memory, abbreviated: ROM), random access memory (English: Random Access Memory, abbreviated: RAM), disk or optical disk and other media that can store program codes.

[0141] The above are only preferred embodiments of the present invention and are not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. An image stitching method, characterized in that: include: Acquire a target image to be stitched, where the target image is an image after the Nth frame of the image captured by the shooting device, where N is an integer greater than 1; Determine the target POS data of the target image according to the same-name point data of each reference image in the reference stitched image obtained by stitching the images before the target image, wherein the reference image is an image in the reference stitched image having the same-name point number as the target image greater than a preset threshold; According to the target POS data of the target image, the target image is spliced ​​into the reference spliced ​​image to obtain a new reference spliced ​​image.

2. The method according to claim 1, characterized in that Determining target POS data of the target image based on the same-name point data of each reference image in the reference spliced ​​image obtained by splicing each image before the target image comprises: Determine the initial POS data of the target image according to the same-name point data of each reference image; The initial POS data of the target image is optimized using a preset optimization algorithm to obtain target POS data of the target image.

3. The method according to claim 2, characterized in that Determining the initial POS data of the target image according to the same-name point data of each reference image includes: The initial POS data of the target image is determined according to the same-name point data of each reference image and the model coordinates of each same-name point in each reference image.

4. The method according to any one of claims 1 to 3, characterized in that: Before determining the target POS data of the target image based on the same-name point data of each reference image in the reference spliced ​​image obtained by splicing each image before the target image, the method further includes: Matching the first N frames of images among the images captured by the shooting device; According to the matching result, the first N frames of images are spliced ​​to form an initial reference spliced ​​image.

5. The method according to claim 4, characterized in that The step of stitching the first N frames of images to form an initial reference stitched image according to the matching result includes: Selecting a reference image pair from the first N frames of images, wherein the two frames of images in the reference image pair are the two frames of images with the largest number of points with the same name among the first N frames of images; Obtaining relative POS data of each remaining image in the first N frames of images according to the same-name point data of the reference image pair and the model coordinates of the same-name points; Determine the POS data of each image in the first N frames of images according to the relative POS data of each image in the first N frames of images and the position information of each image when the shooting device shoots the image; The first N frames of images are spliced ​​according to the POS data of each image in the first N frames of images to obtain an initial reference spliced ​​image.

6. The method according to claim 5, characterized in that The determining of the POS data of each image in the first N frames of images according to the relative POS data of each image in the first N frames of images and the position information of each image when the shooting device shoots the image comprises: The POS data of each image in the first N frames of images are determined according to the POS data of each remaining image and the global positioning system data when the photographing device photographs the remaining images.

7. The method according to claim 5, characterized in that Before obtaining the relative POS data of each remaining image in the first N frames of images according to the same-name point data and the model coordinates of the same-name points of the reference image pair, the method further includes: Based on each image in the reference image pair, relative orientation is performed to obtain relative POS data and model coordinates of the same-name points of each image in the reference image pair.

8. An image stitching device, characterized in that: include: Acquiring units, determining units, and splicing units; The acquisition unit is used to acquire a target image to be stitched, where the target image is an image after the Nth frame of the image captured by the shooting device, where N is an integer greater than 1; The determination unit is used to determine the target POS data of the target image according to the same-name point data of each reference image in the reference stitched image obtained by stitching each image before the target image, wherein the reference image is an image in the reference stitched image having the same-name point number as the target image greater than a preset threshold; The stitching unit is used to stitch the target image into the reference stitching image according to the target POS data of the target image to obtain a new reference stitching image.

9. An electronic device, characterized in that: include: A processor, a storage medium and a bus, wherein the storage medium stores machine-readable instructions executable by the processor, and when the electronic device is running, the processor and the storage medium communicate via the bus, and the processor executes the machine-readable instructions to perform the steps of the method as described in any one of claims 1 to 7.

10. A storage medium, characterized in that: The storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are executed.

Images