Image stitching method, device, processing equipment and storage medium
The method creates aligned 'bottom' images for each input image to stitch them losslessly, addressing data alteration issues in existing methods, ensuring accurate and complete stitching for low-contrast images.
Patent Information
- Application Number
- CN202210274308.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-03-18
- Publication Date
- 2025-07-15
- Estimated Expiration
- 2042-03-18
AI Technical Summary
In the prior art, the image stitching method has high requirements for stitching images, which can easily lead to local data tampering and loss, especially in image scenes with single color and low contrast, which cannot be effectively stitched.
By creating a base map and extracting alignment points, using feature points to stitch the set image to avoid scaling, distorting or blurring of local images, ensuring the integrity and accuracy of the image.
Lossless image stitching is realized, data integrity and accuracy of local images are maintained, and stitching efficiency and effect are improved.
Smart Images
Figure CN114926328B_ABST
Abstract
Description
Technical Field
[0001] This application generally relates to the field of image processing technology, and particularly to an image stitching method, apparatus, processing device, and storage medium. Background Art
[0002] Image processing is an act of using a computer to process image information to meet the visual psychology of people or application requirements. Among them, image stitching is a basic step for further image understanding, and its application scenarios are gradually becoming widespread, such as drone aerial photography, remote sensing images, medical diagnosis, etc. The quality of the image stitching directly affects the subsequent work.
[0003] In the related art, an image stitching algorithm based on SIFT feature detection is adopted. Specifically, a clustering algorithm is used to screen matching points to reduce the false matching rate, and then the stitching line is determined by connecting the feature point pairs to achieve the purpose of seamlessly stitching two images; at the same time, during the stitching process, in order to make the stitched image have no obvious stitching traces, the local image to be stitched will be scaled, distorted, or blurred to a certain extent.
[0004] In the above related art, the image stitching method has high requirements for the local image to be stitched, and the processing of the image to be stitched is likely to cause tampering and loss of local data. Summary of the Invention
[0005] In view of the above-mentioned defects or deficiencies in the prior art, it is desirable to provide an image stitching method, processing device, and storage medium. By creating a base map and extracting the alignment points of the images to be stitched, the lossless stitching of two local images to be stitched is achieved by using the created base map and the extracted alignment points.
[0006] In a first aspect, an embodiment of the present application provides an image stitching method, which includes:
[0007] Obtain the feature points of the first local image and the second local image to be stitched;
[0008] Match the obtained feature points to obtain a set of matching feature point pairs in the first local image and the second local image;
[0009] Create a base map of the first local image and the second local image, and respectively determine the alignment points of the first local image and the second local image from the set of feature point pairs;
[0010] Based on the base map, the alignment points, and the set of feature point pairs, stitch the first local image and the second local image to obtain a stitched image.
[0011] Optionally, for the image stitching method according to an embodiment of the present application, the base map of the first partial image is the first base map, the base map of the second partial image is the second base map, the alignment point of the first partial image is the first alignment point, and the alignment point of the second partial image is the second alignment point.
[0012] Then, based on the base map, the alignment point, and the set of feature point pairs, stitching the first partial image and the second partial image to obtain the stitched image includes:
[0013] Transfer the first partial image onto the first base map respectively, and transfer the second partial image onto the second base map, such that the first alignment point is aligned with the center point of the first base map, and the second alignment point is aligned with the center point of the second base map.
[0014] Calculate the overlapping part on the first partial image and the second partial image, and delete the overlapping part on the first base map.
[0015] Transfer the second partial image onto the first base map, and make the first alignment point aligned with the center point of the second base map to obtain the stitched image.
[0016] Optionally, for the image stitching method according to an embodiment of the present application, calculating the overlapping part of the first partial image and the second partial image includes:
[0017] Use the obtained set of matched feature point pairs to calculate the overlapping part of the first partial image and the second partial image.
[0018] Optionally, for the image stitching method according to an embodiment of the present application, the size of the first base map is three times the size of the first partial image, and the size of the second base map is three times the size of the second partial image.
[0019] Optionally, for the image stitching method according to an embodiment of the present application, after performing the matching of the feature descriptors, the method further includes:
[0020] Perform feature point pair screening on the feature point pairs in the obtained set of feature point pairs after matching to obtain a filtered set of target feature point pairs.
[0021] Optionally, for the image stitching method according to an embodiment of the present application, performing feature point pair screening on the feature point pairs in the obtained set of feature point pairs after matching to obtain a filtered set of target feature point pairs includes:
[0022] Place the first partial image and the second partial image in the same coordinate system.
[0023] Calculate the coordinate values of all the feature points in the set of feature point pairs in the coordinate system.
[0024] Calculate the Euclidean distance between two feature points in all feature point pairs using the coordinate values of the feature points, and the angle between the line connecting the feature point pair and the x-axis in this coordinate system;
[0025] Statistically count the feature point pairs whose Euclidean distance and angle of all feature point pairs are within a preset range respectively, and form the target feature point pair set.
[0026] Optionally, for the image stitching method of the embodiment of the present application, the first fiducial point and the second alignment point are feature points in the filtered target feature point set;
[0027] Then calculating the overlapping part of the first local image and the second local image includes:
[0028] Calculate the overlapping part of the first local image and the second local image using the feature points in the target feature point pair set.
[0029] In a second aspect, an image stitching device provided by an embodiment of the present application includes:
[0030] An acquisition module, configured to acquire feature points of a first local image and a second local image to be stitched;
[0031] A matching module, configured to match the acquired feature points to obtain a set of matching feature point pairs in the first local image and the second local image;
[0032] A creation module, configured to create base maps of the first local image and the second local image, and respectively determine alignment points of the first local image and the second local image from the set of feature point pairs;
[0033] A stitching module, configured to stitch the first local image and the second local image based on the base maps, the alignment points, and the set of feature point pairs to obtain a stitched image.
[0034] In a third aspect, an embodiment of the present application provides a processing device, including a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the program, it implements the image stitching method as described in the first aspect above.
[0035] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, on which a computer program is stored, and the computer program is used to implement the image stitching method as described in the first aspect above.
[0036] The image stitching method, device, processing equipment and storage medium provided by the embodiments of the present application extract the feature points of two partial images to be stitched, and then match the extracted feature points to obtain a set of corresponding feature point pairs in the two partial images. Then, the base maps corresponding to the two partial images are created respectively, and a pair of feature points is randomly selected from the set of matched feature point pairs as the alignment points of the images to be stitched. Finally, the created base maps, the selected alignment points and the set of matched feature point pairs are used to realize the lossless stitching of the two partial images, avoiding data tampering and loss caused by the processing of the partial images to be stitched, improving the stitching efficiency, and ensuring the integrity and accuracy of the stitched images. BRIEF DESCRIPTION OF THE DRAWINGS
[0037] Other features, objects and advantages of the present application will become more apparent from the following detailed description of non-limiting embodiments read with reference to the accompanying drawings:
[0038] FIG. 1(a) to FIG. 1(b) are schematic diagrams of partial images to be stitched according to the embodiments of the present application;
[0039] Figure 2 is a schematic flowchart of the image stitching method according to the embodiments of the present application;
[0040] FIG. 3(a) to FIG. 3(b) are schematic diagrams of feature point extraction according to the embodiments of the present application;
[0041] Figure 4 is a schematic diagram of feature point matching according to the embodiments of the present application;
[0042] Figure 5 is a schematic flowchart of the method for feature point screening according to the embodiments of the present application;
[0043] Figure 6 is a schematic flowchart of the image stitching method according to some embodiments of the present application;
[0044] Figure 7 is a schematic diagram of the base map and alignment points of the partial image according to the embodiments of the present application;
[0045] Figure 8 is a schematic diagram of the base map and alignment points of the partial image according to the embodiments of the present application;
[0046] Figure 9 is a schematic diagram of the partial image after deleting the overlapping part according to the embodiments of the present application;
[0047] Figure 10 is a schematic diagram of the two partial images after overlapping according to the embodiments of the present application;
[0048] Figure 11 is a schematic diagram of the stitched partial images according to some embodiments of the present application;
[0049] Figure 12 Schematic diagram of a spliced image according to some embodiments of the present application;
[0050] Figure 13 Schematic diagram after local image splicing according to some embodiments of the present application;
[0051] Figure 14 Schematic diagram of the structure of an image splicing device according to an embodiment of the present application;
[0052] Figure 15 Schematic diagram of the structure of a computer of a processing device according to an embodiment of the present application. Detailed implementation manners
[0053] The present application will be further described in detail below with reference to the accompanying drawings and embodiments. It can be understood that the specific embodiments described herein are only used to explain the relevant disclosure, rather than limiting the disclosure. Additionally, it should be noted that for the convenience of description, only parts related to the disclosure are shown in the drawings.
[0054] It should be noted that, without conflict, the embodiments in the present application and the features in the embodiments can be combined with each other. The present application will be described in detail below with reference to the drawings and embodiments.
[0055] It can be understood that in the related art of image splicing, such as the method of finding the splicing line by connecting feature point pairs, although the splicing effect can be improved, at least 1 / 3 or more of the overlapping parts of the two local images to be spliced are required, that is, the splicing method in the related art has relatively high requirements for the shooting of local images.
[0056] Moreover, in some related arts, in order to make the spliced image have no obvious splicing traces, the local images to be spliced are usually scaled, distorted or blurred to a certain extent, which is likely to cause tampering and loss of local data.
[0057] It can be understood that for image splicing in some scenarios, due to the limitations of image acquisition conditions, the local images to be spliced collected have characteristics such as single color, small feature differences, and low contrast, making it impossible to use the splicing methods in the above-mentioned related arts for image splicing.
[0058] Furthermore, it can also be understood that in some special scenarios, it is necessary to keep the features in the local images consistent before and after splicing the collected local images to ensure the integrity of the data.
[0059] For example, for the pathological images collected in the medical field, as shown in FIGS. 1(a) and 1(b), two retinal diabetic retinopathy pathological images collected by fluorescence, most of the areas are black, with a single color, only the blood vessels having relatively obvious features and a low foreground-background contrast. Therefore, the images in this scenario cannot meet the requirements for the spliced local images in the related technologies.
[0060] In addition, for the pathological images in medical diagnosis, they need to be accurate, complete and clear in order to provide reliable support for the objectivity of subsequent diagnosis by pathological experts. For the splicing of retinal diabetic retinopathy pathological images, any processing such as scaling, distortion or blurring is unacceptable.
[0061] The image splicing method provided in the embodiments of the present application, during the image splicing process, in order to splice the locally collected images with a single color in the above special scenarios, and ensure the data integrity and accuracy of the local images, and avoid data loss caused by performing scaling, distortion or blurring processing on the local images, which ultimately affects the usage effect of the spliced image. By constructing a black backplane, that is, a base map, and selecting alignment points, and then using the created base map and alignment points to achieve efficient and fast splicing of two local images, and all data in the local images is retained.
[0062] To better understand the image splicing method provided in the embodiments of the present application, the following will be Figures 2 to 10 elaborated in detail.
[0063] Figure 2 FIG. shows the schematic flowchart of the image splicing method of the embodiments of the present application. As Figure 2 shown, the method specifically includes:
[0064] S110, extract feature points from the images to be spliced, where the images to be spliced include a first local image and a second local image.
[0065] Specifically, the image splicing algorithm of the embodiments of the present application obtains the locally collected images to be spliced, such as two local images of retinal diabetic retinopathy collected by fluorescence technology as shown in FIGS. 1(a) and 1(b), that is, the first local image and the second local image.
[0066] Furthermore, a feature description algorithm can be used to extract feature points from the obtained images to be spliced, that is, extract the feature descriptors of each feature point.
[0067] It can be understood that feature extraction is a method of analyzing useful information or redundant information in a signal or an image. Through transformation, the representative features (feature descriptors) in the signal or the image can be highlighted, and the information of interest can be extracted as needed.
[0068] Among them, in some embodiments of the present application, SIFT (Scale Invariant Feature Transform) can be used to extract the feature points of the local images to be stitched. This algorithm is a feature description method for feature extraction, which has scale invariance, and the feature descriptors obtained in this way have the property of translational invariance.
[0069] In practice, when using the SIFT feature description algorithm for feature point extraction, first, representative feature points in the image can be detected, such as corner points, contour edge points, points with rapid changes in light and dark, etc. Then, for the gradient direction histogram in the area around each feature point, a vector describing this feature point is generated, such as a 128-dimensional vector, as the feature descriptor for each specific point. Finally, the feature descriptors of all feature points of the two local images to be stitched can be obtained.
[0070] For example, for the two local images of the retina shown in Fig. 1(a) and Fig. 1(b), through the above method, the feature points shown in Fig. 3(a) and Fig. 3(b) can be collected.
[0071] It can be understood that the above feature point extraction algorithm is only an exemplary illustration. In some other embodiments of the present application, other feature description algorithms can also be used, such as SURF, FAST, BRIEF, ORB, HOG and other feature description algorithms, which can all achieve the purpose of extracting all feature descriptors in this link, and the embodiments of the present application do not limit this.
[0072] Further, after performing the above steps and extracting the feature points of the first local image and the second local image, the feature points of the two extracted local images can be matched to identify the same features. That is, S120 is executed.
[0073] S120, match the extracted feature points to obtain a set of mutually matching feature point pairs in the first local image and the second local image.
[0074] Specifically, for the feature descriptor matching in this step, the feature matching algorithm of the Fast Library for Approximate Nearest Neighbors (FLANN) can be used to quickly search for candidate feature descriptors with a high probability of correct matching by constructing a KD-tree.
[0075] Further, the RANSAC (RANdom SAmple Consensus) algorithm can be used to filter out the wrongly matched descriptor pairs in the candidate feature descriptor pairs, and finally only the correctly matched feature point pairs, that is, the set of feature point pairs, are left.
[0076] It can be understood that the two feature points in each pair of feature points in the set of feature point pairs are the feature points that match each other in the first local image and the second image.
[0077] For example, for the acquisition results of two local images to be stitched as shown in FIGS. 3(a) and 3(b), the matching result can be as Figure 4 shown. The feature points in the first local image have corresponding matching feature points in the second local image.
[0078] It can be understood that the above-mentioned feature matching algorithm is only for illustrative purposes. In some other embodiments, feature matching algorithms such as BF and SSD can also be used, and all can achieve the purpose of obtaining correctly matched feature descriptor pairs, and they are all common and frequently used feature matching tools. The embodiments of the present application do not limit the specific algorithm.
[0079] It can also be understood that for local images to be stitched in general scenarios, which are characterized by bright colors, obvious features, high contrast, etc., the above-mentioned general feature point extraction and feature point matching methods can completely screen out incorrectly matched feature point pairs.
[0080] However, in special scenarios such as medical diagnosis, for local images acquired by fluorescence or other means, such as local images of diabetic retinopathy, which are characterized by a single color, only blood vessels having relatively obvious features and low foreground-background contrast, after feature point matching using the above method, there are still incorrectly matched feature point pairs that are not screened out, which easily leads to the failure of the subsequent stitching process and thus the inability to obtain a complete image.
[0081] Therefore, in some other embodiments of the present application, after the computer device finishes the above-mentioned feature descriptor matching, a targeted feature point pair screening step can be further performed, that is, the following S130 is executed.
[0082] S130: Screen the feature point pairs in the set of feature point pairs obtained after matching to obtain a set of target feature point pairs after screening.
[0083] Specifically, after obtaining two sets of feature point pairs to be stitched through feature matching, in order to ensure the accuracy of stitching, each pair of feature points in the set of feature point pairs is further screened to eliminate incorrectly matched feature point pairs.
[0084] For example, as Figure 4 shown are two local images of diabetic retinopathy to be stitched with feature point matching. Among them, the endpoints at both ends of the light gray lines and the dark gray lines are feature point pairs after feature matching.
[0085] Obviously, the two feature points connected by all the dark gray lines are not correctly matched feature points.
[0086] In order to further filter out the feature descriptor pairs with matching errors corresponding to the dark gray lines, in some embodiments of the present application, as Figure 5 shown, the feature screening may specifically include the following steps:
[0087] S131, placing the first local image and the second local image in the same coordinate system.
[0088] S132, calculating the coordinate values of all feature points in the feature point pair set in this coordinate system.
[0089] S133, using the coordinate values of the feature points to calculate the Euclidean distance between the two feature points in all feature point pairs, and the angle between the line connecting the feature point pair and the x-axis in this coordinate system.
[0090] S134, respectively counting the feature point pairs whose Euclidean distance and angle are within a preset range among all feature point pairs, to form the target feature point pair set.
[0091] Specifically, first, the two local images to be stitched can be placed in the same coordinate system, that is, placed in a new coordinate system, and then the coordinate values of all feature points in the feature point pair set in this new coordinate system are calculated.
[0092] Further, in the new coordinate system, calculate the Euclidean distance between the two feature points of all matching feature point pairs, and the angle between the line connecting the two feature points of the feature point pair and the positive direction of the x-axis.
[0093] After obtaining the Euclidean distance between the two feature points of all feature point pairs and the angle between the line and the x-axis through the above calculations, the feature point pairs whose Euclidean distance and angle are within a preset range among all feature point pairs can be respectively counted. Then, all feature point pairs within the preset range form the target feature point pair set.
[0094] Specifically, the histogram of the Euclidean distance of all feature point pairs and the histogram of the included angle degrees can be counted, and then the mode of the Euclidean distance between the feature point pairs and the mode of the included angle degrees are calculated through the histograms. Furthermore, the obtained modes can be used to determine the interval for screening all feature point pairs, that is, the preset range, such as the mode ± 10. Finally, the obtained preset range can be used to retain the feature point pairs whose Euclidean distance and included angle degree values are both within the preset range, and delete the other feature point pairs outside the preset range, to obtain the target feature point pair set, that is, the screening of the feature points is completed.
[0095] It can be understood that if the Euclidean distance and the included angle degree between the feature point pairs are within the preset range, it means that this pair of feature points is a correctly matched feature point pair and needs to be retained; otherwise, it means that it is a mismatched feature point and needs to be excluded.
[0096] It can also be understood that the above preset range is a numerical range determined based on experience, and the embodiments of the present application do not limit this.
[0097] It can be understood that after obtaining the correctly matched feature point pairs of the two partial images to be stitched through the above steps, the two partial images can be stitched by using the matched feature point pairs, and specifically, S140 and S150 can be executed.
[0098] It can also be understood that in the image stitching algorithms in the related art, in order to make the stitched image have no obvious stitching marks, the partial images are usually scaled, distorted or blurred to a certain extent, so that some features of the partial images deviate or are lost.
[0099] In some scenarios, such as in the case diagnosis scenario, it is necessary to maintain the integrity and accuracy of the partial images to provide reliable support for the objectivity of subsequent pathological expert diagnosis. That is, in some scenarios, for the images to be stitched, such as retinal diabetic retinopathy pathological images, any scaling, distortion or blurring is unacceptable.
[0100] In some embodiments of the present application, in order to avoid processing that affects local data such as scaling, distortion or blurring of the images to be stitched, the image stitching is performed through the following steps to ensure the integrity and accuracy of the images.
[0101] S140, create the base maps of the first partial image and the second partial image, and respectively determine the alignment points of the first partial image and the second partial image from the set of feature point pairs or the set of target feature point pairs.
[0102] S150, based on the base maps and alignment points, and the set of feature point pairs or the set of target feature point pairs, stitch the first partial image and the second partial image to obtain the stitched image.
[0103] Specifically, after obtaining the set of correctly matched feature point pairs or the set of target feature point pairs, first create base maps with sizes meeting preset conditions for each partial image, that is, create a first base map for the first partial image and a second base map for the second partial image. Then, arbitrarily select two correctly matched feature points from the two partial images in the set of feature point pairs or the set of target feature point pairs as the standard points, that is, the first alignment point of the first partial image and the second alignment point of the second partial image. Furthermore, the partial images to be stitched can be transferred, aligned, the overlapping parts can be deleted, and finally stitched through the created base maps of the first partial image and the second partial image and the selected alignment points.
[0104] Optionally, the preset condition that the size of the base map needs to meet may be that its size is at least three times the size of the corresponding partial image to be stitched. Specifically, it can be determined according to the actual situation, and the embodiments of the present application do not limit this.
[0105] It can be understood that in this step, for the set of feature point pairs used, if the above-mentioned feature point screening step is performed, the arbitrarily selected alignment point is the feature point in the target feature point pair set, and moreover, in the subsequent stitching process, the feature point pairs used are also the feature points in this target feature point pair set.
[0106] In some other embodiments, when it is determined that there is no need to perform the specific screening step anymore, the arbitrarily selected alignment point is the feature point in the feature point pair set, and moreover, in the subsequent stitching process, the feature point pairs used are also the feature points in this feature point pair set.
[0107] Optionally, as Figure 6 shown, in some embodiments, using the base map, alignment points, and the set of feature point pairs to stitch the partial images may specifically include the following steps:
[0108] S141, Transfer the first partial image onto the first base map, and transfer the second partial image onto the second base map, such that the first alignment point is aligned with the center point of the first base map, and the second alignment point is aligned with the center point of the second base map.
[0109] S142, Calculate the overlapping part of the first partial image and the second partial image.
[0110] S143, Delete the overlapping part of the first partial image.
[0111] S144, Transfer the second partial image onto the first base map, and make the first alignment point aligned with the center point of the second base map, to obtain the stitched image.
[0112] Specifically, further transfer the two partial images onto the base map with the alignment point as the center, such that the first alignment point is aligned with the center point of the first base map, and the second alignment point is aligned with the center point of the second base map. Then calculate the overlapping part of the first partial image and the second partial image transferred onto the base map, and delete the overlapping part on one of the base maps, for example, delete the overlapping part of the first partial image and the second partial image on the first base map, and keep the non-overlapping part.
[0113] For example, in some embodiments, the correctly matched target feature point pair set screened out in the above steps can be used to calculate the overlapping part of the two partial images, and the positions where the matched feature points are located represent the overlapping part of the two partial images.
[0114] For example, in some other embodiments, all pixels of the local image area in the first local image and the second local image base image can be set to white, then the set of all pixels whose pixel values in the first local image and the second local image are white is the overlapping part of the two images.
[0115] Alternatively, you can also compare and calculate two local Figure Four The coordinate values of the strip boundary in the base map can also be used to obtain the overlapping parts of the two local maps.
[0116] It can be understood that the specific calculation method for the overlapping part of two local images can be flexibly selected according to actual conditions, and the embodiments of the present application do not limit this.
[0117] Finally, transfer another complete partial image to the base map that has been deleted, and use the alignment points to align them, ensuring that the center points of the two base maps can be aligned, so that the two partial images can be smoothly spliced according to the preset requirements, that is, the remaining part of the partial image where the overlapping part is deleted is accurately aligned to the other partial image. For example, transfer the second base map where the second partial image is located to the first base map, and use the selected alignment points to align the center points of the first base map with the second base map to complete the perfect splicing of the two partial images and ensure no data loss.
[0118] It can be understood that the conditions that the size of the two solid color background plates needs to meet can be understood to be determined according to the size of the local images to be spliced, and their length and width are at least larger than the corresponding local images, such as their length and width are 3 times the size of the corresponding local images.
[0119] For example, for the two partial images to be stitched as shown in FIG. 1 (a) and FIG. 1 (b), after the above-mentioned steps as shown in FIG. 3 (a), FIG. 3 (b) and Figure 4 After the feature point extraction, matching and screening steps shown, stitching can be performed.
[0120] First, as shown in FIG. 1 (a) and FIG. 1 (b), the two local images of retinal diabetic retinopathy to be stitched, such as the first local image and the second local image, can be marked as ImgA and ImgB, and then one pair of correctly matched target feature point pairs selected in the feature screening step can be selected as the alignment points of the first local image and the second local image, and marked as A and B respectively.
[0121] Further, if Figure 7 and Figure 8 As shown, two pure color background images with sizes meeting the requirements are created, such as pure black rectangular images, namely the first base image and the second base image, which are used as the base images of the two partial images to be spliced, and can be marked as ImgBaseA and ImgBaseB respectively.
[0122] It can be understood that the length and width of the created ImgBaseA are three times the length and width of ImgA respectively, and the length and width of the created ImgBaseB are three times the length and width of ImgB respectively.
[0123] Further, after creating the black base maps for the two partial images, align the coordinates of the first alignment point A randomly selected in the above steps with the center of the first base map ImgBaseA, and copy all other pixel points of the first partial image ImgA onto the first base map ImgBaseA. Similarly, align the coordinates of the second alignment point B randomly selected in the above steps with the center of the second base map ImgBaseA, and copy all other pixel points of the partial image ImgB onto the base map ImgBaseB, as Figure 7 and Figure 8 shown.
[0124] Further, after transferring the two partial images to the corresponding base maps respectively and ensuring that the specific feature points selected in the above steps are located at the center positions of the corresponding base maps, the overlapping parts of the two partial images on the two base maps can be calculated.
[0125] Further, after calculating the overlapping parts, the duplicate parts in one of the base maps can be deleted. For example, delete the overlapping parts of the two partial images on the base map ImgBaseB to obtain ImgBaseB - temp, as Figure 9 shown.
[0126] Finally, as Figure 10 shown, the complete partial image on the other base map can be transferred to the base map after deleting the overlapping parts, and during the transfer process, ensure that the first alignment point, that is, point A is aligned with the center point of the second base map, then the splicing of the two partial images can be completed, as Figure 11 shown.
[0127] For example, transfer the partial image ImgA to ImgBaseB - temp, and ensure that the coordinates of the corresponding feature point A of the partial image ImgA are aligned with the center point of ImgBaseB - temp.
[0128] In the embodiment of the present application, by creating corresponding black base maps for each image to be spliced, and then selecting a correctly matched pair of feature points as the alignment points, so that the two partial images are transferred to the base maps with the alignment points as the center points of the base maps, and then by calculating and deleting the overlapping parts in one of the images, finally moving the other complete partial image to the remaining part after deleting the overlapping parts, the lossless splicing of the two partial images is completed, so as to avoid data damage caused by performing any processing on the partial images and ensure the integrity of the partial images to be spliced.
[0129] To better understand the data protection of the images to be stitched and the high precision of stitching in the embodiments of the present application, Figure 12 multiple locally collected images of diabetic retinopathy are provided.
[0130] It can be understood that the multiple locally collected images are stitched in sequence by using the stitching method provided in the above embodiments, and finally a complete retinal image after stitching is obtained, as Figure 13 shown.
[0131] Specifically, after obtaining the 16 locally collected images as Figure 12 shown, first, the above method can be used to complete the stitching of A and B, extract the feature points in images A and B, then perform matching and screening, and finally use the base map and the selected alignment points to complete the stitching of images A and B to obtain the stitched image AB. Furthermore, the stitched image AB and image C are used as two locally collected images, and the above steps of feature point extraction, matching, screening, and stitching are repeated. Then, the obtained stitched image is used as a locally collected image and continues to be stitched with the subsequent locally collected images until the last image P is stitched on to obtain the complete image as Figure 13 shown, that is, a complete diabetic retinopathy image.
[0132] On the other hand, the embodiments of the present application also provide an image stitching device, as Figure 14 shown. The device 200 includes:
[0133] An acquisition module 210, configured to acquire the feature points of the first locally collected image and the second locally collected image to be stitched;
[0134] A matching module 220, configured to match the acquired feature points to obtain a set of matched feature point pairs in the first locally collected image and the second locally collected image;
[0135] A creation module 230, configured to create the base maps of the first locally collected image and the second locally collected image, and respectively determine the alignment points of the first locally collected image and the second locally collected image from the set of feature point pairs;
[0136] A stitching module 240, configured to stitch the first locally collected image and the second locally collected image based on the base maps, the alignment points, and the set of feature point pairs to obtain a stitched image.
[0137] Optionally, for the image stitching device of the embodiments of the present application, the base map of the first locally collected image is the first base map, the base map of the second locally collected image is the second base map, the alignment point of the first locally collected image is the first alignment point, the alignment point of the second locally collected image is the second alignment point, and the stitching module 240 includes:
[0138] A transfer unit 241, configured to transfer the first partial image onto the first base map and transfer the second partial image onto the second base map respectively, such that the first alignment point is aligned with the center point of the first base map and the second alignment point is aligned with the center point of the second base map.
[0139] A first calculation unit 242, configured to calculate an overlapping part on the first partial image and the second partial image, and delete the overlapping part on the first base map.
[0140] A splicing unit 243, configured to transfer the second partial image onto the first base map and make the first alignment point aligned with the center point of the second base map, so as to obtain a spliced image.
[0141] Optionally, for the image splicing device according to an embodiment of the present application, the first calculation unit 242 is specifically configured to:
[0142] Calculate the overlapping part of the first partial image and the second partial image by using the set of matched feature point pairs.
[0143] Optionally, for the image splicing device according to an embodiment of the present application, the size of the first base map is three times the size of the first partial image, and the size of the second base map is three times the size of the second partial image.
[0144] Optionally, for the image splicing device according to an embodiment of the present application, the device further includes:
[0145] A screening module 250, configured to screen the feature point pairs in the set of feature point pairs obtained after matching to obtain a set of target feature point pairs after screening.
[0146] Optionally, for the image splicing device according to an embodiment of the present application, the screening module 250 includes:
[0147] A coordinate conversion unit 251, configured to place the first partial image and the second partial image in the same coordinate system.
[0148] A second calculation unit 252, configured to calculate the coordinate values of all the feature points in the coordinate system in the set of feature point pairs.
[0149] A third calculation unit 253, configured to calculate the Euclidean distance between two feature points in all the feature point pairs by using the coordinate values of the feature points, and the included angle between the connection line of the feature point pair and the x-axis in the coordinate system.
[0150] A statistics unit 254, configured to respectively count the feature point pairs whose Euclidean distance and included angle are within a preset range among all the feature point pairs to form the set of target feature point pairs.
[0151] Optionally, in the image stitching device according to an embodiment of the present application, the first alignment point and the second alignment point are feature points in the filtered target feature point set;
[0152] Then the first calculation unit 242 is specifically configured to:
[0153] Use the feature points in the target feature point pair set to calculate the overlapping part of the first partial image and the second partial image.
[0154] On the other hand, an embodiment of the present application further provides a processing device, which includes a memory, a processor, and a computer program stored on the memory and executable on the processor. The processor is used to implement the image stitching method described in the above embodiment when executing the program.
[0155] Next, refer to Figure 15 , Figure 15 which is a schematic structural diagram of a computer electronic device of the processing device according to an embodiment of the present application.
[0156] As Figure 15 shown, the computer electronic device includes a central processing unit (CPU) 301, which can perform various appropriate actions and processes according to the program stored in the read-only memory (ROM) 302 or the program loaded from the storage part 302 into the random access memory (RAM) 303. In the RAM 303, various programs and data required for the operation of the electronic device 300 are also stored. The CPU 301, ROM 302, and RAM 303 are connected to each other through a bus 304. The input / output (I / O) interface 305 is also connected to the bus 304.
[0157] The following components are connected to the I / O interface 305: an input part 306 including a keyboard, a mouse, etc.; an output part 307 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc. and a speaker, etc.; a storage part 308 including a hard disk, etc.; and a communication part 309 including a network interface card such as a LAN card, a modem, etc. The communication part 309 performs communication processing via a network such as the Internet. The drive 310 is also connected to the I / O interface 305 as needed. A removable medium 311, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 310 as needed, so that the computer program read from it can be installed into the storage part 308 as needed.
[0158] In particular, according to an embodiment of the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present application includes a computer program product that includes a computer program carried on a machine-readable medium, and the computer program includes program code for performing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through the communication section 309, and / or installed from the removable medium 311. When the computer program is executed by the central processing unit (CPU) 301, the above functions defined in the electronic device of the present application are executed.
[0159] It should be noted that the computer-readable medium shown in the present application can be a computer-readable signal medium, a computer-readable storage medium, or any combination of the two. A computer-readable storage medium can be, for example, but not limited to, an electronic device, apparatus, or device of electricity, magnetism, light, electromagnetic, infrared, or semiconductor, or any combination of the above. More specific examples of the computer-readable storage medium can include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present application, the computer-readable storage medium can be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction-executing electronic device, apparatus, or device. And in the present application, a computer-readable signal medium can include a data signal propagated in a baseband or as part of a carrier wave, which carries the computer-readable program code. Such a propagated data signal can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. A computer-readable signal medium can also be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in conjunction with an instruction-executing electronic device, apparatus, or device. The program code contained on the computer-readable medium can be transmitted by any appropriate medium, including but not limited to: wireless, wire, optical cable, RF, etc., or any suitable combination of the above.
[0160] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architectures, functions, and operations of processing devices, methods, and computer program products according to various embodiments of the present application. In this regard, each block in the flowchart or block diagram may represent a module, a segment of a program, or a part of code, and the aforementioned module, segment of a program, or part of code includes one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than that marked in the accompanying drawings. For example, two consecutive blocks shown may actually be executed substantially in parallel, and they may sometimes be executed in the reverse order, depending on the functions involved. It should also be noted that each block in the block diagram and / or flowchart, as well as combinations of blocks in the block diagram and / or flowchart, can be implemented by a dedicated hardware-based electronic device that performs the specified functions or operations, or can be implemented by a combination of dedicated hardware and computer instructions.
[0161] The units or modules involved in the embodiments described in the present application can be implemented in software or in hardware. The described units or modules can also be provided in a processor. For example, it can be described as: a processor, including: an acquisition module, a matching module, a creation module, and a splicing module. Among them, the names of these units or modules do not constitute a limitation on the units or modules themselves in some cases. For example, the creation module can also be described as "used to create the base maps of the first partial image and the second partial image, and respectively determine the alignment points of the first partial image and the second partial image from the set of feature point pairs".
[0162] As another aspect, the present application also provides a computer-readable storage medium. This computer-readable storage medium can be included in the electronic device described in the above embodiments; it can also exist separately and not be assembled into the electronic device. The above computer-readable storage medium stores one or more programs, and when the aforementioned programs are executed by one or more processors to perform the image splicing method described in the present application:
[0163] Acquire the feature points of the first partial image and the second partial image to be spliced;
[0164] Match the acquired feature points to obtain a set of matching feature point pairs in the first partial image and the second partial image;
[0165] Create the base maps of the first partial image and the second partial image, and respectively determine the alignment points of the first partial image and the second partial image from the set of feature point pairs;
[0166] Based on the base map, the alignment points, and the set of feature point pairs, the first partial image and the second partial image are stitched to obtain a stitched image.
[0167] In summary, for the image stitching method, processing device, and storage medium provided in the embodiments of the present application, after the software design is completed by the developer, the software data of the software to be delivered is stored in advance in the server database. As a result, when the delivery personnel perform the software data delivery of the software to be delivered at the user's site, they can be verified through the delivery password to remotely download and obtain the file data of the software to be delivered from the server database, avoiding data loss or leakage caused by the delivery personnel carrying the file data externally, and ensuring the security of the software to be delivered.
[0168] The above description is only a preferred embodiment of the present application and an explanation of the applied technical principles. Those skilled in the art should understand that the scope of disclosure involved in the present application is not limited to the technical solution formed by the specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the foregoing disclosure concept. For example, the technical solutions formed by mutually replacing the above features with the (but not limited to) technical features with similar functions disclosed in the present application.
Claims
1. An image stitching method, characterized in that, The method includes: Obtaining feature descriptors of feature points of a first partial image and a second partial image to be stitched; the obtaining of the feature descriptors of the feature points of the first partial image and the second partial image to be stitched includes: detecting representative feature points in the first partial image and the second partial image, and generating a vector describing each representative feature point for the histogram of gradient directions in the area around each representative feature point, so as to obtain the feature descriptor of the representative feature point; wherein, the representative feature points include at least one of the following: corner points, contour edge points, and points with rapid brightness change; Matching the obtained feature points to obtain a set of matching feature point pairs in the first partial image and the second partial image; after performing the matching of the feature points, placing the first partial image and the second partial image in the same coordinate system; calculating the coordinate values of all feature points in the set of feature point pairs in the coordinate system; calculating the Euclidean distance between the two feature points in all feature point pairs and the included angle between the line connecting the feature point pair and the x-axis in the coordinate system by using the coordinate values of the feature points; respectively counting the feature point pairs with the Euclidean distance and the included angle of all feature point pairs within a preset range to form a set of target feature point pairs; Creating base maps of the first partial image and the second partial image, and respectively determining alignment points of the first partial image and the second partial image from the set of target feature point pairs; Stitching the first partial image and the second partial image based on the base maps, the alignment points, and the set of target feature point pairs to obtain a stitched image, wherein the base map of the first partial image is a first base map, the base map of the second partial image is a second base map, the alignment point of the first partial image is a first alignment point, and the alignment point of the second partial image is a second alignment point; The stitching the first partial image and the second partial image based on the base maps, the alignment points, and the set of target feature point pairs to obtain a stitched image includes: Respectively transferring the first partial image onto the first base map and the second partial image onto the second base map, such that the first alignment point is aligned with the center point of the first base map and the second alignment point is aligned with the center point of the second base map; Calculating the overlapping part on the first partial image and the second partial image, and deleting the overlapping part on the first base map; Transferring the second partial image onto the first base map and making the first alignment point aligned with the center point of the second base map to obtain a stitched image.
2. The image stitching method according to claim 1, wherein The size of the first base map is three times the size of the first partial image, and the size of the second base map is three times the size of the second partial image.
3. The image stitching method according to claim 1, wherein The first alignment point and the second alignment point are feature points in the filtered set of target feature point pairs; Then the calculating the overlapping part on the first partial image and the second partial image includes: Calculating the overlapping part on the first partial image and the second partial image by using the feature points in the set of target feature point pairs.
4. An image stitching device, characterized in that, The device includes: An acquisition module, configured to acquire feature descriptors of feature points of a first partial image and a second partial image to be stitched; the acquiring of the feature descriptors of the feature points of the first partial image and the second partial image to be stitched includes: detecting representative feature points in the first partial image and the second partial image, and generating a vector describing each of the representative feature points for the histogram of gradient directions in the region around each of the representative feature points, so as to obtain the feature descriptors of the representative feature points; wherein, the representative feature points include at least one of the following: corner points, contour edge points, and points with rapid brightness change; A matching module, configured to match the acquired feature points to obtain a set of matched feature point pairs in the first partial image and the second partial image; after performing the matching of the feature points, place the first partial image and the second partial image in the same coordinate system; calculate the coordinate values of all the feature points in the coordinate system in the set of feature point pairs; calculate the Euclidean distance between the two feature points in all the feature point pairs by using the coordinate values of the feature points, and the included angle between the line connecting the feature point pair and the x-axis in the coordinate system; respectively count the feature point pairs whose Euclidean distance and included angle in all the feature point pairs are within a preset range to form a set of target feature point pairs; A creation module, configured to create base maps of the first partial image and the second partial image, and respectively determine alignment points of the first partial image and the second partial image from the set of target feature point pairs; A stitching module, configured to stitch the first partial image and the second partial image based on the base maps, the alignment points, and the set of target feature point pairs to obtain a stitched image, wherein the base map of the first partial image is a first base map, the base map of the second partial image is a second base map, the alignment point of the first partial image is a first alignment point, and the alignment point of the second partial image is a second alignment point; The stitching module is specifically configured to: Transfer the first partial image to the first base map respectively, and transfer the second partial image to the second base map, so that the first alignment point is aligned with the center point of the first base map, and the second alignment point is aligned with the center point of the second base map; Calculate the overlapping part on the first partial image and the second partial image, and delete the overlapping part on the first base map; Transfer the second partial image to the first base map, and make the first alignment point aligned with the center point of the second base map to obtain the stitched image.
5. A processing device, characterized in that, The processing device includes a memory, a processor, and a computer program stored on the memory and executable on the processor, and the processor is configured to implement the image stitching method according to any one of claims 1-3 when executing the program.
6. A computer-readable storage medium, on which a computer program is stored, and the computer program is configured to implement the image stitching method according to any one of claims 1-3.