Image splicing method and system
By using the feature descriptor of the random circular spot pattern as matching feature points in the array image acquisition system, the splicing quality problem caused by uneven distribution of natural texture feature points is solved, and efficient and accurate image stitching is achieved.
Patent Information
- Application Number
- CN202410981218.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-22
- Publication Date
- 2024-11-15
AI Technical Summary
When stitching images in array image acquisition systems, poor stitching quality is caused by uneven distribution of feature points that can be identified by natural textures and inaccurate extraction.
The random circular spot pattern is used to obtain the splicing matrix. By generating a black circular spot pattern with random positions and sizes, the feature descriptor of the center of the circular spot is extracted as a matching feature point, and the homography matrix is calculated for image splicing.
It improves the accuracy and representativeness of feature extraction, reduces splicing errors, and achieves efficient and high-speed seamless image splicing.
Smart Images

Figure CN118967435A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the technical field of two-dimensional image processing, and in particular relates to an image stitching method and system. Background Art
[0002] An array image acquisition system for planar objects composed of multiple cameras needs to determine the stitching matrix between images taken by different cameras through image features during the image acquisition process. Usually, image stitching is based on natural texture, and matching between images is achieved by extracting identifiable feature points from natural texture. However, the distribution uniformity and extraction accuracy of identifiable feature points of natural texture are not as good as those of artificial features. Therefore, the identifiable feature points of natural texture are unevenly distributed and inaccurately extracted. In this case, stitching errors are likely to occur when stitching images. Summary of the invention
[0003] The purpose of the present invention is to provide an image stitching method, which solves the problem of poor image stitching quality caused by uneven distribution and inaccurate extraction of identifiable feature points of feature images when an array image acquisition system performs image stitching.
[0004] The present invention is achieved through the following technical solutions: An image stitching method is used for stitching images taken by an array camera in an imaging system composed of an array camera and a shooting platform. A stitching matrix for stitching images taken by the array camera is obtained by using a random circular spot pattern, and the images taken by the array camera are stitched according to the stitching matrix. The random circular spot pattern is a pattern composed of a plurality of randomly generated non-overlapping circular spots, and the position distribution of each circular spot in the same random circular spot pattern is different, and the position distribution of the circular spots between different random circular spot patterns is different.
[0005] In some embodiments, the step of using a random circular spot pattern to obtain a stitching matrix for stitching images captured by an array camera includes: Laying a plurality of pictures containing only random circular spot patterns on a shooting platform, so that at least one of the pictures is within the shooting range of each camera in the array camera and at least one of the same pictures is included in the images shot by adjacent cameras; The pictures are photographed with an array camera, stitching characteristic information of random circular spot patterns in each photographed image is extracted, and a stitching matrix for stitching the photographed images is obtained according to the extracted stitching characteristic information.
[0006] In some embodiments, the generation of the random circular spot pattern comprises the following steps: S11, generating a white pattern, and generating a black border on the white pattern; S12, randomly selecting a point in the black border, arbitrarily selecting a radius value within the set radius range, taking the selected point as the center of the circle and the selected radius value as the radius, generating a black circular spot in the black border, and making the black circular spot not overlap with the black border; S13, repeat step S12 until the number of circular spots in the black frame reaches a set value.
[0007] In some embodiments, in step S13, when generating circular spots within the black border, the step of making the generated circular spots non-overlapping comprises: S131, when generating a new circular spot, taking the currently selected point as the center of the circle and the multiple of the currently selected radius value as the radius, determining whether there are black pixels in the circular area; S132: If there are black pixels, reselect other points and radius values and repeat step S131; if there are no black pixels, generate a circular spot with the currently selected points and radius values.
[0008] In some embodiments, in step S12, the upper limit value is the length of the black border multiplied by 10 and then divided by the set number of circular spots, and the lower limit value is the width of the black border multiplied by 10 and then divided by the set number of circular spots. The upper limit value and the lower limit value are used as the set radius range.
[0009] In some embodiments, the step of extracting the stitching characteristic information of the random circular spot pattern in each captured image includes: Binarize the captured image and find the connected domain in the captured image; Sort the connected domains in descending order according to their area, select multiple connected domains with larger areas, and fill the unselected connected domains; Identify the circular spots in the image that are located in the selected connected domain, and obtain the center of the identified circular spots; Calculate the feature descriptor of the center of the identified circular spot.
[0010] In some embodiments, the step of obtaining a stitching matrix for stitching the captured images according to the extracted stitching feature information includes: The feature descriptors extracted from the images taken by different cameras are matched, the homography matrix is calculated based on the coordinates of the circle center obtained by matching, and the stitching matrix for stitching the captured images is obtained according to the homography matrix.
[0011] In some embodiments, during the operations of acquiring the stitching matrix and stitching the images captured by the array camera according to the stitching matrix, the relative position between the array camera and the capturing platform remains unchanged.
[0012] The present invention also relates to an image stitching system, comprising: An image processing unit, the image processing unit is used to process the image taken by the array camera and extract the splicing feature information of the random circular spot pattern in the image; a stitching matrix acquisition unit, wherein the stitching matrix acquisition unit acquires a stitching matrix for stitching the captured images according to the extracted stitching feature information; An image stitching unit is used to stitch images taken by the array camera according to a stitching matrix.
[0013] In some embodiments, the image stitching system further includes a random circular spot pattern generating unit, and the random circular spot pattern generating unit is used to generate a random circular spot pattern.
[0014] Compared with the prior art, the present invention has the following advantages and beneficial effects: 1. The present invention adopts the feature descriptor of the center of the circular spot in the random circular spot pattern as the matching feature point for the stitching of the captured images, so that the feature extraction is simpler, and the information of the feature extraction is more accurate and representative, which solves the problem that the extraction of the identifiable feature points is inaccurate due to the uneven distribution of the identifiable feature points of the natural texture, which affects the effectiveness of image matching and causes errors in image stitching. 2. The position and size of the black circular spots in the random circular spot pattern are randomly generated, so each generated circular spot pattern is a random circular spot pattern that does not repeat each other, and any solid black circular spot in the random circular spot pattern has a unique topological form. Therefore, it is easy to match the identifiable feature points of the captured image, thereby improving the accuracy of matching the identifiable feature points of the captured image.
[0015] 3. The present invention adopts a random circular spot pattern with a black border, and can separate the area inside the black border of the random circular spot pattern into a connected area when shooting images, which is not affected by the background image and is convenient for finding the connected domain in the captured image after the captured image is binarized.
[0016] 4. The present invention calculates the homography matrix based on the matched circle center coordinates, and obtains the stitching matrix for stitching the captured images according to the homography matrix, thereby achieving efficient and high-speed image stitching to form a seamless complete image.
[0017] 5. The present invention maintains the relative position between the array camera and the shooting platform unchanged, so that the shooting environment for obtaining the stitching matrix and stitching the images taken by the array camera according to the stitching matrix is consistent, avoiding image stitching errors caused by different shooting environments and achieving seamless stitching of captured images. BRIEF DESCRIPTION OF THE DRAWINGS
[0018] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings in the embodiments will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without creative work.
[0019] Figure 1 The figure is a flow chart of obtaining a splicing matrix in an embodiment of the present invention.
[0020] Figure 2 The flowchart of generating a random circular spot pattern in an embodiment of the present invention.
[0021] Figure 3 Schematic diagram of a random circular spot pattern generated in an embodiment of the present invention.
[0022] Figure 4 Schematic diagram of a white pattern with a black border generated in an embodiment of the present invention.
[0023] Figure 5 : is an image captured by the left camera in an embodiment of the present invention.
[0024] Figure 6 : is an image captured by the right camera in an embodiment of the present invention.
[0025] Figure 7 Schematic diagram of the connected domain of the image captured by the left camera in an embodiment of the present invention.
[0026] Figure 8 Schematic diagram of circular spot recognition in an image captured by the left camera in an embodiment of the present invention.
[0027] Fig. 9 Schematic diagram of a stitched image obtained by stitching images in an embodiment of the present invention. DETAILED DESCRIPTION
[0028] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments.
[0029] An array image acquisition system is usually composed of an array camera and a shooting platform. In the array image acquisition system, the object to be shot, such as a large mechanical drawing, is usually laid flat on the shooting platform, and multiple cameras are suspended in an array to shoot the object vertically downward, and then the captured images are stitched to obtain a complete image of the object to be shot. In order to enable the system to automatically realize the stitching operation of each captured image and ensure the quality and accuracy of the stitching, a drawing with features can be laid on the shooting platform, and the features in the feature drawings in the captured image can be extracted and matched to calculate the stitching matrix; then the object to be shot is laid on the shooting platform and shot again. Since the positions of the array camera and the shooting platform have not changed during the whole process, the stitching matrix calculated previously can be used to stitch the image.
[0030] In the above-mentioned method of obtaining the stitching matrix by extracting features from the feature drawing, the accuracy of the stitching matrix obtained is low due to the uneven distribution of the identifiable feature points of the feature drawing itself, which affects the effectiveness of the stitching matrix in subsequent image stitching, resulting in errors in image stitching and making it difficult to meet the stitching requirements.
[0031] Example 1 An image stitching method is used for stitching images taken by an array camera in an imaging system composed of an array camera and a shooting platform. A stitching matrix for stitching images taken by the array camera is obtained by using a random circular spot pattern, and the images taken by the array camera are stitched according to the stitching matrix. The random circular spot pattern is a pattern composed of a plurality of randomly generated non-overlapping circular spots, and the positions of the circular spots in the same random circular spot pattern are all different, and the positions of the circular spots in different random circular spot patterns are all different. The specific steps are as follows: S1, generate a white pattern, and generate a black border on the white pattern; The length of the white pattern is preset as A, the width of the white pattern is preset as B, the number of circular spots required in the white pattern is preset as C, and a black border is drawn around the white pattern.
[0032] The black border can be used to separate the area inside the white pattern of the black border into a connected area when shooting the image, which is not affected by the background image and is convenient for finding the connected domain in the captured image after the image is binarized.
[0033] S2. Randomly select a point in the black border, take the length of the black border multiplied by 10 and then divided by the set number of circular spots as the upper limit value, take the width of the black border multiplied by 10 and then divided by the set number of circular spots as the lower limit value, take the upper limit value and the lower limit value as the set radius range, arbitrarily select a radius value within the set radius range, take the selected point as the center of the circle, and the selected radius value as the radius, generate a black circular spot in the black border, and make the black circular spot not overlap with the black border; Among them, a point is randomly selected and denoted as P(x, y), the upper limit value is denoted as 10*A / C, the lower limit value is denoted as 10*B / C, and the selected radius value is denoted as R.
[0034] S3. When generating a new circular spot, take the currently selected point as the center of the circle and the multiple of the currently selected radius value as the radius to determine whether there are black pixels in the circular area; if there are black pixels, repeat step S2 and reselect other points and radius values; if there are no black pixels, generate a circular spot with the currently selected point and radius value.
[0035] S4, repeating steps S2 to S3 until the number of circular spots in the black frame reaches the set value C, generating a random circular spot pattern, and generating a flow chart of a random circular spot pattern, refer to Figure 2 ; S5, laying a plurality of pictures containing only random circular spot patterns on a shooting platform, and making sure that there is at least one picture within the shooting range of each camera in the array camera and that the images shot by adjacent cameras contain at least one identical picture; shooting the pictures with the array camera to obtain a shot image; Among them, the positions and sizes of the circular spots in multiple pictures containing only random circular spot patterns are randomly produced, so the circular spot patterns in each picture are random circular spot patterns that do not repeat each other, and any circular spot in the random circular spot pattern has a unique topological form. Therefore, it is convenient to match the identifiable feature points of the captured image, thereby improving the accuracy of matching the identifiable feature points of the captured image.
[0036] S6, performing binarization processing on the captured image, and finding a connected domain in the captured image; The captured image is binarized so that the captured image presents a distinct visual effect of only black and white, which facilitates the search for connected domains in the captured image.
[0037] S7, sorting the connected domains in descending order according to their areas, selecting multiple connected domains with larger areas, and filling the unselected connected domains; When multiple random circular spot patterns are laid on the shooting table, there may be gaps between the random circular spot patterns, which makes the connected domains in the shot image scattered, so multiple connected domains with larger areas are selected; at the same time, the black circular spots in the random circular spot patterns are affected by printing and shooting, so that white spots may exist on the black circular spots in the shot image to form smaller connected domains. At this time, it is necessary to fill the smaller connected domains on the black circular spots with black to form a complete black circular spot, which is convenient for the subsequent identification of the circular spots.
[0038] S8, identifying the circular spot in the image that is located in the selected connected domain, and obtaining the center of the identified circular spot; The connected domain is a white background. Identifying the circular spots in the selected connected domain can eliminate the interference of similar circular spot patterns in other areas and improve the accuracy of extracting identifiable feature points.
[0039] S9, calculating a feature descriptor of the center of the identified circular spot; S10, matching feature descriptors extracted from pictures taken by different cameras, calculating a homography matrix based on the coordinates of the circle center obtained by matching, obtaining a stitching matrix for stitching the captured images according to the homography matrix, and obtaining a flowchart of the stitching matrix, refer to Figure 1 ; Among them, the feature descriptor of the center of the circular spot in the random circular spot pattern is used as the matching feature point for stitching of the captured images, which makes feature extraction simpler and the information extracted by feature is more accurate and representative. It solves the problem that the extraction of identifiable feature points is inaccurate due to the uneven distribution of identifiable feature points of natural texture, which affects the effectiveness of image matching and causes errors in image stitching.
[0040] In the operations of acquiring the stitching matrix and stitching the images taken by the array camera according to the stitching matrix, the relative position between the array camera and the shooting platform remains unchanged, so that the shooting environment of the operations of acquiring the stitching matrix and stitching the images taken by the array camera according to the stitching matrix is consistent, avoiding image stitching errors caused by different shooting environments, and realizing seamless stitching of the captured images.
[0041] A stitching matrix for stitching the captured images is obtained according to the homography matrix, so as to achieve efficient and high-speed image stitching and form a seamless complete image.
[0042] S11. Stitching the images taken by the array camera according to the stitching matrix.
[0043] The present invention also relates to an image stitching system, comprising: A random circular spot pattern generating unit, the random circular spot pattern generating unit is used to generate a random circular spot pattern; An image processing unit, the image processing unit is used to process the image taken by the array camera and extract the splicing feature information of the random circular spot pattern in the image; A stitching matrix acquisition unit, the stitching matrix acquisition unit acquires a stitching matrix for stitching the captured images according to the extracted stitching feature information; The image stitching unit is used to stitch the images taken by the array camera according to the stitching matrix.
[0044] Example 2 A specific embodiment of image stitching is implemented by using an image stitching method of the present invention, and the specific steps are as follows: Take A4 paper printing random circular pattern as an example, refer to Figure 3 , a schematic diagram of the generated random circular pattern, setting the length of the random circular pattern to 2970 pixels, the width of the random circular pattern to 2100 pixels, and the number of black circular spots in the random circular pattern to 1000. Figure 4 , schematic diagram of the generated black border and white pattern, setting the width of the black border of the black border and white pattern to 20 pixels.
[0045] A point P (x, y) is randomly generated in the white blank area of the black border and white pattern. With point P as the center, the horizontal coordinate x of the center is between 20+2R and 2080-2R, and the vertical coordinate y is between 20+2R and 2950-2R. The radius value is set to R, and R is between 21 pixels and 29 pixels. First, check whether there are black pixels in the circular area with point P (x, y) as the center and 2R as the radius; if there are no black pixels, it proves that this circular area is blank, then draw a black circular spot with point P (x, y) as the center and R as the radius value, and repeat the above process for 1000 times to generate a random circular spot pattern. Refer to Figure 3 .
[0046] Repeat the previous steps of generating random circular spot patterns, and generate another random circular spot pattern. Print the two generated random circular spot patterns on A4 paper respectively, and lay the two printed A4 papers flat on the shooting platform, so that two adjacent cameras can simultaneously shoot two A4 papers printed with non-repeating random circular spot patterns, that is, two A4 papers printed with non-repeating random circular spot patterns are laid flat in the overlapping area of the field of view of two adjacent cameras, and then start the array camera to shoot, and obtain two captured images, which are the images shot by the left camera, and refer to Figure 5 , and the image captured by the right camera, refer to Figure 6 .
[0047] Binarize the image taken by the left camera, find and retain the top two connected domains in terms of area, and obtain a schematic diagram of the connected domain of the image taken by the left camera. Figure 7 , search and identify circular spots in the connected domain of the image taken by the left camera, and obtain a schematic diagram of circular spot recognition of the image taken by the left camera, referring to Figure 8 The same processing operation is performed on the image captured by the right camera, which will not be repeated here.
[0048] Calculate the feature descriptors of the center of the circular spot in the images taken by the left and right cameras, match the feature descriptors of the center of the circular spot in the left and right images, calculate the homography matrix based on the coordinates of the center of the circle obtained by matching, and obtain the stitching matrix for stitching the left and right images according to the homography matrix. Multiply the image taken by the right camera by the stitching matrix obtained, and fuse it with the image taken by the left camera to obtain a schematic diagram of the stitched image obtained by stitching the images. Refer to Fig. 9 .
[0049] The above description is only a preferred embodiment of the present invention and does not limit the present invention in any form. Any simple modification or equivalent change made to the above embodiment based on the technical essence of the present invention shall fall within the protection scope of the present invention.
Claims
1. An image stitching method, used for stitching images taken by an array camera in an imaging system composed of an array camera and a shooting platform, characterized in that: A random circular spot pattern is used to obtain a stitching matrix for stitching images taken by the array camera, and the images taken by the array camera are stitched according to the stitching matrix; the random circular spot pattern is a pattern composed of a number of randomly generated non-overlapping circular spots, and the position distribution of each circular spot in the same random circular spot pattern is different, and the position distribution of the circular spots between different random circular spot patterns is different.
2. The image stitching method according to claim 1, characterized in that: The step of using a random circular spot pattern to obtain a stitching matrix for stitching images taken by an array camera includes: Laying a plurality of pictures containing only random circular spot patterns on a shooting platform, so that at least one of the pictures is within the shooting range of each camera in the array camera and at least one of the same pictures is included in the images shot by adjacent cameras; The pictures are photographed with an array camera, stitching characteristic information of random circular spot patterns in each photographed image is extracted, and a stitching matrix for stitching the photographed images is obtained according to the extracted stitching characteristic information.
3. The image stitching method according to claim 1 or 2, characterized in that: The generation of random circular spot patterns includes the following steps: S11, generating a white pattern, and generating a black border on the white pattern; S12, randomly selecting a point in the black border, arbitrarily selecting a radius value within the set radius range, taking the selected point as the center of the circle and the selected radius value as the radius, generating a black circular spot in the black border, and making the black circular spot not overlap with the black border; S13, repeat step S12 until the number of circular spots in the black frame reaches a set value.
4. The image stitching method according to claim 3, characterized in that: In step S13, when generating circular spots within the black frame, the step of making the generated circular spots non-overlapping with each other includes: S131, when generating a new circular spot, taking the currently selected point as the center of the circle and the multiple of the currently selected radius value as the radius, determining whether there are black pixels in the circular area; S132: If there are black pixels, reselect other points and radius values and repeat step S131; if there are no black pixels, generate a circular spot with the currently selected points and radius values.
5. The image stitching method according to claim 3, characterized in that: In step S12, the upper limit is obtained by multiplying the length of the black frame by 10 and then dividing it by the set number of circular spots, and the lower limit is obtained by multiplying the width of the black frame by 10 and then dividing it by the set number of circular spots. The upper limit and the lower limit are used as the set radius range.
6. The image stitching method according to claim 2, characterized in that: The step of extracting the stitching characteristic information of the random circular spot pattern in each captured image includes: Binarize the captured image and find the connected domain in the captured image; Sort the connected domains in descending order according to their area, select multiple connected domains with larger areas, and fill the unselected connected domains; Identify the circular spots in the image that are located in the selected connected domain, and obtain the center of the identified circular spots; Calculate the feature descriptor of the center of the identified circular spot.
7. The image stitching method according to claim 6, characterized in that: The step of obtaining a stitching matrix for stitching the captured images according to the extracted stitching feature information includes: The feature descriptors extracted from the images taken by different cameras are matched, the homography matrix is calculated based on the coordinates of the circle center obtained by matching, and the stitching matrix for stitching the captured images is obtained according to the homography matrix.
8. The image stitching method according to claim 1, characterized in that: In the operations of acquiring the stitching matrix and stitching the images captured by the array camera according to the stitching matrix, the relative position between the array camera and the shooting platform remains unchanged.
9. An image stitching system, characterized in that: include: An image processing unit, the image processing unit is used to process the image taken by the array camera and extract the splicing feature information of the random circular spot pattern in the image; a stitching matrix acquisition unit, wherein the stitching matrix acquisition unit acquires a stitching matrix for stitching the captured images according to the extracted stitching feature information; An image stitching unit is used to stitch images taken by the array camera according to a stitching matrix.
10. The image stitching system according to claim 9, characterized in that: It also includes a random circular spot pattern generating unit, which is used to generate a random circular spot pattern.
