A method for obtaining seamless image stitching parameters for a panoramic observation device

Abstract

This invention discloses a method for obtaining seamless image stitching parameters for panoramic observation equipment, relating to the field of image processing technology. The method involves calculating the linear weights of the target feature vector group and the reference vector group for each type of defect. Based on the defect type, the linear weights are aggregated to obtain the linear weight matrix of the target feature vector and each type of defect. The contribution of each type of image feature in similarity comparison is calculated to obtain important contributing features. In the current stitched image pair, image slices containing important contributing features are obtained. Images contained in these image slices are not selected as reference images for the stitching algorithm, or the weight of images contained in the image slices in the stitching algorithm is reduced. The stitched image pair is then stitched again to obtain a new stitched image. By combining multimodal feature extraction, error mode reasoning, and dynamic optimization strategies, dynamic analysis and self-correction are performed, significantly improving the accuracy of image stitching.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, specifically a method for obtaining seamless image stitching parameters for panoramic observation equipment. Background Technology

[0002] Panoramic image stitching is the process of combining multiple images into a seamless panoramic or high-resolution image. A common method is to use feature matching algorithms to match feature points, ensuring that corresponding points from two images coincide, thus obtaining the stitched result.

[0003] To improve the accuracy of image stitching, the selection of feature points needs to be adjusted. For example, Chinese patent CN119048344A, entitled "A Remote Sensing Image Stitching Method, Apparatus, Computer Equipment, and Medium," discloses a method for filtering feature points within the neighborhood of a matching point to improve feature point selection accuracy. However, if the feature points within the neighborhood of a matching point describe similar information, effective filtering of feature points becomes impossible, easily leading to stitching misalignment. Furthermore, performing pairwise operations on feature points within the neighborhood of a matching point significantly increases the computational load on the computer system, hindering the deployment of image stitching algorithms. Summary of the Invention

[0004] The purpose of this invention is to provide a method for obtaining seamless image stitching parameters for panoramic observation devices, so as to solve the problems raised in the prior art.

[0005] To achieve the above objectives, the present invention provides the following technical solution: a method for obtaining seamless image stitching parameters for a panoramic observation device;

[0006] Step S100: Obtain the image stitching history of the distributed panoramic observation device, collect image information with defects after image stitching algorithm processing, label and classify the defective images, and collect image data sets corresponding to various defect types.

[0007] Step S200: Collect image features from images of each type of defect, compile feature vectors for each defect feature, and obtain a reference vector set for all defect types;

[0008] Step S300: Identify defects in the current image stitching area, obtain the defects existing in the current image stitching area and related images including the defects, and record the images as target defect images;

[0009] Step S400: Extract image features from the target defect image, compile the image feature vectors of the target defect image, and obtain the target feature vector group;

[0010] Step S500: Calculate the linear weights of the target feature vector group and the reference vector group for each type of defect, and aggregate the linear weights according to the defect type to obtain the linear weight matrix of the target feature vector and each type of defect.

[0011] Step S600: Calculate the contribution of each type of image feature in the similarity comparison to obtain the important contributing features in the image features;

[0012] Step S700: In the current image pair to be stitched, obtain the image slices that have important contributing features, and do not select the images contained in the image slices as reference images for the stitching algorithm or reduce the weight of the images contained in the image slices in the stitching algorithm;

[0013] Step S800: Perform image stitching again on the stitched image pair to obtain a new stitched image.

[0014] Furthermore, step S100 includes:

[0015] Step S101: Combine two images to be stitched into an image pair. The image after stitching the two images to be stitched is called the first stitched image. In the first stitched image, obtain the stitching area of ​​the two images to be stitched. The stitching area includes the seam of the images and the area where the images overlap and merge.

[0016] Step S102: Obtain the first spliced ​​image with defects in the splicing area. Defects include visible splicing marks, ghosting in the splicing area, color difference between the images before and after splicing, distortion of the images before and after splicing, and incompleteness of the spliced ​​image. Label and classify the defects, and collect the first spliced ​​images of each defect type to obtain a defect image classification set.

[0017] Step S103: Mark the location of the defect in the stitching area, set a distance threshold d, and record the image area in the first stitched image that is within the range of the defect location at a distance less than or equal to d as the defect-associated area.

[0018] In commonly used image stitching algorithms, such as SIFT, the basic process is to detect feature points in the image, describe the corresponding feature points through relevant algorithms, collect the feature point set of the feature points, and perform feature point matching based on the feature points in the feature set to stitch the image together.

[0019] If the features in an image are not obvious or there are repetitive features, the region matching algorithm may have difficulty finding the matching region accurately. Therefore, it is necessary to guide the image stitching algorithm to improve the accuracy of image stitching.

[0020] Furthermore, step S200 includes:

[0021] Step S201: In the defect image classification set corresponding to each defect type, obtain the image of the defect-associated region of each first stitched image, and obtain the image features of the image. The image features include: image texture features, image frequency domain features, image geometric features, or a combination of two or more of the image matching features.

[0022] Step S202: Collect the feature vector of each image feature. In each type of defect, according to the category of image feature, the feature vector of the image feature is collected separately to obtain the vector cluster of each image feature category. By using the vector clustering method, the vector corresponding to the clustering feature of each vector cluster is obtained, and the vector is recorded as the reference vector of the image feature category.

[0023] Step S203: Collect the reference vectors corresponding to all image features of the same defect type to obtain a reference vector group for each defect type.

[0024] Furthermore, step S300 includes:

[0025] Step S301: Obtain the current image pair composed of the two images to be stitched together, and record the image after stitching the current image pair as the second stitched image. Perform defect identification along the stitching area of ​​the second stitched image.

[0026] Step S302: When a defect is detected, obtain the defect-associated region of the corresponding defect, and record the image of the defect-associated region as the target defect image.

[0027] Furthermore, step S500 includes:

[0028] Step S501: Obtain the linear components of the target feature vector group in the linear space formed by the reference vector group, and collect the linear components to obtain the linear weight matrix of the reference vector group.

[0029] Step S502: Collect the weight matrices of the target vector group with respect to all reference vector groups to obtain a set of weight matrices;

[0030] Step S503: Calculate the matching probability between the linear weight matrix of each defect type and the target feature vector group, and take the weight matrix with the highest matching probability as the principal weight matrix.

[0031] Obtain the target feature vector set φ new The linear correlation weight ω in the linear space formed by the reference vector set of the c-th defect type c Calculate the logarithmic probability z c , z c =ω c T φ new +b c b cThe bias term corresponding to the defect type is used to improve the classification accuracy. It can be updated iteratively by an iterative algorithm, such as the quasi-Newton method, to finally obtain a suitable bias term.

[0032] The z-value represents the strength of the linear mapping of the target vector group in the error category space. The larger the z-value, the more the target feature vector group conforms to the distribution characteristics of a certain type of defect.

[0033] Calculate the matching probability P of the c-th defect type. c , Where C represents the total number of defect types, exp represents an exponential function with the natural constant e as the base, and zn represents the logarithmic probability corresponding to the nth defect type.

[0034] Furthermore, step S600 includes:

[0035] Step S601: Multiply the image feature vector corresponding to the j-th type of image feature in the target reference vector group with the j-th linear weight in the principal weight matrix to obtain the feature contribution value of the j-th type of image feature;

[0036] Step S602: Traverse the feature contribution values ​​of all categories of image features in the target reference vector group, calculate the normalization ratio of each feature contribution value, and obtain the feature contribution degree of each category of image features;

[0037] Step S603: Sort all image feature categories by feature contribution from high to low, and take the top k image feature categories in the target reference vector group as important contributing features.

[0038] Furthermore, step S700 includes:

[0039] Step S701: In the two images of the current image pair, obtain the overlapping images of the images in the defect-related area respectively, and combine the two overlapping images in the image pair to obtain the retrieval and analysis area;

[0040] In the second stitched image, the fusion region or seam of the two stitched images is obtained. According to step S103, the range d at a distance from the fusion region or seam is defined as the defect-related region. The original image of the defect-related region is obtained in the original image, and feature analysis is performed on the image in the original image.

[0041] Step S702: Divide the retrieval and analysis area into several unit regions and obtain image slices corresponding to the unit regions with important contributing features.

[0042] By identifying the image features in the current defective area and matching them with different defective features in historical data, further analysis is conducted to determine which image features affect the image stitching result. In this way, the weight of these areas is reduced, or the relevant image stitching algorithm is prohibited from collecting feature points in image areas with these image features, thereby improving the success rate of image stitching.

[0043] Compared with existing technologies, the beneficial effects of this invention are as follows: By systematically analyzing historical stitching failure cases and accumulating error features in historical data, this invention uses pattern recognition and feature verification methods to identify the specific reasons affecting image stitching accuracy. By combining multimodal feature extraction, error pattern reasoning, and dynamic optimization strategies, dynamic analysis and self-correction are performed, significantly improving image stitching accuracy. Simultaneously, this invention, through image feature transport, gradually narrows the scope of image feature processing, which helps improve the efficiency of image stitching work. Attached Figure Description

[0044] Figure 1 This is a schematic diagram of the structure of a method for obtaining seamless image stitching parameters for a panoramic observation device according to the present invention. Detailed Implementation

[0045] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0046] Example: Figure 1 As shown, the present invention provides a technical solution: 1. A method for obtaining seamless image stitching parameters of a panoramic observation device.

[0047] Step S100: Obtain the image stitching history of the distributed panoramic observation device, collect image information with defects after image stitching algorithm processing, label and classify the defective images, and collect image data sets corresponding to various defect types.

[0048] Step S100 includes:

[0049] Step S101: Combine two images to be stitched into an image pair. The image after stitching the two images to be stitched is called the first stitched image. In the first stitched image, obtain the stitching area of ​​the two images to be stitched. The stitching area includes the seam of the images and the area where the images overlap and merge.

[0050] Step S102: Obtain the first spliced ​​image with defects in the splicing area. Defects include visible splicing marks, ghosting in the splicing area, color difference between the images before and after splicing, distortion of the images before and after splicing, and incompleteness of the spliced ​​image. Label and classify the defects, and collect the first spliced ​​images of each defect type to obtain a defect image classification set.

[0051] Step S103: Mark the location of the defect in the stitching area, set a distance threshold d, and record the image area in the first stitched image that is within the range of the defect location at a distance less than or equal to d as the defect-associated area.

[0052] Step S200: Collect image features from images of each type of defect, compile feature vectors for each defect feature, and obtain a reference vector set for all defect types;

[0053] Step S200 includes:

[0054] Step S201: In the defect image classification set corresponding to each defect type, obtain the image of the defect-associated region of each first stitched image, and obtain the image features of the image. The image features include: image texture features, image frequency domain features, image geometric features, or a combination of two or more of the image matching features.

[0055] Step S202: Collect the feature vector of each image feature. In each type of defect, according to the category of image feature, the feature vector of the image feature is collected separately to obtain the vector cluster of each image feature category. By using the vector clustering method, the vector corresponding to the clustering feature of each vector cluster is obtained, and the vector is recorded as the reference vector of the image feature category.

[0056] Step S203: Collect the reference vectors corresponding to all image features of the same defect type to obtain a reference vector group for each defect type.

[0057] Step S300: Identify defects in the current image stitching area, obtain the defects existing in the current image stitching area and related images including the defects, and record the images as target defect images;

[0058] Step S300 includes:

[0059] Step S301: Obtain the current image pair composed of the two images to be stitched together, and record the image after stitching the current image pair as the second stitched image. Perform defect identification along the stitching area of ​​the second stitched image.

[0060] Step S302: When a defect is detected, obtain the defect-associated region of the corresponding defect, and record the image of the defect-associated region as the target defect image.

[0061] Step S400: Extract image features from the target defect image, collect the image feature vectors of the target defect image, and obtain the target feature vector group.

[0062] Step S500: Calculate the linear weights of the target feature vector group and the reference vector group for each type of defect, and aggregate the linear weights according to the defect type to obtain the linear weight matrix of the target feature vector and each type of defect.

[0063] Step S500 includes:

[0064] Step S501: Obtain the linear components of the target feature vector group in the linear space formed by the reference vector group, and collect the linear components to obtain the linear weight matrix of the reference vector group.

[0065] Step S502: Collect the weight matrices of the target vector group with respect to all reference vector groups to obtain a set of weight matrices;

[0066] Step S503: Calculate the matching probability between the linear weight matrix of each defect type and the target feature vector group, and take the weight matrix with the highest matching probability as the principal weight matrix.

[0067] In the embodiment, there are two types of defects, which are labeled as abnormal texture error and duplicate texture error respectively. Abnormal texture error indicates that there is obvious seam in the splicing result, and duplicate texture error indicates that there is ghosting in the splicing result.

[0068] Extract image features from the target defect image and normalize the image features to obtain φ. new , φ new = (0.8, 1.2, ..., -0.3);

[0069] Obtain the relevant parameters ω1 and b1 for the abnormal texture error, where ω1 = (0.5, -0.2, ..., b1).

[0070] 1.1), b1 = 0.3;

[0071] Obtain the relevant parameters ω2 and b2 for the duplicate texture error, where ω2 = (-0.1, 0.6, ..., b2).

[0072] 0.9), b2 = -0.5;

[0073] Calculate the log-probability z1 of the anomalous texture error, z1 = 0.5 × 0.8 + (-0.2) × 1.2 + ...

[0074] +1.1×(-0.3)+0.3=2.1;

[0075] Calculate the logarithmic probability z2 of the duplicate texture error, z2 = (-0.1) × 0.8 + 0.6 × 1.2 + ...

[0076] +0.9×(-0.3)-0.5=1.4

[0077] Calculate the matching probability P1 for abnormal texture errors and the matching probability P1 for duplicate texture errors.

[0078] Probability P2;

[0079]

[0080] P1 > P2, therefore abnormal texture errors are the main cause.

[0081] Step S600: Calculate the contribution of each type of image feature in the similarity comparison to obtain the important contributing features in the image features;

[0082] Step S600 includes:

[0083] Step S601: Multiply the image feature vector corresponding to the j-th type of image feature in the target reference vector group with the j-th linear weight in the principal weight matrix to obtain the feature contribution value of the j-th type of image feature;

[0084] Step S602: Traverse the feature contribution values ​​of all categories of image features in the target reference vector group, calculate the normalization ratio of each feature contribution value, and obtain the feature contribution degree of each category of image features;

[0085] Step S603: Sort all image feature categories by feature contribution from high to low, and take the top k image feature categories in the target reference vector group as important contributing features.

[0086] In the embodiment, φ is calculated. new Contrib b , Where, ω 1，b φ represents the b-th element in ω1. new b φ new The b-th element in ω 1，q Let φ represent the q-th element in ω1. new q φ new The q-th element in the matrix, M represents φ. new The total number of elements in the middle;

[0087] Based on the contribution of the elements contained in the feature vector, calculate the sum of the contributions of each image feature vector, including the elements, and denot it as the feature vector contribution. From φ new The feature vector containing these elements with large contributions is obtained from φ. newIt is composed of several feature vectors. By identifying which feature vectors played a role in the similarity comparison, the image features represented by these feature vectors match the image features when the defect occurred.

[0088] The first determined image features of class j are in φ new In this embodiment, the vector elements of the feature vector of the j-th image are the 4th to 6th elements of φnew, denoted as φ. new 4. φ new 5 and φ new 6, respectively φ new 4 =-0.5, φ new 5 =0.3, φ new 6 =0.4;

[0089] In the embodiments

[0090] Get ω 1，4 =-0.1, ω 1，5 =0.4, ω 1，6 =0.2

[0091] ω 1，4 ×φ new 4 =0.05, ω 1，5 ×φ new 5 =0.12, ω 1，6 ×φ new 6 =0.08;

[0092] The contribution of the j-th image feature is calculated as (0.05 + 0.12 + 0.08) / 4.48 = 0.056.

[0093] To prevent the cancellation of positive and negative values ​​from reducing the total number of denominators, a positive value is used as the reference base for normalization for the same denominator. The numerator retains the enhancement or suppression effect of image features on erroneous analogies by distinguishing positive and negative values. A positive value in the numerator indicates that the image feature enhances the probability of the current defect category, while a negative value in the numerator indicates that the image feature suppresses the probability of the current defect category.

[0094] Based on the contribution of the positional elements, select the elements with the larger contribution, starting from φ. new The feature vector containing these elements with large contributions is obtained from φ. new It is composed of several feature vectors. By identifying the elements, it is determined which feature vectors played a role in the similarity comparison. The image features represented by these feature vectors match the image features at the time the defect occurred.

[0095] Step S700: In the current image pair to be stitched, obtain the image slices that have important contributing features, and do not select the images contained in the image slices as reference images for the stitching algorithm or reduce the weight of the images contained in the image slices in the stitching algorithm;

[0096] Step S700 includes:

[0097] Step S701: In the two images of the current image pair, obtain the overlapping images of the images in the defect-related area respectively, and combine the two overlapping images in the image pair to obtain the retrieval and analysis area;

[0098] Step S702: Divide the retrieval and analysis area into several unit regions and obtain image slices corresponding to the unit regions with important contributing features.

[0099] In the second stitched image, the fusion region or seam of the two stitched images is obtained. Following the method in step S103, the range 'd' from the fusion region or seam is defined as the defect-associated region. In the two images of the current image pair, the image region overlapping with the defect-associated region is obtained, and this image region is recorded as the retrieval and analysis region.

[0100] In this embodiment, two images, pic1 and pic2, are stitched together to form an image picx.

[0101] In the first embodiment, two images are overlapped and merged. The merged region F0 is obtained in picx. The region with a distance d from F0 is defined as the defect-associated region. When F0 is the overlapping region of pic1 and pic2, the overlapping regions are obtained in the common parts of pic1 and pic2, and are denoted as F01 and F02 respectively.

[0102] A portion of the image in the defect-related region is the same as the image in pic1. This portion of the image is denoted as F1. F1 and F01 together form the first part of the retrieval and analysis region, and F2 and F02 together form the first part of the retrieval and analysis region.

[0103] In another embodiment, two images, pic1 and pic2, are directly stitched together without overlapping areas. The seam between the two images, within a distance d from the seam, is defined as the defect-associated region, which is the retrieval and analysis region.

[0104] Step S800: Perform image stitching again on the stitched image pair to obtain a new stitched image.

[0105] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the invention can be implemented in other specific forms without departing from its spirit or essential characteristics. Therefore, the embodiments should be considered in all respects as exemplary and non-limiting, and the scope of the invention is defined by the appended claims rather than the foregoing description. Thus, all variations falling within the meaning and scope of equivalents of the claims are intended to be included within the present invention. No reference numerals in the claims should be construed as limiting the scope of the claims.

Claims

1. A method for obtaining seamless image stitching parameters for a panoramic observation device, characterized in that: The method includes the following steps: Step S100: Obtain the image stitching history of the distributed panoramic observation device, collect image information with defects after image stitching algorithm processing, label and classify the defective images, and collect image data sets corresponding to various defect types. Step S200: Collect image features from images of each type of defect, compile feature vectors for each defect feature, and obtain a reference vector set for all defect types; Step S300: Identify defects in the current image stitching area, obtain the defects existing in the current image stitching area and related images including the defects, and record the images as target defect images; Step S400: Extract image features from the target defect image, compile the image feature vectors of the target defect image, and obtain the target feature vector group; Step S500: Calculate the linear weights of the target feature vector group and the reference vector group for each type of defect, and aggregate the linear weights according to the defect type to obtain the linear weight matrix of the target feature vector and each type of defect. Step S600: Calculate the contribution of each type of image feature in the similarity comparison to obtain the important contributing features in the image features; Step S700: In the current image pair to be stitched, obtain an image slice with important contributing features, and do not select the image contained in the image slice as the reference image of the stitching algorithm, or reduce the weight of the image contained in the image slice in the stitching algorithm; Step S800: The stitched image pair is stitched together again to obtain a new stitched image.

2. The method for obtaining seamless image stitching parameters for a panoramic observation device according to claim 1, characterized in that: Step S100 includes: Step S101: Combine two images to be stitched into an image pair, and record the image after stitching the two images to be stitched as the first stitched image. In the first stitched image, obtain the stitching area of ​​the two images to be stitched. The stitching area includes the seam of the images and the area where the images overlap and merge. Step S102: Obtain the first spliced ​​image with defects in the splicing area. The defects include visible splicing marks, ghosting in the splicing area, color difference between the images before and after splicing, distortion of the images before and after splicing, and incompleteness of the spliced ​​image. Mark and classify the defects, and collect the first spliced ​​images of each defect type to obtain a defect image classification set. Step S103: Mark the location of the defect in the stitching area, set a distance threshold d, and record the image area in the first stitched image that is within the range of the defect location at a distance less than or equal to d as the defect-associated area.

3. The method for obtaining seamless image stitching parameters for a panoramic observation device according to claim 2, characterized in that: Step S200 includes: Step S201: In the defect image classification set corresponding to each defect type, obtain the image of the defect-associated region of each first stitched image, and obtain the image features of the image, wherein the image features include: a combination of two or more of the following: image texture features, image frequency domain features, image geometric features, or image matching features; Step S202: Collect the feature vector of each image feature. In each type of defect, according to the category of image feature, the feature vector of the image feature is collected separately to obtain the vector cluster of each image feature category. By using the vector clustering method, the vector corresponding to the clustering feature of each vector cluster is obtained, and the vector is recorded as the reference vector of the image feature category. Step S203: Collect the reference vectors corresponding to all image features of the same defect type to obtain a reference vector group for each defect type.

4. The method for obtaining seamless image stitching parameters for a panoramic observation device according to claim 3, characterized in that: Step S300 includes: Step S301: Obtain the current image pair composed of the two images to be stitched together, and record the image after stitching the current image pair as the second stitched image. Perform defect identification along the stitching area of ​​the second stitched image. Step S302: When a defect is detected, obtain the defect-associated region of the corresponding defect, and record the image of the defect-associated region as the target defect image.

5. The method for obtaining seamless image stitching parameters for a panoramic observation device according to claim 4, characterized in that: Step S500 includes: Step S501: Obtain the linear components of the target feature vector group in the linear space formed by the reference vector group, and collect the linear components to obtain the linear weight matrix of the reference vector group. Step S502: Collect the weight matrices of the target vector group with respect to all reference vector groups to obtain a set of weight matrices; Step S503: Calculate the matching probability between the linear weight matrix of each defect type and the target feature vector group, and take the weight matrix with the highest matching probability as the principal weight matrix.

6. The method for obtaining seamless image stitching parameters for a panoramic observation device according to claim 5, characterized in that: Step S600 includes: Step S601: Multiply the image feature vector corresponding to the j-th type of image feature in the target reference vector group with the j-th linear weight in the principal weight matrix to obtain the feature contribution value of the j-th type of image feature; Step S602: Traverse the feature contribution values ​​of all categories of image features in the target reference vector group, calculate the normalization ratio of each feature contribution value, and obtain the feature contribution degree of each category of image features; Step S603: Sort all image feature categories by feature contribution from high to low, and take the top k image feature categories in the target reference vector group as important contributing features.

7. The method for obtaining seamless image stitching parameters for a panoramic observation device according to claim 6, characterized in that: Step S700 includes: Step S701: In the two images of the current image pair, obtain the overlapping images of the images in the defect-related area respectively, and combine the two overlapping images in the image pair to obtain the retrieval and analysis area; Step S702: Divide the retrieval and analysis area into several unit regions and obtain image slices corresponding to the unit regions with important contributing features.

8. The method for obtaining seamless image stitching parameters for a panoramic observation device according to claim 7, characterized in that: Methods for obtaining the search and analysis region include: In the second stitched image, the fusion region or seam of the two stitched images is obtained. According to the method in step S103, the range of distance d from the fusion region or seam is defined as the defect-associated region. In the two images in the current image pair, the image region that overlaps with the defect-associated region is obtained, and the image region is recorded as the retrieval analysis region.

Images