A panoramic image detection method and device based on image stitching

Abstract

The present application relates to the technical field of image detection, in particular to a panoramic image detection method and device based on image stitching, comprising: performing edge detection processing on each local detection image to obtain an edge image; extracting feature points from the edge image to obtain a feature point set of each local detection image; calculating the number of feature points and the proportion of feature point types in the overlapping area of the images according to the feature point set of each local detection image; and determining the stitching priority of the local detection image according to the number of feature points and the proportion of feature point types, combined with the image definition difference value. The present application determines the stitching priority according to the number and proportion of feature points, which helps to select the best image order and area in the stitching process, thereby reducing stitching errors and visual inconsistency and improving the overall quality of panoramic images.

Description

Technical Field

[0001] This invention relates to the field of image detection technology, specifically to a panoramic image detection method and device based on image stitching. Background Technology

[0002] Optical devices are equipment or components used to manipulate, regulate, and utilize light. They are widely used in various technical fields, including but not limited to lighting, imaging, communication, scientific research, and medicine. Optical devices require extremely high precision in manufacturing and application, and any tiny defect can affect their performance. In the production process, panoramic image inspection can be used to achieve real-time monitoring, promptly identify problems and make adjustments, prevent defective products from entering the market, and ensure product quality.

[0003] Currently, traditional methods typically rely on simple feature matching and stitching algorithms, which are easily affected by factors such as image quality, lighting changes, and motion blur, leading to stitching errors and visual inconsistencies that affect the overall quality of panoramic images. Furthermore, many traditional detection methods use fixed detection strategies, which have poor adaptability to regions of different complexity and cannot perform targeted detection for regions with different features and texture complexities, resulting in low efficiency.

[0004] Furthermore, traditional methods often rely on a single defect detection approach, such as focusing only on grayscale abrupt changes, making it difficult to comprehensively cover different types of defects and potentially missing some subtle or special types of defects. In addition, traditional methods usually lack in-depth analysis and classification of defect types after defect detection, providing limited guidance for subsequent repair or treatment, which affects practicality and effectiveness. Summary of the Invention

[0005] To achieve the above objectives, the present invention provides the following technical solution: a panoramic image detection method based on image stitching, comprising:

[0006] Acquire multiple local detection images of the surface of the optical device; perform edge detection processing on each local detection image to obtain an edge image; extract feature points from the edge images to obtain a feature point set for each local detection image;

[0007] Based on the feature point set of each local detection image, calculate the number of feature points and the proportion of feature point types in the overlapping area of ​​the image; based on the number of feature points and the proportion of feature point types, and combined with the image sharpness difference value, determine the stitching priority of the local detection images.

[0008] The image stitching order is determined according to the stitching priority, and the feature point sets of adjacent local detection images are described to obtain the feature description vector of each feature point.

[0009] Calculate the similarity between feature descriptor vectors of feature points in adjacent local detection images, and determine the feature point pairs with similarity greater than a preset similarity threshold as matching feature point pairs in the overlapping region to obtain the feature point matching degree of the overlapping region;

[0010] Based on the matching degree of feature points in the overlapping area, the stitching parameter adjustment method is determined; based on the stitching parameter adjustment method, the transformation matrix between adjacent local images is determined; the transformation matrix is ​​used to transform and fuse the adjacent local images to generate a panoramic image of the surface of the optical device.

[0011] Preferably, after transforming and fusing adjacent local images using a transformation matrix to generate a panoramic image of the optical device surface, the method further includes:

[0012] Histogram equalization, sharpening, and noise reduction are performed on the panoramic image to obtain the enhanced panoramic image.

[0013] Based on the distribution of texture features in the enhanced panoramic image, determine whether the texture distribution is uniform or non-uniform.

[0014] For areas with uniform texture distribution, the entire area with uniform texture distribution is taken as the defect detection area. For areas with uneven texture distribution, the boundary of the defect detection area is determined according to the location of texture abrupt change.

[0015] Based on the texture complexity within the defect detection area, the defect detection area is divided into a different number of sub-regions; for each sub-region, the defect recognition method is determined based on the pixel grayscale variation characteristics within the sub-region.

[0016] Defects are identified in sub-regions according to the defect identification method. When a defect is identified, its shape and size features are extracted. Based on the shape and size features of the defect and a pre-set defect feature library, the defect type is determined.

[0017] Display a panoramic image containing the detection results. The detection results include the defect type and the location information of the defect in the panoramic image. During the display of the panoramic image containing the detection results, if no operation to adjust the annotation style of the detection results is received, the detection results are annotated in the panoramic image using the default annotation style. If an operation to adjust the annotation style of the detection results is received, the annotation style of the detection results in the panoramic image is switched to another annotation style other than the current annotation style among a variety of preset annotation styles.

[0018] Preferably, based on the feature point set of each local detection image, the number of feature points and the proportion of feature point types within the overlapping region of the images are calculated, including:

[0019] Corner detection is performed on the edge image to obtain multiple corner points;

[0020] Texture feature analysis is performed on the edge image to obtain multiple feature points with specific texture features;

[0021] By combining multiple corner points and multiple feature points with specific texture characteristics, the total number of feature points in the overlapping area and the proportion of different types of feature points are counted.

[0022] Preferably, the stitching priority of local detection images is determined based on the number of feature points and the proportion of feature point types, combined with the image sharpness difference value, including:

[0023] If the number of feature points is greater than or equal to the preset feature point number threshold and the image sharpness difference value is less than the preset sharpness difference value threshold, the stitching priority is high priority;

[0024] If the number of feature points is less than the preset feature point number threshold or the image clarity difference value is greater than or equal to the preset clarity difference value threshold, the stitching priority is low.

[0025] Preferably, the splicing parameters can be adjusted by adjusting the rotation angle and the translation distance;

[0026] Based on the matching degree of feature points in the overlapping area, the method for adjusting the splicing parameters is determined, including:

[0027] If the feature point matching degree is less than the preset feature point matching degree threshold, the rotation angle adjustment method is to determine the rotation angle adjustment amount based on the feature point distribution density.

[0028] If the feature point matching degree is greater than or equal to the preset feature point matching degree threshold, the rotation angle adjustment method is to determine the rotation angle adjustment amount based on the edge alignment degree.

[0029] Specifically, the rotation angle adjustment is determined based on the density of feature point distribution or edge alignment, and the translation distance adjustment is determined based on the number of matched feature points or the rate of change of overlapping area.

[0030] Preferably, the transformation matrix between adjacent local images is determined according to the stitching parameter adjustment method, and the adjacent local images are transformed and fused using the transformation matrix to generate a panoramic image of the optical device surface, including:

[0031] Construct an affine transformation matrix based on the rotation angle adjustment and translation distance adjustment;

[0032] The affine transformation matrix is ​​used to map the local images to be stitched onto the coordinate system of the reference image, so that the overlapping areas are aligned.

[0033] The seams of the aligned images are fused to eliminate the stitching gaps and generate a panoramic image of the optical device surface.

[0034] Preferably, based on the texture complexity within the defect detection area, the defect detection area is divided into a different number of sub-regions, including:

[0035] For defect detection regions with texture complexity less than a preset complexity threshold, the defect detection region is divided into sub-regions of a first detection quantity;

[0036] For defect detection regions with texture complexity greater than or equal to a preset complexity threshold, the defect detection region is divided into sub-regions with a second number of detections; wherein, the first number of detections is less than the second number of detections.

[0037] Among them, the area of ​​a single sub-region is negatively correlated with the texture complexity.

[0038] Preferably, the defect identification method is gray-scale abrupt change feature identification and gray-scale gradual change feature identification;

[0039] For each sub-region, the defect identification method is determined based on the pixel grayscale variation characteristics within the sub-region, including:

[0040] If the pixel grayscale change feature is that there is a jump within a preset first distance, the defect identification method is to identify based on the grayscale abrupt change feature.

[0041] If the pixel grayscale change feature is that it gradually changes within a preset second distance, the defect identification method is based on the grayscale gradient feature; wherein, the preset first distance is smaller than the preset second distance.

[0042] Preferably, based on the shape and size characteristics of the defect, and in conjunction with a pre-defined defect feature library, the defect type is determined, including:

[0043] If the edge contour of the defect is more or less similar to the contour of the preset regular shape and the size feature is within the preset size range, the defect type is a repairable defect.

[0044] If the edge contour of the defect is less similar to the contour of the preset regular shape than the preset similarity or the size feature is not within the preset size range, the defect type is an unrepairable defect.

[0045] A panoramic image detection device based on image stitching, applicable to the aforementioned panoramic image detection method based on image stitching, comprising:

[0046] The image acquisition unit is used to acquire multiple local detection images of the surface of the optical device; perform edge detection processing on each local detection image to obtain an edge image; and extract feature points from the edge image to obtain a feature point set for each local detection image.

[0047] The feature point calculation unit is used to calculate the number of feature points and the proportion of feature point types in the overlapping area of ​​each local detection image based on the feature point set of each local detection image; and to determine the stitching priority of the local detection images based on the number of feature points and the proportion of feature point types, combined with the image sharpness difference value.

[0048] The feature description unit is used to determine the image stitching order according to the stitching priority, perform feature description on the feature point set of adjacent local detection images, and obtain the feature description vector of each feature point.

[0049] The similarity calculation unit is used to calculate the similarity between feature point feature description vectors in adjacent local detection images. Feature point pairs with similarity greater than a preset similarity threshold are identified as matching feature point pairs in the overlapping region, and the feature point matching degree in the overlapping region is obtained.

[0050] The image adjustment unit is used to determine the stitching parameter adjustment method based on the feature point matching degree of the overlapping area; determine the transformation matrix between adjacent local images based on the stitching parameter adjustment method; and use the transformation matrix to transform and fuse the adjacent local images to generate a panoramic image of the surface of the optical device.

[0051] Compared with the prior art, the beneficial effects of the present invention are:

[0052] (1) This invention determines the stitching priority based on the number and type ratio of feature points, which helps to select the best image order and region during the stitching process, thereby reducing stitching errors and visual inconsistencies and improving the overall quality of panoramic images; and by dividing the defect detection area into different sub-regions according to the texture complexity, different detection strategies can be adopted for regions with different complexities, thereby improving the flexibility and efficiency of detection.

[0053] (2) This invention provides two defect recognition methods: gray-scale abrupt change and gray-scale gradient. The gray-scale changes of different pixels can be flexibly adjusted to more comprehensively cover possible defect types. By combining the shape and size features of the defect with the preset defect feature library, the defect type can be effectively determined and guidance can be provided for subsequent repair or processing, thus improving practicality. Furthermore, by performing histogram equalization, sharpening and noise reduction on the generated panoramic image, the final output image is ensured to be clearer and more detailed, making the defect easier to identify and analyze. Attached Figure Description

[0054] Figure 1 This is a schematic flowchart of the overall method in one embodiment of the present invention;

[0055] Figure 2 This is a schematic diagram of the overall device architecture in one embodiment of the present invention.

[0056] In the diagram: 1. Image acquisition unit; 2. Feature point calculation unit; 3. Feature description unit; 4. Similarity calculation unit; 5. Image adjustment unit. Detailed Implementation

[0057] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0058] Example 1, please refer to Figure 1 This invention provides a technical solution: a panoramic image detection method based on image stitching, comprising:

[0059] S1. Acquire multiple local detection images of the optical device surface; perform edge detection processing on each local detection image to obtain an edge image; extract feature points from the edge images to obtain a feature point set for each local detection image;

[0060] S2. Based on the feature point set of each local detection image, calculate the number of feature points and the proportion of feature point types in the overlapping area of ​​the image; based on the number of feature points and the proportion of feature point types, and combined with the image sharpness difference value, determine the stitching priority of the local detection images.

[0061] S3. Determine the image stitching order according to the stitching priority, perform feature description on the feature point set of adjacent local detection images, and obtain the feature description vector of each feature point.

[0062] S4. Calculate the similarity between feature point feature description vectors in adjacent local detection images, and determine the feature point pairs with similarity greater than the preset similarity threshold as matching feature point pairs in the overlapping region to obtain the feature point matching degree of the overlapping region.

[0063] S5. Determine the stitching parameter adjustment method based on the feature point matching degree of the overlapping area; determine the transformation matrix between adjacent local images based on the stitching parameter adjustment method; use the transformation matrix to transform and fuse the adjacent local images to generate a panoramic image of the surface of the optical device.

[0064] It should be noted that multiple local detection images are captured from the surface of the optical device; for example, images of multiple regions can be captured, each covering a different part of the device surface, such as the center or edge of a lens. Edge detection is performed on each local detection image, for example, using the Canny algorithm or the Sobel operator. The purpose of this step is to extract edge information from the image for subsequent feature point extraction. After edge detection processing, an edge image of each local detection image is obtained. For example, if a local image shows a part of a lens, edge detection may reveal the outer contour of the lens and some fine scratch edges. Feature points are extracted from the edge image. Common methods include Harris corner detection, SIFT, or ORB. Through these methods, a set of feature points for each local detection image can be obtained. For example, in the edge image of a lens, feature points may correspond to obvious edge turning points or texture change locations.

[0065] Based on the feature point sets of each local detection image, the number of feature points in the overlapping area and the proportion of different types of feature points are calculated. This process helps determine which local images have higher priority during stitching. For example, if image A has 100 feature points, of which 80 are corner points and 20 are edge points, while image B has 150 feature points, of which 120 are corner points and 30 are edge points, their feature point distribution in the overlapping area can be analyzed. Based on the number of feature points, the proportion of feature point types, and the difference in image sharpness, the stitching priority of each local detection image is determined. The difference in sharpness can be realized by calculating the variance or gradient of the image. For example, if image A has a sharpness score of 0.8 and image B has a sharpness score of 0.6, and the number of feature points in their overlapping areas is similar, then image A has a higher stitching priority. The stitching priority determines the order of image stitching; for example, images with high sharpness and abundant feature points are stitched first.

[0066] Feature descriptions are performed on the feature point sets of adjacent local detection images to obtain feature description vectors. Then, the similarity between the feature point description vectors in adjacent images is calculated (e.g., using Euclidean distance or cosine similarity). Feature point pairs with similarity greater than a preset threshold are identified as matching feature point pairs in the overlapping region. For example, if a feature point in image A and a feature point in image B have a similarity in description vectors exceeding the threshold, these two feature points are considered a match. Based on the matching degree of feature points in the overlapping region, the stitching parameters (such as translation, rotation, etc.) are adjusted to optimize the image fusion effect. Based on the adjusted stitching parameters, the transformation matrix between adjacent local images is calculated. This matrix is ​​applied to transform and fuse the adjacent local images, ultimately generating a panoramic image of the optical device surface. For example, by calculating the transformation matrix, images A and B are translated and rotated to make the feature point positions in the overlapping region coincide as much as possible. Finally, the two images are seamlessly stitched together to generate a complete image of the optical device surface.

[0067] In an optional embodiment, after transforming and fusing adjacent local images using a transformation matrix to generate a panoramic image of the optical device surface, the method further includes:

[0068] Histogram equalization, sharpening, and noise reduction are performed on the panoramic image to obtain the enhanced panoramic image.

[0069] Based on the distribution of texture features in the enhanced panoramic image, determine whether the texture distribution is uniform or non-uniform.

[0070] For areas with uniform texture distribution, the entire area with uniform texture distribution is taken as the defect detection area. For areas with uneven texture distribution, the boundary of the defect detection area is determined according to the location of texture abrupt change.

[0071] Based on the texture complexity within the defect detection area, the defect detection area is divided into a different number of sub-regions; for each sub-region, the defect recognition method is determined based on the pixel grayscale variation characteristics within the sub-region.

[0072] Defects are identified in sub-regions according to the defect identification method. When a defect is identified, its shape and size features are extracted. Based on the shape and size features of the defect and a pre-set defect feature library, the defect type is determined.

[0073] Display a panoramic image containing the detection results. The detection results include the defect type and the location information of the defect in the panoramic image. During the display of the panoramic image containing the detection results, if no operation to adjust the annotation style of the detection results is received, the detection results are annotated in the panoramic image using the default annotation style. If an operation to adjust the annotation style of the detection results is received, the annotation style of the detection results in the panoramic image is switched to another annotation style other than the current annotation style among a variety of preset annotation styles.

[0074] It's important to note that before starting defect detection, a series of enhancement processes are required for the panoramic image to improve image quality and recognizability. Histogram equalization is an image processing technique that adjusts the image contrast to make the distribution of pixels at different brightness levels more uniform, thereby enhancing image details. For example, if an image of an optical lens appears too dark in some areas due to uneven lighting, histogram equalization can make all areas more clearly display the lens details. Sharpening increases the edge contrast of the image, making the details in the image more prominent and aiding in subsequent defect identification. For example, after sharpening, tiny scratches or dust particles on the lens surface become more obvious, aiding in subsequent detection. Noise reduction removes random noise from the image, improving image quality and ensuring that subsequent analysis focuses only on true texture features. For example, if the original image contains many random noise points, noise reduction will remove this noise, preserving a clear lens surface texture.

[0075] The processed panoramic image will display clearer texture features. The next step is to analyze the distribution of these texture features. By analyzing the texture features in the image, we can determine which areas have a uniform texture distribution (i.e., consistent texture features) and which areas have an uneven texture distribution (i.e., abrupt changes or variations). For example, in the processed image, some areas exhibit a uniform texture pattern, while other areas show obvious texture abrupt changes, such as a significant scratch or stain on a part of the lens. Based on the analysis results of the texture features, we determine the defect detection areas. We consider the entire uniform texture area as a potential defect detection area. For example, if a certain area of ​​a lens has a uniform texture, it may indicate that this part has no obvious defects and is therefore saved as a normal area. We determine the boundaries of the defect detection area based on the location of texture abrupt changes. For example, near a discovered scratch, this area can be marked as a defect detection area for further analysis.

[0076] The defect detection area is further subdivided, and the characteristics of each sub-region are analyzed for defect identification. Based on texture complexity, the detection area is divided into multiple sub-regions for more detailed analysis. For example, a large, uneven area can be divided into several sub-regions based on different texture features, such as scratch areas and blemish areas. For each sub-region, a specific defect identification method is determined based on its pixel grayscale variation characteristics. For example, if a rapid change in grayscale is detected within a sub-region, it can be considered that a defect exists in that area, and a specific algorithm (such as edge detection) is used for identification. After detecting a defect, its shape and size features need to be extracted and combined with a pre-set defect feature library to determine the defect type. For example, if a scratch is detected, the system will extract the length, width, and shape features of the scratch and compare them with data in the pre-set feature library to ultimately determine that it is a "minor scratch."

[0077] Finally, the panoramic image containing the detection results is displayed, including the defect type and its location information in the panoramic image. If no style adjustment request is received, the default annotation style is used for annotation. For example, when displaying the image, the defect may be marked with a red box and labeled "minor scratch" next to it. If a style adjustment request is received, other preset annotation styles can be switched, such as changing the color or shape. For example, the user may want to change the defect annotation to a yellow circle to distinguish it from the background and other elements.

[0078] In an optional embodiment, based on the feature point sets of each local detection image, the number of feature points and the proportion of feature point types within the overlapping regions of the images are calculated, including:

[0079] Corner detection is performed on the edge image to obtain multiple corner points;

[0080] Texture feature analysis is performed on the edge image to obtain multiple feature points with specific texture features;

[0081] By combining multiple corner points and multiple feature points with specific texture characteristics, the total number of feature points in the overlapping area and the proportion of different types of feature points are counted.

[0082] It's important to note that corner detection refers to finding points with significant variations in an edge image. These points are typically located at the corners or edges of an object. Common corner detection algorithms include Harris corner detection and Shi-Tomasi corner detection. For example, when detecting the surface of a lens, multiple corners may be found at the lens's edge, near the center, or in other areas with prominent features. These corners provide important information for subsequent analysis, such as the lens's geometry and location. Texture feature analysis is used to extract regions with specific texture patterns from an image. Texture can provide important information about surface properties, such as roughness and wear. Common texture feature extraction methods include gray-level co-occurrence matrix (GLCM) and LBP (Local Binary Pattern).

[0083] In the inspection of optical devices, minute imperfections, scratches, or other irregular texture features on the lens surface may be analyzed. Through these analyses, the presence of surface defects can be identified. After completing corner detection and texture feature analysis, the next step is to combine these two types of feature points. By performing overlap analysis on the positions of corner points and texture feature points, the total number of feature points in their overlapping areas and the proportion of different types of feature points can be calculated. For example, suppose 10 corner points and 15 texture feature points are detected in a specific area, where 5 corner points overlap with 7 texture feature points. In this case, we can conclude that: Total number of feature points: 10 + 15 - 5 = 20; Proportion of different types of feature points: Corner points account for 5 / 20 of the total number of feature points in the overlapping area, and texture feature points account for 7 / 20.

[0084] In an optional embodiment, the stitching priority of the local detection images is determined based on the number of feature points and the proportion of feature point types, combined with the image sharpness difference value, including:

[0085] If the number of feature points is greater than or equal to the preset feature point number threshold and the image sharpness difference value is less than the preset sharpness difference value threshold, the stitching priority is high priority;

[0086] If the number of feature points is less than the preset feature point number threshold or the image clarity difference value is greater than or equal to the preset clarity difference value threshold, the stitching priority is low.

[0087] It should be noted that the feature point count threshold is a preset standard used to determine whether the number of feature points extracted from an image is sufficient. Feature points can be corner points or texture feature points, which provide key information for image analysis. In the detection of optical devices, sufficient feature points mean that the image can be reconstructed and analyzed more accurately. If the number of feature points in an image is greater than or equal to this preset threshold, it means that the image contains enough information and can be effectively stitched together. If the number of feature points is less than this threshold, it indicates that the image information is insufficient, which may lead to poor stitching results and thus give it low priority processing.

[0088] Sharpness difference is an indicator used to evaluate the sharpness between different images. Generally, images with higher sharpness can better present details, while images with lower sharpness may lose information due to blur. The calculation of sharpness difference is usually based on features such as image contrast and texture. If the sharpness difference between two images is less than a preset threshold, it means that they are visually similar and are easier to blend together to form a natural transition. If the sharpness difference exceeds the preset threshold, it indicates that there is a large visual difference, and the stitching may produce obvious seams or a sense of disharmony, so the priority is reduced.

[0089] In practical applications, the stitching priority is determined by a combination of two factors: the number of feature points and the difference in sharpness. For example, a high-priority image might be defined as follows: when inspecting an optical lens, if 150 feature points are extracted, which meets the preset threshold (100), and the image sharpness difference is 0.02, less than the set threshold of 0.05, then the stitching priority is high, meaning high-quality image stitching can be performed. Conversely, a low-priority image might be defined as follows: if only 80 feature points are extracted from another image, below the threshold of 100, and the image sharpness difference is 0.06, higher than the set threshold of 0.05, then the stitching priority is low, indicating that the stitching result may be unsatisfactory and should be handled with caution or additional image correction should be performed.

[0090] In one optional embodiment, the splicing parameters can be adjusted by means of rotation angle adjustment and translation distance adjustment;

[0091] Based on the matching degree of feature points in the overlapping area, the method for adjusting the splicing parameters is determined, including:

[0092] If the feature point matching degree is less than the preset feature point matching degree threshold, the rotation angle adjustment method is to determine the rotation angle adjustment amount based on the feature point distribution density.

[0093] If the feature point matching degree is greater than or equal to the preset feature point matching degree threshold, the rotation angle adjustment method is to determine the rotation angle adjustment amount based on the edge alignment degree.

[0094] Specifically, the rotation angle adjustment is determined based on the density of feature point distribution or edge alignment, and the translation distance adjustment is determined based on the number of matched feature points or the rate of change of overlapping area.

[0095] It should be noted that feature point matching degree is an indicator of the similarity between two images, representing the spatial correspondence of extracted feature points. A high matching degree indicates good similarity between the images, while a low matching degree may indicate a significant deviation. If the feature point matching degree is below a preset threshold, it means that the similarity between the two images is insufficient, and more refined adjustments are needed to improve the alignment. If the matching degree reaches or exceeds the threshold, it means that the basic alignment of the images has been completed, and more effective methods can be used for fine-tuning. Adjusting the rotation angle based on feature point distribution density: When the feature point matching degree is low, the method for adjusting the rotation angle depends on the feature point distribution density. Feature point distribution density refers to the ratio of the number of feature points in a certain region to the area of ​​that region.

[0096] For example, suppose in a blurry optical lens image, only a few feature points are extracted, and these feature points are sparsely distributed. In this case, the feature point density is low, so it's necessary to analyze the relative positions and distribution of these sparse feature points to estimate a suitable rotation angle to ensure more points can be accurately matched. Adjusting the rotation angle based on edge alignment: When the feature point matching degree is high, the rotation angle adjustment is based on edge alignment. Edge alignment refers to the alignment of two images at their edges, usually evaluated by calculating the angular difference between edge segments. For example, when processing a clear optical lens image, the number of extracted feature points is sufficient and the matching degree is high. In this case, the rotation angle can be fine-tuned by analyzing the image edges (e.g., the lens outline), thereby further improving the image overlap and ensuring precise edge alignment.

[0097] Regardless of whether the feature point matching degree is high or low, translation adjustment is necessary. The amount of translation distance adjustment can be determined based on the number of matching feature points or the rate of change of the overlapping area. Adjusting the translation distance based on the number of matching feature points: If the number of matching feature points is large, a smaller translation amount can be used for fine-tuning to align the images more accurately. For example, if the current image and the target image have 80 matching feature points, it means they are very close in position, and a small translation amount (such as a few pixels) can be selected for adjustment. Adjusting the translation distance based on the rate of change of the overlapping area: If the overlapping area is large, it means the two images have a high degree of overlap, and a smaller translation adjustment can be used to optimize alignment. Conversely, if the overlapping area is small, a larger translation amount is needed to achieve better alignment. For example, if the detected overlapping area only accounts for 10% of the target image, a large translation may be needed to ensure more areas overlap; while if the overlapping area accounts for more than 70%, only fine-tuning is required.

[0098] In an optional embodiment, a transformation matrix between adjacent local images is determined according to the stitching parameter adjustment method. The adjacent local images are then transformed and fused using the transformation matrix to generate a panoramic image of the optical device surface, including:

[0099] Construct an affine transformation matrix based on the rotation angle adjustment and translation distance adjustment;

[0100] The affine transformation matrix is ​​used to map the local images to be stitched onto the coordinate system of the reference image, so that the overlapping areas are aligned.

[0101] The seams of the aligned images are fused to eliminate the stitching gaps and generate a panoramic image of the optical device surface.

[0102] It's important to note that an affine transformation matrix is ​​a mathematical tool used to describe linear transformations between images, including operations such as rotation, translation, scaling, and shearing. A suitable affine transformation matrix can be constructed by adjusting the rotation angle and translation distance. Based on the feature point matching degree and edge alignment degree mentioned earlier, the required rotation angle is determined (e.g., a 15-degree clockwise rotation). The translation distance is determined based on the number of matching feature points or the rate of change of the overlapping area (e.g., a 20-pixel right translation). Once the affine transformation matrix is ​​constructed, it can be used to map the local image to be stitched onto the coordinate system of the reference image. The purpose of this step is to align the overlapping areas of the two images, thus preparing for subsequent stitching processing. For example, suppose there are two images, which are local images taken from different perspectives of an optical lens. By applying the affine transformation matrix, one image is rotated and translated onto the other image, so that identical parts (such as an edge of the lens) in the two images can overlap.

[0103] Once the two images are successfully aligned, the next step is to address the seam issue. Seam processing aims to eliminate noticeable gaps created during image stitching, resulting in a more natural and coherent panoramic image. Common techniques include gradient blending, feathering, and multi-band fusion. These methods enable smooth transitions in color and brightness within the seam area. For example, suppose the edges of a lens in the aligned image exhibit color or brightness differences between the two images. Gradient blending can gradually transition the colors and textures of the two images within the seam area, making the seam less noticeable and resulting in a more natural visual effect.

[0104] After seam processing, the final panoramic image will show the complete and detailed surface of the optical device. This image not only reflects the overall structure of the optical device, but also retains rich details, which is convenient for subsequent analysis and evaluation. For example, the final panoramic image shows the entire surface of the optical lens, with every detail clearly visible, such as the uniformity of the coating and surface defects. This is very important for quality control and performance evaluation.

[0105] In an optional embodiment, the defect detection region is divided into a different number of sub-regions based on the texture complexity within the defect detection region, including:

[0106] For defect detection regions with texture complexity less than a preset complexity threshold, the defect detection region is divided into sub-regions of a first detection quantity;

[0107] For defect detection regions with texture complexity greater than or equal to a preset complexity threshold, the defect detection region is divided into sub-regions with a second number of detections; wherein, the first number of detections is less than the second number of detections.

[0108] Among them, the area of ​​a single sub-region is negatively correlated with the texture complexity.

[0109] It should be noted that texture complexity refers to the richness of surface details in an image. Regions with low texture complexity typically have smooth surfaces and lack obvious details, while regions with high texture complexity have rich structural and detailed features. When inspecting the surface of optical devices, different processing strategies can be adopted according to different texture complexities to more effectively identify defects. For defect detection areas with low texture complexity, these areas usually have relatively simple surface structures. Therefore, for more detailed inspection, they can be divided into a first number of sub-regions. The first number of sub-regions is assumed to be set in areas with texture complexity less than a threshold, dividing them into 4 sub-regions. These sub-regions are relatively large because the surface texture is simple and the required detection accuracy is relatively low. For example, the edge area of ​​an optical lens has a relatively smooth surface with only a few small bubbles. When inspecting this area, it can be divided into 4 larger areas for inspection to ensure that potential defects are covered.

[0110] For defect detection areas with high texture complexity, the surface features are rich, and there may be more complex defects. Therefore, it is necessary to divide this area into sub-regions with a second detection number. The second detection number is set here to divide the area with texture complexity greater than or equal to a threshold into 8 sub-regions. These sub-regions are relatively small so that every minute defect can be detected more carefully. For example, in the central area of ​​an optical lens, there are many complex coating textures and fine scratches on the surface. In order to ensure that these small imperfections can be detected, this area is divided into 8 small sub-regions, thereby providing higher detection accuracy.

[0111] During the partitioning process, the area of ​​a single sub-region is negatively correlated with the texture complexity. That is, in regions with higher texture complexity, the area of ​​the sub-region is smaller, while in regions with lower texture complexity, the area of ​​the sub-region is larger. This design is to ensure that defect information can be captured more effectively in more complex regions. For example, if a region has high texture complexity, such as a complex coating structure, defects may be difficult to detect, so it needs to be divided into multiple smaller sub-regions (e.g., 8, each with a smaller area). Conversely, if a region has low texture complexity, such as a smooth surface, it can be divided into larger sub-regions (e.g., 4, each with a larger area) because the probability of defects occurring is lower.

[0112] In an optional embodiment, the defect identification method is grayscale abrupt change feature identification and grayscale gradual change feature identification;

[0113] For each sub-region, the defect identification method is determined based on the pixel grayscale variation characteristics within the sub-region, including:

[0114] If the pixel grayscale change feature is that there is a jump within a preset first distance, the defect identification method is to identify based on the grayscale abrupt change feature.

[0115] If the pixel grayscale change feature is that it gradually changes within a preset second distance, the defect identification method is based on the grayscale gradient feature; wherein, the preset first distance is smaller than the preset second distance.

[0116] It should be noted that pixel grayscale refers to the brightness value of each pixel in an image. In optical device inspection, grayscale variation characteristics can reveal surface defects such as scratches, bubbles, and uneven coatings. Different detection methods can be used for different types of grayscale variation characteristics. When a pixel grayscale value jumps significantly within a preset first distance, this usually indicates a relatively obvious defect. This jump may be caused by surface cracks, dents, or other types of protrusions. For example, suppose there is a noticeable scratch on the surface of an optical lens. This scratch causes the grayscale value of that area to suddenly decrease, forming a grayscale band that is significantly different from the surrounding area. When measuring the grayscale variation between pixels, if the grayscale value variation exceeds a certain threshold within a first distance (e.g., adjacent pixels or nearby areas), this scratch defect can be identified.

[0117] In another scenario, if the pixel grayscale changes gradually rather than abruptly within a preset second distance, this usually indicates a subtle defect on the surface, such as coating inhomogeneity or minor surface texture variations. For example, consider the coating portion of the same optical lens; due to uneven coating thickness, the grayscale value in a certain area gradually transitions from dark to light, creating a gradient effect. In this case, measuring the grayscale change in that area reveals a smooth transition, exhibiting a gradient characteristic at a greater distance (i.e., the second distance, which may be several pixels or more). Such a change suggests a coating issue, but not a significant defect. In both recognition methods described above, the preset first distance is smaller than the preset second distance. This means that the observation scale of the area to be detected differs. For obvious defects, a smaller distance can more sensitively capture abrupt changes; while for subtle changes, a larger distance is needed to identify gradient features.

[0118] In an optional embodiment, the defect type is determined based on the shape and size characteristics of the defect, combined with a preset defect feature library, including:

[0119] If the edge contour of the defect is more or less similar to the contour of the preset regular shape and the size feature is within the preset size range, the defect type is a repairable defect.

[0120] If the edge contour of the defect is less similar to the contour of the preset regular shape than the preset similarity or the size feature is not within the preset size range, the defect type is an unrepairable defect.

[0121] It should be noted that defect classification is mainly based on two criteria: the similarity between the edge contour and a preset regular shape, and whether the dimensional features are within a preset size range. When the similarity between the edge contour of a defect and a preset regular shape (e.g., circle, rectangle) is greater than or equal to a set preset similarity, and the dimensional features of the defect are within the specified size range, the defect is defined as a repairable defect. This indicates that the defect has a certain regularity and is relatively easy to handle during the repair process. For example, suppose the surface of an optical lens is being inspected; during the inspection, a small circular bubble with a diameter of 2 mm is found; the set regular shape is a circle, and the preset similarity is 0.8 (out of 1); through algorithm detection, it is found that the similarity between the edge contour of the bubble and a standard circle is 0.85, and its diameter is within the allowable repairable range (e.g., 1 to 3 mm); therefore, this bubble will be classified as a repairable defect and can be handled by recoating or other repair methods.

[0122] Conversely, when the similarity between the edge contour of a defect and a preset regular shape is less than the preset similarity, or when the size of the defect is outside the preset size range, the defect is defined as an unrepairable defect. This usually means that the defect is irregular in shape or too large in size, making it difficult to handle with simple repair methods. For example, continuing with the example of an optical lens, if an irregular scratch is found during the inspection process, with a length of 6 mm and a width of 1 mm; the similarity score of the edge contour of this scratch with any preset regular shape (such as a line or a circle) is 0.5, which is far lower than the preset similarity of 0.8; in addition, the size of the scratch exceeds the set repairable range, which is set to 1 to 3 mm; therefore, this scratch will be classified as an unrepairable defect, which may require replacing the entire lens or adopting other complex repair measures.

[0123] Example 2, please refer to Figure 2 This invention provides a technical solution: a panoramic image detection device based on image stitching, applicable to the aforementioned panoramic image detection method based on image stitching, comprising:

[0124] Image acquisition unit 1 is used to acquire multiple local detection images of the surface of optical devices; perform edge detection processing on each local detection image to obtain an edge image; extract feature points from the edge image to obtain a feature point set for each local detection image;

[0125] Feature point calculation unit 2 is used to calculate the number of feature points and the proportion of feature point types in the overlapping area of ​​the image based on the feature point set of each local detection image; and to determine the stitching priority of the local detection images based on the number of feature points and the proportion of feature point types, combined with the image sharpness difference value.

[0126] Feature description unit 3 is used to determine the image stitching order according to the stitching priority, perform feature description on the feature point set of adjacent local detection images, and obtain the feature description vector of each feature point.

[0127] Similarity calculation unit 4 is used to calculate the similarity between feature point feature description vectors in adjacent local detection images, and to determine feature point pairs with similarity greater than a preset similarity threshold as matching feature point pairs in overlapping regions, thereby obtaining the feature point matching degree in overlapping regions.

[0128] Image adjustment unit 5 is used to determine the stitching parameter adjustment method based on the feature point matching degree of the overlapping area; determine the transformation matrix between adjacent local images based on the stitching parameter adjustment method; and use the transformation matrix to transform and fuse the adjacent local images to generate a panoramic image of the surface of the optical device.

[0129] The embodiments of the present invention have been described in detail above with reference to the accompanying drawings. However, the present invention is not limited thereto. Various changes can be made within the scope of knowledge possessed by those skilled in the art without departing from the spirit of the present invention.

Claims

1. A panoramic image detection method based on image stitching, characterized in that, include: Acquire multiple local detection images of the surface of the optical device; perform edge detection processing on each local detection image to obtain an edge image; extract feature points from the edge images to obtain a feature point set for each local detection image; Based on the feature point sets of each local detected image, calculate the number of feature points and the proportion of feature point types within the overlapping regions of the images, including: Corner detection is performed on the edge image to obtain multiple corner points; Texture feature analysis is performed on the edge image to obtain multiple feature points with specific texture features; Combine multiple corner points and multiple feature points with specific texture features, and count the total number of feature points in the overlapping area and the proportion of different types of feature points. Based on the number and proportion of feature points and the difference in image sharpness, the stitching priority of the locally detected images is determined, including: If the number of feature points is greater than or equal to the preset feature point number threshold and the image sharpness difference value is less than the preset sharpness difference value threshold, the stitching priority is high priority; If the number of feature points is less than the preset feature point number threshold or the image clarity difference value is greater than or equal to the preset clarity difference value threshold, the stitching priority is low. The image stitching order is determined according to the stitching priority, and the feature point sets of adjacent local detection images are described to obtain the feature description vector of each feature point. Calculate the similarity between feature descriptor vectors of feature points in adjacent local detection images, and determine the feature point pairs with similarity greater than a preset similarity threshold as matching feature point pairs in the overlapping region to obtain the feature point matching degree of the overlapping region; Based on the matching degree of feature points in the overlapping area, the stitching parameter adjustment method is determined; based on the stitching parameter adjustment method, the transformation matrix between adjacent local images is determined; the transformation matrix is ​​used to transform and fuse the adjacent local images to generate a panoramic image of the surface of the optical device.

2. The panoramic image detection method based on image stitching according to claim 1, characterized in that, After transforming and fusing adjacent local images using a transformation matrix to generate a panoramic image of the optical device surface, the method further includes: Histogram equalization, sharpening, and noise reduction are performed on the panoramic image to obtain the enhanced panoramic image. Based on the distribution of texture features in the enhanced panoramic image, determine whether the texture distribution is uniform or non-uniform. For areas with uniform texture distribution, the entire area with uniform texture distribution is taken as the defect detection area. For areas with uneven texture distribution, the boundary of the defect detection area is determined according to the location of texture abrupt change. Based on the texture complexity within the defect detection area, the defect detection area is divided into a different number of sub-regions; for each sub-region, the defect recognition method is determined based on the pixel grayscale variation characteristics within the sub-region. Defects are identified in sub-regions according to the defect identification method. When a defect is identified, its shape and size features are extracted. Based on the shape and size features of the defect and a pre-set defect feature library, the defect type is determined. Display a panoramic image containing the detection results. The detection results include the defect type and the location information of the defect in the panoramic image. During the display of the panoramic image containing the detection results, if no operation to adjust the annotation style of the detection results is received, the detection results are annotated in the panoramic image using the default annotation style. If an operation to adjust the annotation style of the detection results is received, the annotation style of the detection results in the panoramic image is switched to another annotation style other than the current annotation style among a variety of preset annotation styles.

3. The panoramic image detection method based on image stitching according to claim 2, characterized in that, The splicing parameters can be adjusted by rotating the angle and shifting the distance. Based on the matching degree of feature points in the overlapping area, the method for adjusting the splicing parameters is determined, including: If the feature point matching degree is less than the preset feature point matching degree threshold, the rotation angle adjustment method is to determine the rotation angle adjustment amount based on the feature point distribution density. If the feature point matching degree is greater than or equal to the preset feature point matching degree threshold, the rotation angle adjustment method is to determine the rotation angle adjustment amount based on the edge alignment degree. Specifically, the rotation angle adjustment is determined based on the density of feature point distribution or edge alignment, and the translation distance adjustment is determined based on the number of matched feature points or the rate of change of overlapping area.

4. The panoramic image detection method based on image stitching according to claim 3, characterized in that, The transformation matrix between adjacent local images is determined based on the stitching parameter adjustment method. The adjacent local images are then transformed and fused using the transformation matrix to generate a panoramic image of the optical device surface, including: Construct an affine transformation matrix based on the rotation angle adjustment and translation distance adjustment; The affine transformation matrix is ​​used to map the local images to be stitched onto the coordinate system of the reference image, so that the overlapping areas are aligned. The seams of the aligned images are fused to eliminate the stitching gaps and generate a panoramic image of the optical device surface.

5. The panoramic image detection method based on image stitching according to claim 4, characterized in that, Based on the texture complexity within the defect detection region, the defect detection region is divided into a different number of sub-regions, including: For defect detection regions with texture complexity less than a preset complexity threshold, the defect detection region is divided into sub-regions of a first detection quantity; For defect detection regions with texture complexity greater than or equal to a preset complexity threshold, the defect detection region is divided into sub-regions with a second number of detections; wherein, the first number of detections is less than the second number of detections. Among them, the area of ​​a single sub-region is negatively correlated with the texture complexity.

6. The panoramic image detection method based on image stitching according to claim 5, characterized in that, The defect identification methods are gray-scale abrupt change feature identification and gray-scale gradual change feature identification; For each sub-region, the defect identification method is determined based on the pixel grayscale variation characteristics within the sub-region, including: If the pixel grayscale change feature is that there is a jump within a preset first distance, the defect identification method is to identify based on the grayscale abrupt change feature. If the pixel grayscale change feature is that it gradually changes within a preset second distance, the defect identification method is based on the grayscale gradient feature; wherein, the preset first distance is smaller than the preset second distance.

7. The panoramic image detection method based on image stitching according to claim 6, characterized in that, Based on the shape and size characteristics of the defect, and in conjunction with a pre-defined defect feature library, the defect type is determined, including: If the edge contour of the defect is more or less similar to the contour of the preset regular shape and the size feature is within the preset size range, the defect type is a repairable defect. If the edge contour of the defect is less similar to the contour of the preset regular shape than the preset similarity or the size feature is not within the preset size range, the defect type is an unrepairable defect.

8. A panoramic image detection device based on image stitching, applicable to the panoramic image detection method based on image stitching as described in any one of claims 1-7, characterized in that, include: The image acquisition unit is used to acquire multiple local detection images of the surface of the optical device; perform edge detection processing on each local detection image to obtain an edge image; and extract feature points from the edge image to obtain a feature point set for each local detection image. The feature point calculation unit is used to calculate the number of feature points and the proportion of feature point types in the overlapping area of ​​each local detection image based on the feature point set of each local detection image; and to determine the stitching priority of the local detection images based on the number of feature points and the proportion of feature point types, combined with the image sharpness difference value. The feature description unit is used to determine the image stitching order according to the stitching priority, perform feature description on the feature point set of adjacent local detection images, and obtain the feature description vector of each feature point. The similarity calculation unit is used to calculate the similarity between feature point feature description vectors in adjacent local detection images. Feature point pairs with similarity greater than a preset similarity threshold are identified as matching feature point pairs in the overlapping region, and the feature point matching degree in the overlapping region is obtained. The image adjustment unit is used to determine the stitching parameter adjustment method based on the feature point matching degree of the overlapping area; determine the transformation matrix between adjacent local images based on the stitching parameter adjustment method; and use the transformation matrix to transform and fuse the adjacent local images to generate a panoramic image of the surface of the optical device.

Images