Camera distortion removing anomaly detection method and device and computer equipment

Abstract

The invention relates to a camera distortion removing anomaly detection method and device and computer equipment. The method comprises the following steps: acquiring an internal parameter containing a camera and a target image; performing lossless distortion correction on the target image according to the internal parameters to obtain a first full distortion-removed image; extracting a target contour in the first full distorted image; determining distribution characteristics and geometric characteristics of the target contour in the first full distortion-removed image; and verifying the distribution characteristics and / or the geometric characteristics, and determining a distortion removing abnormal result of the first full distortion removing image. By adopting the method, the accuracy of distortion removal anomaly detection of the camera can be improved.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to a method, apparatus and computer device for detecting camera distortion anomalies. Background Technology

[0002] Camera distortion correction and anomaly detection is a crucial step in computer vision and image processing. Traditional camera intrinsic parameter calibration methods typically involve a one-time calibration in a laboratory environment, with the obtained distortion parameters being used consistently. However, when the calibration process is not standardized (e.g., improper placement of the calibration plate, insufficient number of calibration images) or an inappropriate calibration algorithm is chosen, the obtained distortion parameters themselves contain errors, which the system cannot identify. Even worse, these aberration parameters are repeatedly used, causing all subsequent image processing and analysis to be based on incorrect geometry.

[0003] However, anomaly detection in camera distortion correction is a core component in ensuring the accuracy, safety, and reliability of vision systems. Therefore, a method is needed to improve the accuracy of anomaly detection in camera distortion correction. Summary of the Invention

[0004] Therefore, it is necessary to provide a camera distortion correction anomaly detection method, apparatus, computer equipment, computer-readable storage medium, and computer program product that can improve the accuracy of camera distortion correction anomaly detection in response to the above-mentioned technical problems.

[0005] Firstly, this application provides a method for detecting camera distortion anomalies, including:

[0006] Acquire internal parameters from the camera and the target image;

[0007] Based on the internal parameters, the target image is subjected to non-destructive distortion correction to obtain a first full-frame distortion-free image;

[0008] Extract the target contour from the first full-frame distortion-free image;

[0009] Determine the distribution and geometric features of the target contour in the first full-frame distortion-free image;

[0010] The distribution features and / or geometric features are verified to determine the distortion anomaly result of the first full-frame distortion map.

[0011] In one embodiment, extracting the target contour from the first full-frame distortion-free image includes:

[0012] The first full-frame distortion-free image is converted to obtain a single-channel target image;

[0013] The single-channel target image is subjected to threshold segmentation to obtain a binary image containing only foreground and background pixels;

[0014] Contour extraction is performed on the binary image to obtain contour information of all extracted candidate contours, and the target contour is selected based on the contour information.

[0015] In one embodiment, before determining the distribution and geometric features of the target contour in the first full-frame distortion map, the method further includes:

[0016] Create a mask image with the same resolution as the first full-frame distortion-free image;

[0017] Based on the position of the target contour in the first full-frame distortion correction image, the target contour in the mask image is determined, and the pixel value of each pixel in the pixel filling region corresponding to the target contour in the mask image is set to the first pixel value.

[0018] Determine the average grayscale value of the pixel-filled region in the first full-frame distortion-free image;

[0019] If the difference between the first pixel value and the mean gray value is within a preset range, then the target contour is valid.

[0020] In one embodiment, determining the distribution and geometric features of the target contour in the first full-frame distortion-free image includes:

[0021] Determine the filling region formed by the target contour and the image edge of the first full-frame distortion map, determine the first number of minimum bounding polygons formed by the filling region, and obtain the distribution characteristics of the target contour in the first full-frame distortion map;

[0022] Determine the position of the minimum bounding polygon in the first full-frame distortion image, and determine the second number of the minimum bounding polygons corresponding to each image edge in the first full-frame distortion image based on the position;

[0023] Based on the distribution characteristics of the pixel filling region, determine the geometric difference between the minimum bounding polygons corresponding to the image edges in the preset direction corresponding to the distribution characteristics;

[0024] The second quantity and / or the geometric difference are used to determine the geometric features of the target contour in the first full-frame distortion map.

[0025] In one embodiment, verifying the distribution features and / or the geometric features to determine the distortion anomaly result of the first full-frame distortion-corrected image includes:

[0026] If at least one of the following conditions exists: the first quantity is not a first preset quantity value, the second quantity is not a second preset quantity value, and the geometric difference is less than a preset deviation, then the distortion anomaly result of the first full-frame distortion-corrected image is determined to have a distortion anomaly.

[0027] In one embodiment, the method further includes:

[0028] Based on the preset geometric reference information and the resolution of the target image, at least one preset image including the preset geometric reference information is generated;

[0029] According to the internal parameters, each preset image is subjected to lossless distortion correction to obtain the corresponding second full-frame distortion-free image.

[0030] The geometric information in each of the second full-frame distortion-free images is detected to obtain the detection results. Based on the detection results, the abnormal results of the distortion coefficients in the internal parameters are determined.

[0031] In one embodiment, the preset geometric reference information includes preset straight line information. The step of detecting the geometric information in each of the second full-frame distortion-free images to obtain detection results, and determining abnormal results of the distortion coefficients in the internal parameters based on each detection result, includes:

[0032] For each of the second full-frame distortion maps, traverse each pixel in the second full-frame distortion map, extract the non-zero pixel positions in the second full-frame distortion map based on the pixel values ​​of the traversed pixels, and obtain the first set of non-zero pixel points used to represent the straight line;

[0033] The number of fitted lines is obtained by fitting the non-zero pixel positions of each non-zero pixel in the first set of non-zero pixels, and the straightness of the fitted lines is determined.

[0034] If the number of straight lines and / or the straightness do not meet their respective preset values, then the abnormal result of the distortion coefficient in the internal parameters is determined to be an abnormality of the distortion coefficient.

[0035] In one embodiment, the preset geometric reference information includes preset concentric circle information. The step of detecting the geometric information in each of the second full-frame distortion-free images to obtain detection results, and determining abnormal results of the distortion coefficients in the internal parameters based on each detection result, includes:

[0036] Traverse each pixel in the second full-frame distortion-free image, and extract the non-zero pixel positions in the second full-frame distortion-free image based on the pixel values ​​of the traversed pixels to obtain the second set of non-zero pixels used to represent concentric circles;

[0037] Based on the second set of non-zero pixels, determine the closure data and roundness of the concentric circles in the first full-frame distortion correction image;

[0038] If the closed data and / or the roundness of the concentric circles do not meet their respective preset values, then the abnormal result of the distortion coefficient in the internal parameters is determined to be an abnormal distortion coefficient.

[0039] Secondly, this application also provides a camera distortion correction anomaly detection device, comprising:

[0040] The data acquisition module is used to acquire internal parameters of the camera and target images;

[0041] An image generation module is used to generate a preset image including the preset geometric reference information based on preset geometric reference information and the resolution of the target image;

[0042] The distortion correction module is used to perform non-destructive distortion correction on the target image according to the internal parameters to obtain a first full-frame distortion-free image;

[0043] The contour extraction module is used to extract the target contour from the first full-frame distortion-free image;

[0044] The image detection module is used to determine the distribution features and geometric features of the target contour in the first full-frame distortion-free image; to verify the distribution features and / or the geometric features, and to determine the distortion-free anomaly result of the first full-frame distortion-free image.

[0045] Thirdly, this application also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to perform the following steps:

[0046] Acquire internal parameters from the camera and the target image;

[0047] Based on the internal parameters, the target image is subjected to non-destructive distortion correction to obtain a first full-frame distortion-free image;

[0048] Extract the target contour from the first full-frame distortion-free image;

[0049] Determine the distribution and geometric features of the target contour in the first full-frame distortion-free image;

[0050] The distribution features and / or geometric features are verified to determine the distortion anomaly result of the first full-frame distortion map.

[0051] Fourthly, this application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, performs the following steps:

[0052] Acquire internal parameters from the camera and the target image;

[0053] Based on the internal parameters, the target image is subjected to non-destructive distortion correction to obtain a first full-frame distortion-free image;

[0054] Extract the target contour from the first full-frame distortion-free image;

[0055] Determine the distribution and geometric features of the target contour in the first full-frame distortion-free image;

[0056] The distribution features and / or geometric features are verified to determine the distortion anomaly result of the first full-frame distortion map.

[0057] Fifthly, this application also provides a computer program product, including a computer program that, when executed by a processor, performs the following steps:

[0058] Acquire internal parameters from the camera and the target image;

[0059] Based on the internal parameters, the target image is subjected to non-destructive distortion correction to obtain a first full-frame distortion-free image;

[0060] Extract the target contour from the first full-frame distortion-free image;

[0061] Determine the distribution and geometric features of the target contour in the first full-frame distortion-free image;

[0062] The distribution features and / or geometric features are verified to determine the distortion anomaly result of the first full-frame distortion map.

[0063] The aforementioned camera distortion correction anomaly detection method, apparatus, computer equipment, computer-readable storage medium, and computer program product directly acquire the target image to be processed, along with the camera's internal parameters, without relying on a laboratory environment or manual calibration plate placement to determine the target image. Based on the internal parameters, the target image undergoes non-destructive distortion correction to obtain a first full-frame distortion correction image that maintains the original image size and does not lose pixels. The target contour is extracted from the first full-frame distortion correction image. The distribution and geometric features of the target contour in the first full-frame distortion correction image are determined, and the distribution and / or geometric features are verified to determine the distortion correction anomaly result of the first full-frame distortion correction image. In other words, by extracting the target contour and considering its distribution and geometric features throughout the entire image, anomalies present in the full-frame distortion correction image can be effectively identified, avoiding the limitation of only identifying anomalies in localized areas and improving the accuracy of camera distortion correction anomaly detection. Attached Figure Description

[0064] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the drawings used in the description of the embodiments of this application or related technologies will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0065] Figure 1 This is a flowchart illustrating a camera distortion correction anomaly detection method in one embodiment;

[0066] Figure 2 This is a schematic diagram of a distorted target image in one embodiment;

[0067] Figure 3 This is a schematic diagram of the first full-frame distortion-free image in one embodiment;

[0068] Figure 4 This is a schematic diagram of a full-frame distortion-free image in one embodiment;

[0069] Figure 5 This is a schematic diagram of a full-frame distortion correction anomaly in one embodiment;

[0070] Figure 6 This is a flowchart illustrating a method for extracting the target contour in one embodiment;

[0071] Figure 7 This is a binary image after performing an opening operation in one embodiment;

[0072] Figure 8 This is a schematic outline of the first full-frame distortion-free image in one embodiment;

[0073] Figure 9 This is a schematic diagram of the target contour obtained after filtering and the minimum bounding rectangle corresponding to the target contour in one embodiment;

[0074] Figure 10 This is a schematic diagram of a mask image in one embodiment;

[0075] Figure 11 This is a flowchart illustrating a method for determining the distribution and geometric features of a target contour in one embodiment.

[0076] Figure 12 This is a flowchart of a camera distortion correction anomaly detection method in another embodiment;

[0077] Figure 13 This is a flowchart of a method for camera distortion correction anomaly detection in yet another embodiment;

[0078] Figure 14 This is a schematic diagram of a straight line in a distortion graph in one embodiment;

[0079] Figure 15 This is a flowchart of a camera distortion correction and anomaly detection method based on preset line information in one embodiment;

[0080] Figure 16 This is a flowchart of a method for detecting camera distortion anomalies in another embodiment;

[0081] Figure 17 This is a schematic diagram illustrating a full-image anomaly in one embodiment;

[0082] Figure 18 This is a schematic diagram of edge anomalies in one embodiment;

[0083] Figure 19 This is a schematic diagram of a distortion loop in one embodiment;

[0084] Figure 20 This is a structural block diagram of a camera distortion correction anomaly detection device in one embodiment;

[0085] Figure 21 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0086] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0087] Most existing distortion anomaly detection methods rely on specific calibration scenarios or auxiliary equipment, making them unsuitable for rapid and convenient detection in real-world applications, and they cannot guarantee detection accuracy. Distortion anomalies manifest as complex geometric shapes, making them difficult to describe uniformly and quantitatively using simple rules. Moreover, in engineering applications, it is challenging to ensure that the algorithm has stable detection capabilities for any scene while simultaneously meeting the production requirements of real-time, high-efficiency operation on low-computing-power platforms. Therefore, a method is needed that can ensure the accuracy of camera distortion anomaly detection.

[0088] In one exemplary embodiment, such as Figure 1 As shown, a method for detecting camera distortion anomalies is provided. This example illustrates the method's application to a terminal; however, it can also be applied to a server or a system including both a terminal and a server, and implemented through interaction between the terminal and the server. In this embodiment, the method includes the following steps:

[0089] Step 102: Obtain internal parameters from the camera and the target image.

[0090] The intrinsic parameters include an intrinsic parameter matrix and distortion coefficients, which can include radial distortion coefficients and tangential distortion coefficients. The target image can be a laboratory image used for intrinsic parameter calibration, or an image of any scene actually captured by the camera; the method of acquiring the target image is not limited here. It should be noted that the target image in this embodiment does not rely on a laboratory environment or manual placement of the calibration plate; the target image is an arbitrary image with distortion captured by the camera.

[0091] For example, internal parameters are read from a storage device or camera interface, and a target image is acquired from an image acquisition device.

[0092] Step 104: Perform non-destructive distortion correction on the target image based on internal parameters to obtain the first full-frame distortion-free image.

[0093] The first full-frame distortion-corrected image can be a complete image generated after lossless correction of the target image based on intrinsic parameters, and can be used to preserve all the information of the original image. For example, lossless distortion correction can be understood as correcting the distortion of the target image while preserving its original size and without losing any pixels. Lossless distortion correction can be generated by performing pixel mapping transformation on the target image using intrinsic parameters. For example, a distortion map is generated according to OpenCV's initUndistortRectifyMap function, and then the remap function remaps the target image according to the generated distortion map to obtain the first full-frame distortion-corrected image containing all pixel information of the target image and with known filler grayscale values. It is understood that during the distortion correction process, some pixels in certain areas do not have corresponding data in the original image (e.g., areas where the edges of the corrected image are stretched or expanded), and these blank areas need to be filled with additional pixels.

[0094] The first full-frame distortion correction image includes pixel regions and pixel-filled regions. The pixel regions in the first full-frame distortion correction image can be regions composed of effective pixels in the target image, and the pixel-filled regions can be the background of the regions composed of effective pixels, extending inward from the outermost boundary of the image until they encounter effective pixel regions.

[0095] For example, such as Figure 2 As shown, in an exemplary embodiment, a distorted target image is generated. A distortion map is generated for the target image using OpenCV's `initUndistortRectifyMap` function. Then, the `remap` function remaps the target image based on the generated distortion map, resulting in a first full-frame distorted image containing all pixel information of the target image and with known fill grayscale values. This first full-frame distorted image contains all pixel information of the target image and fill pixels. The fill pixel value can be a fill grayscale value of 128, such as... Figure 3 The first full-frame distortion-free image shown is a gray area that is a filled pixel area, and the fill gray value of the filled pixel area can be 128.

[0096] It should be noted that anomaly detection in the first full-frame distortion-corrected image can be achieved by utilizing the imaging characteristics of the distortion model, preprocessing and extracting features from the target image captured by the camera, and analyzing the geometric properties of the extracted features to determine whether anomalies exist in the corresponding first full-frame distortion-corrected image. In a normal full-frame distortion-corrected image, the effective pixels from the original image constitute a single, continuous connected region. The filled region is the background of this connected region, extending inward from the outermost boundary of the image until it encounters the effective pixel region, such as... Figure 4As shown in the gray area, the distortion correction effect of some abnormal intrinsic parameters diverges sharply at the image edges, resulting in an abnormally small focal length across the entire image, accompanied by holes in the center. Figure 5 As shown. Based on the mathematical properties of distortion correction mapping, the pixel filling regions of the full-frame distortion correction map have attributes such as being located around the image, symmetrically distributed within a certain range, having clear and sharp boundaries between the filling regions and the effective image, and having a certain number of continuous line segments.

[0097] Step 106: Extract the target contour from the first full-frame distortion-free image.

[0098] In this process, the target image undergoes lossless distortion correction, which alters its geometry. For example, the target image may be stretched or compressed, its edges may be stretched or compressed, and / or local areas within the target image may also be stretched or compressed, thus forming the contour in the first full-frame distortion correction image. The target contour can be a set of target edges in the first full-frame distortion correction image, which can be used to analyze the distribution and geometric characteristics of the image structure. The contour type can be one or more of closed contours, open contours, and polygonal contours. The target contour can be extracted from the first full-frame distortion correction image using an edge detection algorithm. Furthermore, the target contour can also be the contour formed by pixel-filled regions in the first full-frame distortion correction image; this contour is the outer contour, which can be understood as the outermost contour located within the same connected region and not surrounded by any other contour. The method for determining the target contour in the first full-frame distortion correction image can be determined by a trained image recognition model.

[0099] Step 108: Determine the distribution and geometric features of the target contour in the first full-frame distortion map.

[0100] The distribution characteristics of the target contour can be understood as the first number of the minimum bounding polygons of the pixel-filled region formed by the target contour and the image edges of the first full-frame distortion-corrected image. The first number of minimum bounding polygons can be achieved using existing methods, which will not be elaborated upon here. The minimum bounding polygons can be, but are not limited to, the minimum bounding rectangle.

[0101] The geometric features of the target contour can be the second number of the minimum bounding polygons corresponding to the image edges of the first full-frame distortion-free image, and / or the geometric difference of the minimum bounding polygons in a preset direction, determined based on the distribution characteristics of the pixel filling region. The determination of geometric features can be achieved using existing methods, which will not be elaborated upon here.

[0102] It should be noted that detecting whether there are distortion anomalies in the first full-frame distortion map by determining the distribution and geometric features of the target contour is to effectively monitor image-level anomalies such as distortion rings, holes, and edge distortion by considering the overall geometric characteristics of the entire first full-frame distortion map.

[0103] Step 110: Verify the distribution characteristics and / or geometric characteristics to determine the distortion anomaly results of the first full-frame distortion map.

[0104] The verification of distribution features and / or geometric features can be performed by separately verifying whether each distribution feature and / or geometric feature meets its corresponding preset conditions. These preset conditions can be determined based on the imaging characteristics of a normal full-frame distortion-corrected image. If at least one of the distribution features and geometric features fails to meet its corresponding preset condition during the verification process, it indicates that the first full-frame distortion-corrected image is abnormal.

[0105] Furthermore, the verification of distribution features and geometric features can be performed in a preset order. For example, the distribution features can be verified first to see if they meet preset distribution features. If they do, the geometric features can then be further verified to see if they meet preset geometric features. It is understandable that determining the distribution and geometric features of the target contour in the first full-frame distortion correction image means assessing whether the pixel-filled areas in the first full-frame distortion correction image meet the imaging characteristics of a normal full-frame distortion correction image to evaluate the distortion correction effect.

[0106] For example, if at least one of the following conditions exists: the first quantity is not a first preset quantity value, the second quantity is not a second preset quantity value, and the geometric difference is less than a preset deviation, then the distortion anomaly result of the first full-frame distortion map is determined to have a distortion anomaly.

[0107] In the aforementioned camera distortion correction anomaly detection method, the target image to be processed and its internal parameters, including those of the camera, are directly acquired, eliminating the need for a laboratory environment and manual calibration plate placement to determine the target image. The target image undergoes non-destructive distortion correction based on the internal parameters, resulting in a first full-frame distortion correction image that maintains the original image size and does not lose pixels. The target contour is extracted from the first full-frame distortion correction image. The distribution and geometric features of the target contour in the first full-frame distortion correction image are determined, and these features are verified to determine the distortion correction anomaly result of the first full-frame distortion correction image. In other words, by extracting the target contour and considering its distribution and geometric features throughout the entire image, anomalies present in the full-frame distortion correction image can be effectively identified, avoiding the limitation of only identifying anomalies in localized areas and improving the accuracy of camera distortion correction anomaly detection.

[0108] In one exemplary embodiment, such as Figure 6As shown, a method for extracting target contours is provided, including the following steps:

[0109] Step 602: Convert the first full-frame distortion-free image to obtain a single-channel target image.

[0110] The conversion of the first full-frame distortion-free image into a single-channel target image can be achieved using existing methods, such as converting the first full-frame distortion-free image into a single-channel target image using OpenCV's cvtColor function.

[0111] Step 604: Threshold segmentation is performed on the single-channel target image to obtain a binary image containing only foreground and background pixels.

[0112] Thresholding segmentation of a single-channel target image can be achieved using existing methods, which will not be elaborated here. For example, according to OpenCV's adaptiveThreshold function, thresholding segmentation of a single-channel target image can be performed to obtain a binary image containing only foreground and background pixels.

[0113] Furthermore, in an exemplary embodiment, all pixel grayscale values ​​of the target image can be directly set to 255 (pure white) before performing lossless distortion correction, and the fill grayscale value can be set to 0 (pure black) to directly obtain a binary image, and then step 606 can be executed.

[0114] Step 606: Extract contours from the binary image to obtain contour information of all extracted candidate contours, and then select the target contour based on the contour information.

[0115] Optionally, before contour extraction from the binary image, the binary image can be optimized to remove small objects and noise, and smooth the contours. The optimization can be, but is not limited to, opening operations. Opening is a morphological operation in image processing, primarily used to optimize binary images by eliminating small objects, suppressing noise, and smoothing object contours. It can be a combination of basic operations based on dilation and erosion. Figure 7 The image shown is a binary image after the opening operation is performed in an exemplary embodiment.

[0116] Furthermore, contour extraction can be performed directly on the optimized binary image, or it can be performed on the unoptimized binary image. The contour extraction method can be, but is not limited to, existing methods. For example, taking contour extraction from an optimized binary image as an example, the OpenCV `findContours` function can be used to search for contours in the optimized binary image, obtaining all contour information within the image. Figure 8The outline diagram shown uses each color (e.g., green, orange, pink, purple) to represent a outline. The outline information for each candidate outline is obtained, and the extracted outline information is filtered to select the target outline. The outline information includes attributes such as whether the outline is an outer outline and whether it includes sub-outlines.

[0117] Optionally, the extracted contour information can be filtered to select target contours. This can be done by determining at least one of the following for each candidate contour: whether the candidate contour is an outer contour, the contour area, the position of the smallest bounding polygon determined by the candidate contour relative to the image boundary, and whether it includes sub-contours.

[0118] For example, in a normal full-frame distortion-free image, the contour formed by the filled region is the outer contour, meaning the outer contour is the contour located at the outermost layer of the same connected component and not surrounded by any other contour. For each candidate contour, it can be determined whether it is an outer contour based on its corresponding contour information. If it is not an outer contour, it is skipped. If it is an outer contour, the contour area of ​​the candidate contour can be calculated, and it can be determined whether the contour area is less than a preset area. If so, it is skipped. If not, it can be further determined whether the smallest bounding polygon determined by the candidate contour is located at the image boundary. If not, it is skipped. If so, it can be further determined whether the candidate contour includes sub-contours. If it does, it is skipped. If it does not, the candidate contour can be determined as the target contour. Figure 9 As shown in the illustration, this represents the target contour obtained after filtering and its corresponding minimum bounding rectangle in an exemplary embodiment. White represents the target contour, and green represents the minimum bounding rectangle. The preset area can be set proportionally based on the resolution of the target image, for example, 0.0001. The contour area can be determined using OpenCV's `contourArea` function, and the minimum bounding polygon can be determined using OpenCV's `boundingRect` function. The specific implementation methods of these two functions can be achieved using existing methods and will not be elaborated upon here.

[0119] In the above embodiments, a binary image is obtained by preprocessing and thresholding the first full-frame distortion-corrected image. Furthermore, the target contour is determined by contour extraction and filtering of the binary image, thus ensuring the reliability of the target contour and the accuracy of the camera distortion anomaly detection.

[0120] Building upon the above embodiments, to further ensure the accuracy of camera distortion anomaly detection, in an exemplary embodiment, before determining the distribution features and geometric features of the target contour in the first full-frame distortion map, the method further includes:

[0121] Create a mask image with the same resolution as the first full-frame distortion-free image; determine the target contour in the mask image based on the position of the target contour in the first full-frame distortion-free image, and set the pixel values ​​of each pixel in the pixel-filled region corresponding to the target contour in the mask image to the first pixel value; determine the gray-scale mean of the pixel-filled region in the first full-frame distortion-free image; if the difference between the first pixel value and the gray-scale mean is within a preset range, the target contour is valid.

[0122] The method for generating the mask image can be achieved using existing methods, and will not be elaborated here.

[0123] For example, a mask image with the same resolution as the first full-frame distortion-free image is created, and the pixel values ​​of each pixel point in all pixel regions of the mask image are filled with a second pixel value (e.g., filled with zero). Based on the position of the target contour in the first full-frame distortion-free image, the position of the target contour in the mask image is determined, and the target contour is drawn at the corresponding position in the mask image. The pixel values ​​of each pixel point inside the target contour are filled with a first pixel value, which is a non-zero pixel value. For example, the pixel values ​​of each pixel point inside the target contour can be filled with a grayscale value of 255, resulting in the white area around the image in the mask image shown in Figure 10.

[0124] With the first pixel value at 255 and the second pixel value at 0, only the target contour-covered area in the mask image has a non-zero pixel value. Using OpenCV's `mean` function, the mean grayscale value of the target contour area in the first full-frame distortion correction image (i.e., the mean grayscale value of the pixel-filled area) can be calculated. Understandably, in a normal full-frame distortion correction image, the mean pixel value of the target contour area should theoretically also be 255. There is a clear boundary between the normal pixel-filled area and the effective image. Therefore, the validity of the target contour can be determined by checking if the deviation between the first pixel value and the mean grayscale value is less than a preset deviation (e.g., the preset deviation can be, but is not limited to, 0.05), i.e., whether it is within the preset value range. If it is less than the preset deviation, the extracted target contour is valid; if it is not less than the preset deviation, it indicates that pixels of the original target image exist within the target contour. This also indicates that the first full-frame distortion correction image has a distortion correction anomaly. Understandably, if there are holes within the target contour, the deviation between the first pixel value and the mean grayscale value will be greater than the preset deviation.

[0125] In the above embodiments, by verifying the grayscale values ​​of the pixel-filled regions corresponding to the target contour in the mask image, the validity of the target contour and the accuracy of the camera's distortion correction anomaly detection are ensured.

[0126] In one exemplary embodiment, such as Figure 11As shown, a method for determining the distribution features and geometric features of a target contour is provided, including the following steps:

[0127] Step 1102: Determine the filling region formed by the target contour and the image edge of the first full-frame distortion map, determine the first number of minimum bounding polygons formed by the filling region, and obtain the distribution characteristics of the target contour in the first full-frame distortion map.

[0128] Understandably, in a normal full-frame distortion correction image, the filled region is distributed around the perimeter of the image. Taking the minimum bounding polygon as the minimum bounding rectangle as an example, the first number of the minimum bounding rectangles of the target contour can be determined based on the aforementioned method for determining the minimum bounding rectangle. This first number can then be used to define the distribution characteristics of the target contour in the first full-frame distortion correction image. Image edges can include four edges: mutually opposing left and right edges, and mutually opposing top and bottom edges.

[0129] Step 1104: Determine the position of the minimum bounding polygon in the first full-frame distortion image, and determine the second number of minimum bounding polygons corresponding to each image edge in the first full-frame distortion image based on the position.

[0130] For example, the minimum bounding rectangle of the target contour is classified and labeled according to the position of its boundary in the image (e.g., top edge, bottom edge, left edge, right edge). If the minimum bounding rectangle has two sides located at the image edge, it is labeled twice. In a normal full-frame distortion image, each edge of the full-frame distortion image has one and only one filled region. The position of the minimum bounding polygon in the first full-frame distortion image is determined by traversing each edge of the first full-frame distortion image and determining a second number of minimum bounding rectangles located at that edge.

[0131] Step 1106: Based on the distribution characteristics of the pixel filling region, determine the geometric difference between the smallest bounding polygons corresponding to the image edges in the preset direction corresponding to the distribution characteristics.

[0132] It is understandable that the distribution characteristics of pixel filling regions can be symmetrical. For example, in a normal full-frame distortion-corrected image, the pixel filling regions are distributed symmetrically both vertically and horizontally within a certain range. Therefore, the preset directions corresponding to these distribution characteristics include a first direction and a second direction. The first and second directions are perpendicular; the first direction can be vertical, and the second direction can be horizontal.

[0133] For example, based on the distribution characteristics of the pixel filling region, the height difference between the minimum bounding rectangles of the first full-frame distorted image located at the upper and lower edges in the first direction is determined, and the width difference between the minimum bounding rectangles of the first full-frame distorted image located at the left and right edges in the second direction is determined.

[0134] Step 1108: Use the second quantity and / or geometric difference to determine the geometric features of the target contour in the first full-frame distortion map.

[0135] Furthermore, based on this, the distribution characteristics and / or geometric characteristics are verified to determine the distortion anomaly result of the first full-frame distortion map, including: if at least one of the following conditions exists: the first quantity is not a first preset quantity value, the second quantity is not a second preset quantity value, and the geometric difference is less than a preset deviation, then the distortion anomaly result of the first full-frame distortion map is determined to have a distortion anomaly.

[0136] For example, it is determined whether the first number of minimum bounding rectangles of the target contour is a first preset number, such as 4. If not, the first full-frame distortion correction image is determined to be abnormal. If it is 4, the minimum bounding rectangles of the target contour can be classified and marked according to the position of their boundaries in the image (top edge, bottom edge, left edge, right edge). If two sides of the minimum bounding rectangle are located at the image edge, it is marked twice. Counting is performed, traversing each edge of the first full-frame distortion correction image, and determining whether the number of minimum bounding rectangles located at that edge is a second preset number, for example, 1. If not, the full-frame distortion correction image is determined to be abnormal. If so, the height difference between the minimum bounding rectangles located at the top and bottom edges of the first full-frame distortion correction image is calculated to see if it is less than a preset deviation (e.g., 0.05). If not, the full-frame distortion correction image is determined to be abnormal. If satisfied, calculate whether the width difference between the smallest bounding rectangles at the left and right edges of the first full-frame distortion map is less than the preset deviation (e.g., 0.05). If not satisfied, determine that the full-frame distortion map has an anomaly.

[0137] In this embodiment, by determining the number of minimum bounding polygons of the pixel-filled region formed by the target contour, the positional relationship between the minimum bounding polygons of the pixel-filled region and the image edge, and the geometric difference of the minimum bounding polygons of the pixel-filled region, the geometric characteristics of the target contour in the whole image are considered. This can effectively detect global distortion anomalies such as distortion rings, holes, and edge distortion, and has a more comprehensive anomaly detection capability, thereby improving the accuracy of camera distortion anomaly detection.

[0138] In one exemplary embodiment, a flowchart of the above-described camera distortion correction anomaly detection method is provided, as follows: Figure 12 As shown, the specific steps include:

[0139] Step 1: Obtain the camera's internal parameters and the target image.

[0140] The internal parameters and target image can be obtained in the manner described above, and are not limited here. It should be noted that the above-described camera distortion anomaly detection method can perform distortion anomaly detection on the full-frame distortion image. Therefore, the target image obtained is the full-frame distortion image, and there is no need to perform lossless distortion correction processing on the target image.

[0141] Step 2: Perform distortion correction on the target image.

[0142] The distortion map is generated using OpenCV's `initUndistortRectifyMap` function, and then the `remap` function remaps the target image based on the generated distortion map, resulting in a first full-frame distorted image containing all pixel information of the target image and with known fill grayscale values. The first full-frame distorted image is generated by... Figure 2 The image consists of all pixels and fill pixels. The fill grayscale value used in this invention is 128. Figure 3 The gray area around the center of the image is shown.

[0143] Step 3: Preprocess and threshold segment the first full-frame distortion-free image.

[0144] This step includes the following sub-steps:

[0145] Step 3.1: Convert the target image into a single-channel target image using OpenCV's cvtColor function.

[0146] Step 3.2: Using OpenCV's adaptiveThreshold function, perform thresholding on the single-channel target image to obtain a binary image containing only foreground and background pixels.

[0147] Step 3.3: Using OpenCV's morphologyEx function, perform an opening operation on the binary image to remove small objects and noise, and smooth the contours. The resulting binary image is shown below. Figure 7 As shown.

[0148] Step 4: Extract and filter the contours of the binary image.

[0149] This step includes the following sub-steps:

[0150] Step 4.1: Using OpenCV's findContours function, perform contour searching on the processed binary image to obtain all contour information within the image. For example... Figure 8 As shown, each color represents an outline.

[0151] Step 4.2: Iterate through all the contour information extracted in Step 4.1 and filter the target contour.

[0152] This step includes the following sub-steps:

[0153] Step 4.2.1: In a normal full-frame distortion correction image, the contour formed by the filled region is the outer contour (the contour located at the outermost layer of the same connected region and not surrounded by any other contour). Determine whether the candidate contour is an outer contour based on the contour attributes returned by the findContours function; if not, skip it.

[0154] Step 4.2.2: Obtain the area of ​​the candidate contour using OpenCV's contourArea function, and determine whether the area of ​​the current contour is less than the preset area. If so, skip it.

[0155] Step 4.2.3: Obtain the minimum bounding rectangle of the candidate contour using OpenCV's boundingRect function, and determine whether the minimum bounding rectangle of the current contour is located at the image boundary. If not, skip it.

[0156] Step 4.2.4: In a normal full-frame distortion correction image, the contour formed by the filled region does not contain sub-contours. The candidate contour is determined based on the contour properties returned by the findContours function; if it does, it is skipped.

[0157] Step 4.2.5: Save all the outlines and their smallest bounding rectangles after filtering in Steps 4.2.1-4. For example... Figure 9 As shown.

[0158] Step 5: Verify the grayscale of the full-frame distortion-free image corresponding to the target contour.

[0159] This step includes the following sub-steps:

[0160] Step 5.1: Create a mask image with the same resolution as the first full-frame distortion-corrected image and fill all pixel areas with zeros. Then, using OpenCV's drawContours function, draw the target contour obtained in Step 4 on the mask image and fill the contour with non-zero values. The fill grayscale value used in this invention is 255, such as... Figure 10 The white area around the center of the image is shown.

[0161] Step 5.2: Since only the pixel values ​​of the target contour-covered area in the mask image are non-zero, the mean gray value of the area corresponding to the target contour in the full-frame distortion-free image is calculated using OpenCV's mean function.

[0162] Step 5.3: In a normal full-frame distortion correction image, there is a clear boundary between the filled area and the valid image. Determine whether the deviation between the grayscale mean obtained in Step 5.2 and the filled grayscale value is less than the preset deviation (e.g., 0.05). If so, the extracted target contour is considered valid; otherwise, it is considered that there are pixels of the original target image inside the target contour, and the full-frame distortion correction image is considered to have distortion correction anomalies.

[0163] Step 6: Verify the distribution and geometric features of the target contour.

[0164] This step includes the following sub-steps:

[0165] Step 6.1: In a normal full-frame distortion correction image, the filled region is distributed around the perimeter of the image. Determine whether the number of the minimum bounding rectangles of the target contour obtained in Step 4.2 is the first preset number (e.g., 4). If not, then the first full-frame distortion correction image is determined to be abnormal.

[0166] Step 6.2: Classify and label the minimum bounding rectangle of the target contour obtained in Step 4.2 according to the position of its boundary in the image (top edge, bottom edge, left edge, right edge); if the minimum bounding rectangle has two sides located at the image edge, then label it twice.

[0167] Step 6.3: In a normal full-frame distortion correction image, each edge of the image has exactly one filled region. Traverse each edge of the image and determine whether the number of the smallest bounding rectangles located at that edge is a second preset number (e.g., 1). If not, the first full-frame distortion correction image is determined to be abnormal.

[0168] Step 6.4: In a normal full-frame distortion correction image, the filled regions are symmetrically distributed vertically within a certain range. Calculate whether the height difference between the minimum bounding rectangles located at the top and bottom edges of the image is less than the corresponding preset deviation. If not, the first full-frame distortion correction image is determined to be abnormal.

[0169] Step 6.5: In a normal full-frame distortion correction image, the filled regions are symmetrically distributed within a certain range. Calculate whether the width difference between the smallest bounding rectangles located at the left and right edges of the image is less than the corresponding preset deviation. If not, the first full-frame distortion correction image is determined to be abnormal.

[0170] It should be noted that in normal full-frame distortion correction, there is only one symmetrical rectangle in the top, bottom, left, and right. These areas have no effective pixels. The first preset number, the second preset number, and the preset deviation mentioned above are determined based on the center alignment of the sensor / lens and the refraction of the lens by the optical fiber.

[0171] In the above embodiments, the target image is subjected to lossless distortion correction based on the internal parameters of the camera to obtain a first full-frame distorted image that maintains the original image size and does not lose any pixels. Based on the fact that the pixels in the original distorted image have certain geometric characteristics in the mapping position of the output full-frame distorted image, the distribution characteristics and geometric features of the target contour in the first full-frame distorted image are determined. Based on the distribution and geometric features, the overall anomalies of the full-frame distorted image are effectively identified, improving the accuracy of anomaly detection.

[0172] Based on the aforementioned camera distortion correction anomaly detection method, anomalies in the full-frame distortion correction image can be identified based on the overall geometric characteristics of the image. To further evaluate the distortion correction effect, anomalies can be detected in the camera's internal distortion parameters.

[0173] In one exemplary embodiment, such as Figure 13 As shown, a method for detecting distortion anomalies in a camera is provided, including the following steps:

[0174] Step 1302: Obtain internal parameters of the camera and target image.

[0175] Understandably, detecting abnormalities in distortion parameters can utilize the distortion principle of the pinhole model to correct distortion in a preset image. The distortion correction effect is evaluated by determining whether a preset special straight line in the full-frame dedistorted image retains its geometric characteristics. For example, the straightness of a preset straight line in the full-frame dedistorted image can be checked to see if it exceeds a threshold to assess the distortion correction effect. The pinhole distortion model is described by both radial and tangential distortion. The expression for radial distortion is:

[0176] ;

[0177] ;

[0178] The expression for tangential distortion is:

[0179] ;

[0180] ;

[0181] here, For radial distortion parameters, These are tangential distortion parameters; For distortion-free normalized image coordinates, The normalized image coordinates represent the distortion. , which represents the normalized distance between a pixel and the center of the image.

[0182] As can be seen from the radial distortion model, its core feature is symmetry: the distortion direction is entirely radial, and the amount of distortion depends only on the radial distance from the point to the image center. Therefore, the degree of distortion is exactly the same on any concentric circles centered at the image center.

[0183] The tangential distortion model shows that although the displacement caused by distortion is not uniform, when the point is located on certain specific straight lines (such as horizontal / vertical midlines and diagonals), tangential distortion does not introduce nonlinear displacements that cause the straight lines to bend.

[0184] Based on the above characteristics, horizontal / vertical midlines and diagonals, due to their geometric specialties, can maintain linearity in distortion correction calculations. Therefore, these straight lines can serve as unchanging "baselines" during distortion correction, allowing for a visual assessment of the distortion correction effect. Figure 14 As shown, in an exemplary embodiment, a straight line in a distortion graph, and Figure 4 In one embodiment shown, straight lines in the full-frame distortion-free image, such as Figure 5 A straight line in a full-width distortion anomaly plot shown in one embodiment.

[0185] Step 1304: Generate at least one preset image including the preset geometric reference information based on the preset geometric reference information and the resolution of the target image.

[0186] The preset geometric reference information serves as a baseline for the distortion correction process, allowing for a direct assessment of the distortion effect. This preset geometric reference information is determined based on maintaining linearity during distortion calculations. It can include horizontal / vertical midlines, diagonals, and concentric circles. The generated preset image contains only one type of preset geometric reference information, and its resolution is the same as the target image. The number of preset images can be one or multiple; evaluating the distortion correction effect based on multiple preset images further ensures accuracy.

[0187] It should be noted that the generated preset image only needs to use the resolution of the target image, and does not concern itself with the target image itself. The generation parameters of the preset image include the grayscale values ​​of the foreground and background, as well as the pixel width of the line. Each image contains only one line to simplify the line detection process. To improve computational efficiency, the background grayscale value in this embodiment is uniformly set to 0 (pure black), and the foreground (i.e., the line) grayscale value is set to 255 (pure white). To comprehensively detect distortion anomalies, the generated preset image may include horizontal / vertical midlines and diagonals, or concentric circles. For example, preset image A includes a horizontal midline, preset image B includes a vertical midline, preset image C includes a diagonal, and preset image D includes concentric circles.

[0188] Step 1306: Perform lossless distortion correction on each preset image according to the internal parameters to obtain the corresponding second full-frame distortion-free image.

[0189] The non-destructive distortion correction of the preset image can be achieved through existing methods, which will not be elaborated here.

[0190] For example, a distortion map is generated according to the OpenCV initUndistortRectifyMap function, and then the remap function remaps each preset image according to the generated distortion map to obtain the second full-frame distortion-free image corresponding to each image, which contains all pixel information of the preset image and is filled with grayscale values ​​of the preset background grayscale value.

[0191] Step 1308: Detect the geometric information in each second full-frame distortion-free image to obtain the detection results, and determine the abnormal results of the distortion coefficients in the internal parameters based on each detection result.

[0192] It is understandable that the geometric information in a normal full-frame distortion-corrected image should not exhibit any anomalies. This geometric information can be either straight lines or concentric circles in the second full-frame distortion-corrected image. Straight lines and concentric circles can be represented by non-zero pixels. Detection results for straight lines can include the number of fitted straight lines and / or their straightness. Detection results for concentric circles can include the closure data and roundness of the concentric circles. For example, if holes appear, the roundness of concentric circles will be disrupted and straight lines will be segmented. Therefore, anomalies in distortion coefficients can be detected by pre-setting concentric circles or pre-setting straight lines.

[0193] If concentric circles exist in multiple second full-frame distortion correction images, then the concentric circles are formed with the center of the image as the center and a preset distance as the radius. The radii of the concentric circles in different second full-frame distortion correction images can be different.

[0194] For example, for each second full-frame distortion-free image, geometric information is extracted from it. The extracted geometric information is then inspected to obtain the inspection result. If the inspection result is not a preset result, an abnormal result for the distortion coefficient in the corresponding internal parameters is considered an anomaly. Only when the distortion coefficients of all second full-frame distortion-free images are normal can the final result of normal distortion coefficients be obtained. Extracting the geometric information from each second full-frame distortion-free image can involve extracting the non-zero pixel positions to obtain the corresponding pixel points. The extraction method can be, but is not limited to, using OpenCV's `findNonZero` function.

[0195] In the aforementioned method for anomaly detection using camera distortion correction, a set of preset images with the same resolution are generated based on the target image's resolution and preset geometric reference information. Subsequently, distortion correction is performed on these images based on the camera's internal parameters. Finally, the preset geometric structures are detected in the resulting full-frame distortion correction image, and their geometrical properties are calculated to evaluate the effectiveness of the current distortion coefficients. This detection only requires the preset geometric reference information and eliminates the need for preprocessing, checkerboard recognition, corner extraction, row and column determination, and line fitting of traditional calibration board methods. Instead, it fully utilizes the constraints of prior information to directly search for target geometric information in the full-frame distortion correction image. This method, combined with optimization strategies, can significantly reduce computational costs while maintaining a high detection rate.

[0196] The following provides two methods for determining anomalous results of distortion coefficients in intrinsic parameters based on geometric information in the second full-frame distortion map, to support the needs of different scenarios.

[0197] Method 1: Preset geometric reference information, including preset straight line information, is used to detect the geometric information in each second full-frame distortion correction image. Detection results are obtained, and based on these results, abnormal distortion coefficients in the internal parameters are determined, including:

[0198] For each second full-frame distortion correction image, traverse each pixel in the second full-frame distortion correction image, extract the non-zero pixel positions in the second full-frame distortion correction image based on the pixel values ​​of the traversed pixels, and obtain the first set of non-zero pixels used to represent the straight line; fit the non-zero pixel positions of each non-zero pixel in the first set of non-zero pixels to obtain the number of fitted straight lines and determine the straightness of the fitted straight lines; if the number of straight lines and / or the straightness do not meet their respective preset values, then the abnormal result of the distortion coefficient in the internal parameters is determined to be an abnormal distortion coefficient.

[0199] As can be understood, based on the above processing steps, the background grayscale value in the preset image is uniformly set to 0 (pure black), and the foreground (i.e., straight line) grayscale value is set to 255 (pure white). Therefore, non-zero pixel positions are extracted from the second full-frame distortion-free image, which is equivalent to extracting straight line information. The fitting method based on non-zero pixel positions can be implemented using existing methods, and will not be elaborated upon here.

[0200] Straightness can be determined by calculating the standard deviation of the straight-line distances from all non-zero pixels to the corresponding fitted straight line. The specific calculation method can be implemented using existing methods and will not be elaborated here. The preset number of straight lines can be 1, and the preset straightness corresponding to the straightness is predetermined.

[0201] In one exemplary embodiment, a camera distortion correction anomaly detection method based on preset line information is provided, such as... Figure 15 As shown, it includes the following steps:

[0202] Step 1: Obtain the camera's internal parameters and the target image.

[0203] It should be noted that the target image is any distorted image captured by the camera.

[0204] Step 2: Generate a set of preset images with the same resolution as the target image based on the preset line information.

[0205] This step can be implemented using the methods described above, and will not be elaborated upon here.

[0206] Step 3: Perform non-destructive distortion correction on the preset image to obtain the first full-frame distortion-free image.

[0207] Step 4: Extract the preset straight line information from the full-frame distortion-free image.

[0208] Extract the non-zero pixel positions in the full-frame distortion-free image using OpenCV's findNonZero function.

[0209] Step 5: Calculate the number of lines and the straightness of the extracted lines.

[0210] It should be noted that the specific implementation of this example can be achieved using the methods described above, and will not be elaborated upon here.

[0211] In this approach, by directly utilizing preset straight line information, a constrained fast search is performed on the full-scale distortion-free image, which significantly improves the processing speed and ensures the detection rate of coefficient anomalies.

[0212] Method 2: Preset geometric reference information, including preset concentric circle information, is used to detect the geometric information in each second full-frame distortion-corrected image. Detection results are obtained, and based on these results, abnormal distortion coefficients in the internal parameters are determined, including:

[0213] Traverse each pixel in the second full-frame distortion-free image, extract the non-zero pixel positions in the second full-frame distortion-free image based on the pixel values ​​of the traversed pixels, and obtain the second set of non-zero pixels used to represent the concentric circles; based on the second set of non-zero pixels, determine the closure data and concentric circle roundness of the concentric circles in the first full-frame distortion-free image; if the closure data and / or concentric circle roundness do not meet their respective preset values, then determine that the abnormal result of the distortion coefficient in the internal parameters is that the distortion coefficient is abnormal.

[0214] The determination of closed data and concentric circle roundness can be achieved using existing methods. For example, if there are N concentric circles in the image before distortion correction, then the number of closed concentric circles in the full-frame distorted image will also be N. For instance, the roundness of concentric circles after distortion correction should be greater than 0.9. The formula for calculating concentric circle roundness is (4π × Area) / (Perimeter × Perimeter), where Area is the area and Perimeter is the perimeter, with a value range of (0,1]. A larger value indicates a circle closer to its shape. The roundness calculation formula is a widely used shape analysis metric and will not be elaborated upon here. The closed data and concentric circle roundness not meeting their respective preset values ​​are all pre-defined.

[0215] Understandably, based on the above processing steps, the background grayscale value in the preset image is uniformly set to 0 (pure black), and the foreground (i.e., concentric circles) grayscale value is set to 255 (pure white). Therefore, non-zero pixel positions are extracted from the second full-frame distortion-free image, which is equivalent to extracting the concentric circle information.

[0216] In this approach, by directly utilizing pre-defined concentric circles, a constrained fast search is performed on the full-frame distortion-free image, which significantly improves processing speed and ensures the detection rate of coefficient anomalies.

[0217] In an exemplary embodiment, to achieve comprehensive closed-loop detection of camera distortion coefficients and their distortion correction effects, a camera distortion anomaly detection method is provided, such as... Figure 16 As shown, it includes the following steps:

[0218] Step 1602: Obtain internal parameters of the camera and target image.

[0219] Step 1604: Generate a preset image including the preset geometric reference information based on the preset geometric reference information and the resolution of the target image.

[0220] Step 1606: Perform lossless distortion correction on the target image and the preset image according to the internal parameters to obtain the corresponding second full-frame distortion correction image and the first full-frame distortion correction image.

[0221] Step 1608: Detect the geometric information in the second full-frame distortion-free image to obtain the detection results, and determine the abnormal results of the distortion coefficients in the internal parameters based on the detection results.

[0222] Step 1610: Extract the target contour from the second full-frame distortion-free image and determine the distribution and geometric features of the target contour in the first full-frame distortion-free image.

[0223] Step 1612: Verify the distribution characteristics and / or geometric characteristics to determine the distortion anomaly results of the first full-frame distortion map.

[0224] The specific implementation of this example can be achieved through the steps defined in the above embodiments, and will not be elaborated here.

[0225] It should be noted that before detecting abnormal distortion results of the first full-frame distorted image, the distortion coefficient in the internal parameters can be detected. If the distortion coefficient in the internal parameters is normal, the relevant steps for detecting abnormal distortion results of the first full-frame distorted image can be performed. Alternatively, the camera distortion coefficient and its distortion correction effect can be detected directly to achieve comprehensive closed-loop detection.

[0226] Based on the above-described camera distortion correction anomaly detection method, each edge of the image (top, bottom, left, and right) has exactly one minimum bounding rectangle. If this criterion is not met, anomalies can be detected. Figure 17 The anomalies shown in the image are as follows: if the height difference between the minimum bounding rectangles of the top and bottom edges of the image is less than a preset deviation, and the width difference between the minimum bounding rectangles of the left and right edges of the image is less than a preset deviation, then anomalies can be detected. Figure 18 The edge anomalies shown can be detected if four contours can be detected at the top, bottom, left, and right edges of the image. Figure 19 The distortion ring shown.

[0227] In the above embodiments, a set of preset images with the same resolution are generated based on the resolution of the target image and preset geometric reference information; then, the image is distorted based on the internal parameters of the camera, and finally the preset geometric structure is detected in the obtained full-frame distortion-free image, and its geometric related attributes are calculated to evaluate the effectiveness of the current distortion coefficient. This detection method only requires pre-set geometric reference information, eliminating the need for preprocessing, checkerboard recognition, corner extraction, row and column judgment, and line fitting of the target image as in traditional calibration plate methods. Instead, it fully utilizes the constraints of prior information to directly search for target geometric information in the full-frame distortion correction image, significantly reducing computational costs while maintaining a high detection rate. Furthermore, by extracting the target contour and considering its distribution and geometric features across the entire image, it can effectively identify anomalies in the full-frame distortion correction image, avoiding the limitation of only identifying anomalies in localized areas and improving the accuracy of camera distortion correction anomaly detection. In other words, based on the overall geometric characteristics of the full-frame distortion correction image, combined with pre-set "baseline" features and invalid pixel (i.e., image filling area) information, it can effectively identify distortion correction anomalies across the entire image, achieving comprehensive closed-loop detection of camera distortion coefficients and their distortion correction effects.

[0228] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0229] Based on the same inventive concept, this application also provides a camera distortion correction anomaly detection device for implementing the camera distortion correction anomaly detection method described above. The solution provided by this device is similar to the solution described in the above method; therefore, the specific limitations of the one or more camera distortion correction anomaly detection device embodiments provided below can be found in the limitations of the camera distortion correction anomaly detection method described above, and will not be repeated here.

[0230] In one exemplary embodiment, such as Figure 20 As shown, a camera distortion correction and anomaly detection device is provided, comprising: a data acquisition module 2002, an image generation module 2004, a distortion correction module 2006, a contour extraction module 2008, and an image detection module 2010, wherein:

[0231] The data acquisition module 2002 is used to acquire internal parameters of the camera and target images.

[0232] Image generation module 2004 is used to generate a preset image including preset geometric reference information based on preset geometric reference information and the resolution of the target image.

[0233] The distortion correction module 2006 is used to perform non-destructive distortion correction on the target image based on internal parameters to obtain the first full-frame distortion-free image.

[0234] The contour extraction module 2008 is used to extract the target contour from the first full-frame distortion-free image.

[0235] The image detection module 2010 is used to determine the distribution features and geometric features of the target contour in the first full-frame distortion-free image; to verify the distribution features and / or geometric features, and to determine the distortion-free anomaly results of the first full-frame distortion-free image.

[0236] The aforementioned camera distortion correction anomaly detection device directly acquires the target image to be processed, along with the camera's internal parameters, eliminating the need for a laboratory environment and manual calibration plate placement to determine the target image. It performs non-destructive distortion correction on the target image based on the internal parameters, obtaining a first full-frame distortion correction image that maintains the original image size and does not lose pixels. The device then extracts the target contour from the first full-frame distortion correction image, determines its distribution and geometric features, and verifies these features to identify any distortion anomalies. In other words, by extracting the target contour and considering its distribution and geometric features across the entire image, the device effectively identifies anomalies present in the full-frame distortion correction image, avoiding the limitation of only identifying anomalies in localized areas and improving the accuracy of camera distortion correction anomaly detection.

[0237] In an exemplary embodiment, the camera distortion correction anomaly detection device further includes an image preprocessing module and a threshold segmentation module, wherein the image preprocessing module is used to convert the first full-frame distortion correction image to obtain a single-channel target image;

[0238] The threshold segmentation module is used to perform threshold segmentation on a single-channel target image to obtain a binary image containing only foreground and background pixels;

[0239] The contour extraction module 2008 is used to extract contours from binary images, obtain contour information of all extracted candidate contours, and filter out the target contour based on the contour information.

[0240] In an exemplary embodiment, the camera distortion correction anomaly detection device further includes a verification module, configured to create a mask image with the same resolution as the first full-frame distortion correction image; determine the target contour in the mask image based on the position of the target contour in the first full-frame distortion correction image, and set the pixel values ​​of each pixel in the pixel-filled region corresponding to the target contour in the mask image as the first pixel value; determine the gray-scale mean of the pixel-filled region in the first full-frame distortion correction image; and if the difference between the first pixel value and the gray-scale mean is within a preset range, the target contour is valid.

[0241] In an exemplary embodiment, the image detection module 2010 is further configured to determine the filling region formed by the target contour and the image edge of the first full-frame distortion map, determine the first number of minimum bounding polygons formed by the filling region, and obtain the distribution characteristics of the target contour in the first full-frame distortion map.

[0242] Determine the position of the minimum bounding polygon in the first full-frame distortion image, and determine the second number of minimum bounding polygons corresponding to each image edge in the first full-frame distortion image based on the position;

[0243] Based on the distribution characteristics of the pixel filling region, determine the geometric difference between the minimum bounding polygons corresponding to the image edges in the preset direction that correspond to the distribution characteristics;

[0244] The second quantity and / or geometric difference are used to determine the geometric features of the target profile in the first full-frame distortion map.

[0245] In an exemplary embodiment, the image detection module 2010 is further configured to determine that the distortion anomaly result of the first full-frame distortion-corrected image is that distortion anomaly exists if at least one of the following conditions exists: the first quantity is not a first preset quantity value, the second quantity is not a second preset quantity value, and the geometric difference is less than a preset deviation.

[0246] In an exemplary embodiment, the camera distortion correction anomaly detection device further includes a parameter detection module, which is used to generate at least one preset image including the preset geometric reference information based on preset geometric reference information and the resolution of the target image.

[0247] Based on the internal parameters, each preset image is subjected to non-destructive distortion correction to obtain its corresponding second full-frame distortion-free image.

[0248] The geometric information in each second full-frame distortion correction image is detected to obtain the detection results. Based on the detection results, the abnormal results of the distortion coefficients in the internal parameters are determined.

[0249] In an exemplary embodiment, the preset geometric reference information includes preset straight line information. The parameter detection module is further configured to, for each second full-frame distortion image, traverse each pixel in the second full-frame distortion image, extract the non-zero pixel positions in the second full-frame distortion image based on the pixel values ​​of the traversed pixels, and obtain a first set of non-zero pixel points used to represent the straight line.

[0250] The number of fitted lines and the straightness of the fitted lines are obtained by fitting the non-zero pixel positions of each non-zero pixel in the first set of non-zero pixels.

[0251] If the number of lines and / or straightness do not meet their respective preset values, then the abnormal result of the distortion coefficient in the internal parameters is determined to be an abnormality of the distortion coefficient.

[0252] In an exemplary embodiment, the preset geometric reference information includes preset concentric circle information. The parameter detection module is further configured to traverse each pixel in the second full-frame distortion image, extract the non-zero pixel positions in the second full-frame distortion image based on the pixel values ​​of the traversed pixels, and obtain a second set of non-zero pixel points used to represent the concentric circles.

[0253] Based on the second set of non-zero pixels, determine the closure data and roundness of the concentric circles in the first full-frame distortion-free image;

[0254] If the closed data and / or the roundness of the concentric circles do not meet their respective preset values, then the abnormal result of the distortion coefficient in the internal parameters is determined to be an abnormality of the distortion coefficient.

[0255] Each module in the aforementioned camera distortion correction anomaly detection device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in the processor of a computer device in hardware form or independent of it, or stored in the memory of a computer device in software form, so that the processor can call and execute the corresponding operations of each module.

[0256] In one exemplary embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be as follows: Figure 21 As shown, the computer device includes a processor, memory, input / output interfaces, a communication interface, a display unit, and an input device. The processor, memory, and input / output interfaces are connected via a system bus, and the communication interface, display unit, and input device are also connected to the system bus via the input / output interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The input / output interfaces are used for exchanging information between the processor and external devices. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, mobile cellular networks, Near Field Communication (NFC), or other technologies. When the computer program is executed by the processor, it implements a method for detecting camera distortion anomalies. The display unit is used to form a visually visible image and can be a display screen, a projection device, or a virtual reality imaging device. The display screen can be an LCD screen or an e-ink screen. The input device of the computer device can be a touch layer covering the display screen, or buttons, trackballs, or touchpads set on the casing of the computer device, or external keyboards, touchpads, or mice, etc.

[0257] Those skilled in the art will understand that Figure 21 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0258] In one embodiment, a computer device is also provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above method embodiments.

[0259] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon that, when executed by a processor, implements the steps in the above method embodiments.

[0260] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above method embodiments.

[0261] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data must comply with relevant regulations.

[0262] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, artificial intelligence (AI) processors, etc., and are not limited to these.

[0263] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0264] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A method for detecting distortion anomalies in a camera, characterized in that, The method includes: Acquire internal parameters from the camera and the target image; Based on the internal parameters, the target image is subjected to non-destructive distortion correction to obtain a first full-frame distortion-free image; Extract the target contour from the first full-frame distortion-free image; Determine the distribution and geometric features of the target contour in the first full-frame distortion-free image; The distribution features and / or geometric features are verified to determine the distortion anomaly result of the first full-frame distortion map.

2. The method according to claim 1, characterized in that, Extracting the target contour from the first full-frame distortion-free image includes: The first full-frame distortion-free image is converted to obtain a single-channel target image; The single-channel target image is subjected to threshold segmentation to obtain a binary image containing only foreground and background pixels; Contour extraction is performed on the binary image to obtain contour information of all extracted candidate contours, and the target contour is selected based on the contour information.

3. The method according to claim 1, characterized in that, Before determining the distribution and geometric features of the target contour in the first full-frame distortion-free image, the method further includes: Create a mask image with the same resolution as the first full-frame distortion-free image; Based on the position of the target contour in the first full-frame distortion correction image, the target contour in the mask image is determined, and the pixel value of each pixel in the pixel filling region corresponding to the target contour in the mask image is set to the first pixel value. Determine the average grayscale value of the pixel-filled region in the first full-frame distortion-free image; If the difference between the first pixel value and the mean gray value is within a preset range, then the target contour is valid.

4. The method according to claim 2, characterized in that, Determining the distribution and geometric features of the target contour in the first full-frame distortion-free image includes: Determine the filling region formed by the target contour and the image edge of the first full-frame distortion map, determine the first number of minimum bounding polygons formed by the filling region, and obtain the distribution characteristics of the target contour in the first full-frame distortion map; Determine the position of the minimum bounding polygon in the first full-frame distortion image, and determine the second number of the minimum bounding polygons corresponding to each image edge in the first full-frame distortion image based on the position; Based on the distribution characteristics of the pixel filling region, determine the geometric difference between the minimum bounding polygons corresponding to the image edges in the preset direction corresponding to the distribution characteristics; The second quantity and / or the geometric difference are used to determine the geometric features of the target contour in the first full-frame distortion map.

5. The method according to claim 4, characterized in that, The step of verifying the distribution features and / or the geometric features to determine the distortion anomaly result of the first full-frame distortion-corrected image includes: If at least one of the following conditions exists: the first quantity is not a first preset quantity value, the second quantity is not a second preset quantity value, and the geometric difference is less than a preset deviation, then the distortion anomaly result of the first full-frame distortion-corrected image is determined to have a distortion anomaly.

6. The method according to any one of claims 1 to 5, characterized in that, The method further includes: Based on the preset geometric reference information and the resolution of the target image, at least one preset image including the preset geometric reference information is generated; According to the internal parameters, each preset image is subjected to lossless distortion correction to obtain the corresponding second full-frame distortion-free image. The geometric information in each of the second full-frame distortion-free images is detected to obtain the detection results. Based on the detection results, the abnormal results of the distortion coefficients in the internal parameters are determined.

7. The method according to claim 6, characterized in that, The preset geometric reference information includes preset straight line information. The step of detecting the geometric information in each of the second full-frame distortion-free images to obtain detection results, and determining abnormal results of the distortion coefficients in the internal parameters based on each detection result, includes: For each of the second full-frame distortion maps, traverse each pixel in the second full-frame distortion map, extract the non-zero pixel positions in the second full-frame distortion map based on the pixel values ​​of the traversed pixels, and obtain the first set of non-zero pixel points used to represent the straight line; The number of fitted lines is obtained by fitting the non-zero pixel positions of each non-zero pixel in the first set of non-zero pixels, and the straightness of the fitted lines is determined. If the number of straight lines and / or the straightness do not meet their respective preset values, then the abnormal result of the distortion coefficient in the internal parameters is determined to be an abnormality of the distortion coefficient.

8. The method according to claim 6, characterized in that, The preset geometric reference information includes preset concentric circle information. The step of detecting the geometric information in each of the second full-frame distortion-free images to obtain detection results, and determining abnormal results of the distortion coefficients in the internal parameters based on each detection result, includes: Traverse each pixel in the second full-frame distortion-free image, and extract the non-zero pixel positions in the second full-frame distortion-free image based on the pixel values ​​of the traversed pixels to obtain the second set of non-zero pixels used to represent concentric circles; Based on the second set of non-zero pixels, determine the closure data and roundness of the concentric circles in the first full-frame distortion correction image; If the closed data and / or the roundness of the concentric circles do not meet their respective preset values, then the abnormal result of the distortion coefficient in the internal parameters is determined to be an abnormal distortion coefficient.

9. A camera distortion correction anomaly detection device, characterized in that, The device includes: The data acquisition module is used to acquire internal parameters of the camera and target images; An image generation module is used to generate a preset image including the preset geometric reference information based on preset geometric reference information and the resolution of the target image; The distortion correction module is used to perform non-destructive distortion correction on the target image according to the internal parameters to obtain a first full-frame distortion-free image; The contour extraction module is used to extract the target contour from the first full-frame distortion-free image; The image detection module is used to determine the distribution features and geometric features of the target contour in the first full-frame distortion-free image; to verify the distribution features and / or the geometric features, and to determine the distortion-free anomaly result of the first full-frame distortion-free image.

10. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 7.

11. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 7.

Images