Image clarity sorting method, device, electronic device and storage medium

Abstract

The present application provides an image clarity sorting method, device, electronic device and storage medium, which are applied to the field of image processing technology. By reading the video frames in the calibration video, the integrity of the video frames is judged frame by frame, and the complete image frames are obtained according to the integrity; then, the clarity and unilateral blur of each complete image frame are calculated, and the quality of the selected complete image frames is evaluated. By setting a first preset value, the first complete image frame that is not suitable for accurate camera calibration due to camera shake or other factors is removed, and the clarity of the remaining second complete image frames is sorted to improve the accuracy and quality of the calibration results. In the above manner, the algorithm can automatically select high-quality checkerboard images from the video, with less calculation, and reduces human intervention, reduces the error rate, and improves the efficiency and accuracy of the entire camera calibration process.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to an image clarity sorting method, device, electronic device and storage medium. Background Art

[0002] In the image measurement process and machine vision applications, in order to determine the relationship between the three-dimensional geometric position of a point on the surface of a spatial object and its corresponding point in the image, a geometric model of camera imaging, namely, camera parameters, must be established. The accuracy of the camera parameter calibration results and the stability of the algorithm directly affect the accuracy of the results produced by the camera operation. Chessboard calibration is a common method for camera calibration. By binarizing the clear camera image and finding quadrilaterals (black chessboard areas) to find candidate points for chessboard corners, only quadrilaterals that meet specific size standards are retained and merged into a regular grid structure. Through a specific process algorithm, the camera's internal and external parameters and distortion coefficients are calculated and then used for image correction.

[0003] An important prerequisite for camera calibration is the need for clear camera images. In the traditional process, there are roughly two ways to obtain checkerboard images: 1. Hold the checkerboard printed on A4 paper and change different angles to shoot videos facing a fixed camera; 2. Fix the checkerboard image and shoot videos from different angles with a handheld camera. After obtaining the checkerboard video in the two ways, multiple high-definition checkerboard images at different angles are screened manually or automatically. However, the above method works well in the calibration of a single camera, but there are still several problems for the calibration of multiple cameras: first, the videos shot from both sides need to be saved before image screening, which puts a great burden on computer storage devices; second, manual screening of images may cause image screening errors due to visual fatigue; finally, most methods use automatic image screening methods including edge detection, frequency domain filtering, etc., which will cause a lot of computational burden on computer hardware.

[0004] Therefore, in the calibration of multiple cameras, the existing method of screening clear chessboard images has the problems of heavy burden on computer software and hardware and low accuracy of manual screening. Summary of the invention

[0005] In view of the above-mentioned shortcomings of the prior art, the present application provides an image clarity sorting method, device, electronic device and storage medium, which are applied to the field of image processing technology. The method automatically reads the video frames in the calibration video, and judges the integrity of the video frames frame by frame. The complete image frames with high integrity are selected to calculate the clarity and unilateral blur of the image, and then the second complete image frames with high clarity are screened out according to the calculation results for sorting, thereby realizing the screening of clear images while shooting the video, reducing the calculation degree of the computer, and the screening process does not require human participation, which can improve the efficiency and quality of camera calibration.

[0006] In a first aspect, the present application provides a method for sorting image clarity, the method comprising the steps of:

[0007] S1: read the video frame in the calibration video;

[0008] S2: determining the integrity of the video frame frame by frame, and obtaining a complete image frame according to the integrity;

[0009] S3: Calculate the clarity and unilateral blur of each complete image frame;

[0010] S4: Screening out the first complete image frame corresponding to the unilateral blur less than the first preset value, and sorting the remaining second complete image frames according to clarity, the complete image frames including the first complete image frame and the second complete image frame.

[0011] The present application proposes an image clarity sorting method, which provides raw materials for subsequent image analysis and processing by reading video frames in the calibration video; judging the integrity of the video frame frame by frame, and obtaining the complete image frame according to the integrity, which can ensure that the selected complete image frame is suitable for camera calibration, and facilitate the subsequent accurate calculation of camera parameters; by calculating the clarity and unilateral blur of each complete image frame, the quality of the selected complete image frame is evaluated, the clarity reflects the overall sharpness of the image, and the unilateral blur detects whether the image is blurred in a specific direction; the first complete image frame corresponding to the unilateral blur less than the first preset value is screened out, and by setting the first preset value, the first complete image frame that is not suitable for accurate camera calibration due to camera shake or other factors is removed, which is conducive to the subsequent accurate calculation of camera parameters, and the clarity of the remaining second complete image frames is sorted, and the image with the highest clarity is preferentially selected for camera calibration to improve the accuracy and quality of the calibration result. In the above manner, the algorithm can automatically select high-quality checkerboard images from the video, with less calculation, and reduces human intervention, reduces the error rate, and improves the efficiency and accuracy of the entire camera calibration process.

[0012] Further, step S2 includes:

[0013] S21: performing grayscale processing on the video frames frame by frame to obtain a grayscale image;

[0014] S23: Sliding a window on the grayscale image to detect corner points, and calculating the corner point strength of the corner points;

[0015] S24: Select a candidate point with the largest intensity and conforming to the checkerboard corner point distribution rule from the corner point intensities, and obtain the video frame corresponding to the candidate point as a complete image frame, wherein the checkerboard corner point distribution rule represents the completeness of the video frame.

[0016] The present application provides an image clarity sorting method, which converts color video frames into grayscale images by grayscale processing the video frames frame by frame, thereby reducing the amount of data to be processed and improving the efficiency of subsequent image processing steps; by sliding a window on the grayscale image, the entire image area can be systematically covered to detect possible corner points, and then the corner point strengths of the corner points are calculated, which helps to identify significant geometric change points in the image; selecting candidate points with the largest intensity and conforming to the checkerboard corner point distribution law from the corner point strengths means that the algorithm will identify corner points that can form a clear and regular checkerboard pattern, and the distribution law of these corner points is used to determine whether the video frame contains a complete checkerboard pattern, thereby judging the integrity of the video frame.

[0017] Further, step S21 includes:

[0018] S211: Splitting the video frame into a left view and a right view frame by frame;

[0019] S212: Perform grayscale processing on the left view and the right view respectively to obtain grayscale images.

[0020] The present application proposes an image clarity sorting method, which horizontally divides the video frame into independent parts, namely the left view and the right view, so that the computer can process each view separately, thereby more accurately analyzing and identifying the checkerboard features captured by each camera, which is beneficial to the parameter calculation of multiple camera calibrations; the left view and the right view are processed by grayscale values ​​respectively, which can remove the color information in the image, reduce the complexity of data processing, and improve the recognition efficiency of corner detection.

[0021] Further, step S3 includes:

[0022] S31: Obtain the corner points of each complete image frame, and cut out a k*k image sub-region with the corner point coordinates as the center;

[0023] S32: Filter out first relative rows and first relative columns that are opposite to each other in pairs among four boundaries parallel to the image sub-region, and calculate first widths and second widths of black-white transition intervals in the first relative rows and the first relative columns;

[0024] S33: Calculate the clarity and unilateral blurriness of the complete image frame according to the first width and the second width.

[0025] The present application proposes an image clarity sorting method, which obtains the corner points of each complete image frame and extracts a small image area, namely a sub-area, with the corner point coordinates as the center. By analyzing the sub-area, the algorithm can more accurately evaluate the local clarity characteristics of the image; in each sub-area, the first relative row and the first relative column parallel to the boundary of the sub-area are identified, and these rows and columns represent the edge of the chessboard. By measuring the width of the transition interval between the black and white checkers, it is helpful to evaluate the local clarity of the image; the clarity and unilateral blur of the complete image frame are calculated using the calculated first width and second width, which is helpful to screen out images that are clear enough and suitable for camera calibration.

[0026] Further, step S33 includes:

[0027] S331: Calculating the definition of the complete image frame according to the sum of the first width and the second width;

[0028] S332: Calculate the unilateral blur of the frame in the complete image according to the difference between the first width and the second width.

[0029] Further, step S4 includes:

[0030] S41: Screening out the first complete image frame whose unilateral blurriness is less than a first preset value, and screening out the third complete image frame whose clarity is less than a second preset value;

[0031] S42: Sort the remaining second complete image frames according to definition, wherein the complete image frames also include the third complete image frame.

[0032] Furthermore, step S4 includes:

[0033] S5: Acquire the number of the second complete image frames, and when the number of the second complete image frames does not meet the preset number of images, continue to read the supplementary video frames in the calibration video;

[0034] S6: According to steps S3 to S4, a fourth complete image frame in which the unilateral blurriness in the supplementary video frame is greater than or equal to a first preset value and the clarity is greater than or equal to a second preset value is obtained, so that the sum of the second complete image frame and the fourth complete image frame meets the preset image quantity.

[0035] In a second aspect, the present application provides an image clarity ranking device, the device comprising:

[0036] Image reading module: used to read the video frames in the calibration video;

[0037] Image judgment module: used to judge the integrity of the video frame frame by frame, and obtain a complete image frame according to the integrity;

[0038] Image calculation module: used to calculate the clarity and unilateral blur of each complete image frame;

[0039] Image processing module: used to filter out the first complete image frame corresponding to the unilateral blur less than the first preset value, and sort the remaining second complete image frames according to clarity, the complete image frames including the first complete image frame and the second complete image frame.

[0040] In a third aspect, the present application provides an electronic device, including a processor and a memory, wherein the memory stores computer-readable instructions, and when the computer-readable instructions are executed by the processor, the steps in any of the above methods are executed.

[0041] In a fourth aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the steps in any of the above methods are executed.

[0042] Beneficial effects: The present application proposes an image clarity sorting method, device, electronic device and storage medium, which reads video frames in the calibration video to provide raw materials for subsequent image analysis and processing; determines the integrity of the video frames frame by frame, and obtains complete image frames based on the integrity, which can ensure that the selected complete image frames are suitable for camera calibration, facilitating the subsequent accurate calculation of camera parameters; calculates the clarity and unilateral blur of each complete image frame, and performs quality assessment on the screened complete image frames, wherein the clarity reflects the overall sharpness of the image, and the unilateral blur detects whether the image is blurred in a specific direction; screens out the first complete image frame corresponding to a unilateral blur less than a first preset value, and by setting the first preset value, removes the first complete image frame that is not suitable for precise camera calibration due to camera shake or other factors, which is conducive to the subsequent accurate calculation of camera parameters, and sorts the clarity of the remaining second complete image frames, and preferentially selects images with the highest clarity for camera calibration to improve the accuracy and quality of the calibration results. In the above way, the algorithm can automatically select high-quality checkerboard images from the video with less calculation, less human intervention, lower error rate, and improve the efficiency and accuracy of the entire camera calibration process. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 This is a flow chart of an image clarity ranking method proposed in this application.

[0044] Figure 2 This is a structural diagram of an image clarity sorting device proposed in this application.

[0045] Figure 3 A schematic diagram of the structure of the electronic device provided in this application.

[0046] Figure 4 It is the second complete image frame obtained by the method of the present application.

[0047] Description of reference numerals: 201, image reading module; 202, image determination module; 203, image calculation module; 204, image processing module; 301, processor; 302, memory; 303, communication bus; 3, electronic device. DETAILED DESCRIPTION

[0048] The technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. The components of the embodiments of the present application usually described and marked in the drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the application claimed for protection, but merely represents the selected embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without making creative work belong to the scope of protection of the present application.

[0049] It should be noted that similar reference numerals and letters represent similar items in the following drawings, so once an item is defined in one drawing, it does not need to be further defined and explained in subsequent drawings. At the same time, in the description of this application, the terms "first, second", etc. are only used to distinguish the description and cannot be understood as indicating or implying relative importance.

[0050] The following disclosure provides many different implementations or examples for achieving the purpose of the present invention, which solve the problems of large amount of calculation, low efficiency and accuracy of manual screening when the traditional camera calibration method is applied to multiple cameras.

[0051] Please refer to Figure 1 The present application provides a method for sorting image clarity, the method comprising the steps of:

[0052] S1: read the video frame in the calibration video;

[0053] S2: Determine the integrity of the video frame frame by frame, and obtain a complete image frame based on the integrity;

[0054] S3: Calculate the clarity and unilateral blur of each complete image frame;

[0055] S4: Screening out first complete image frames corresponding to unilateral blurriness less than a first preset value, and sorting the remaining second complete image frames according to clarity, wherein the complete image frames include the first complete image frame and the second complete image frame.

[0056] Before step S1, a calibration video needs to be shot. To shoot the calibration video, a calibration plate needs to be made. The background of the calibration plate is a checkerboard with white and black squares arranged in a cross-spaced pattern, with a total of 9*6=54 corner points. During the calibration video shooting process, it is necessary to ensure that the calibration plate is within the camera range and keep the calibration plate fixed. The handheld camera is used to shoot from various positions and angles of the calibration plate to obtain the calibration video.

[0057] In step S1, video frames in the calibration video can be read during the shooting process, and a counter can be set at the beginning of the reading to track the number of video frames that have been read, so that when the number of clear images that can be used to calculate the camera parameters is insufficient, supplementary video frames can be continued to be read according to the number of video frames tracked by the counter.

[0058] The purpose of step S2 is to identify and select video frames containing complete checkerboard patterns. Since a complete checkerboard is a prerequisite for accurately calculating camera parameters, the integrity of the checkerboard can be judged to ensure that the selected image is suitable for camera calibration.

[0059] The integrity of the video frame refers to the integrity and recognizability of the required features in each frame. In camera calibration and machine vision applications, it usually means that each frame contains sufficient and clear checkerboard corner points, which are used to determine the collective relationship between the features in the image and the surface of the actual object. Specifically, a complete image frame should at least include: the integrity of the corner points, that is, the image frame should contain all the corner points of the checkerboard without missing; the recognizability of the corner points, that is, the corner points should be clearly visible, not blocked or overly blurred, and can be accurately detected by the algorithm; the distribution pattern of the corner points, that is, the corner points should be arranged according to the pattern of the checkerboard without misalignment or deformation; the coverage of the corner points, that is, the image frame should cover a sufficient range of the checkerboard to facilitate the accurate calculation of the camera parameters; consistency, that is, in the image frames taken by multiple cameras and multiple perspectives, the representation of the checkerboard should be consistent for easy comparison and analysis.

[0060] In some preferred embodiments, in order to reduce the amount of calculation and improve the efficiency of acquiring a complete image frame, step S2 includes:

[0061] S21: grayscale processing is performed on the video frames one by one to obtain a grayscale image;

[0062] S23: Sliding the window on the grayscale image to detect corner points and calculate the corner point strength;

[0063] S24: Select a candidate point with the largest intensity and conforming to the checkerboard corner point distribution rule from the corner point intensities, and obtain a video frame corresponding to the candidate point as a complete image frame. The checkerboard corner point distribution rule represents the completeness of the video frame.

[0064] Among them, grayscale processing is performed on the video frames one by one, and the color video frames can be converted into grayscale images, which can reduce the amount of data processed and improve the efficiency of subsequent image processing steps. Grayscale images only contain brightness information, which helps to highlight the outlines and edges of the chessboard in the image more clearly, providing a more accurate basis for corner detection.

[0065] Among them, by sliding a window on a grayscale image, the entire image area can be systematically covered to detect possible corners. The calculation of corner strength is usually based on the gradient information of the image, which helps to identify significant geometric change points in the image, i.e., corners. Among them, algorithms for sliding a window on a grayscale image to detect corners and calculate corner strength include but are not limited to: Harris corner detection, the Harris method evaluates the possibility of each pixel becoming a corner by calculating the gradient and second-order moment of the image, and determines the corner strength by using the brightness change of the pixel in the window; Shi-Tomasi corner detection, Shi-Tomasi is an improved algorithm of the Harris method, which determines the corners by minimizing a specific criterion function, and this method is more robust to noise.

[0066] Among them, the corner points with the highest corner point intensity and distribution conforming to the checkerboard rule are screened out from the detected corner points, which means that the computer will identify the corner points that can form a clear and regular checkerboard pattern. The distribution rules of these corner points (such as regular grid-like arrangement) are used to determine whether the video frame contains a complete checkerboard pattern, thereby judging the integrity of the video frame and obtaining a complete image frame. Through the above steps, a complete image frame containing a complete and clear checkerboard can be effectively extracted from the video frame, providing high-quality input for subsequent camera calibration and image processing tasks. This method can automatically improve the accuracy and efficiency of image screening, reduce the interference of human factors, and reduce the demand for computing resources.

[0067] In some preferred embodiments, in order to be more suitable for calibration of multiple cameras, step S21 includes:

[0068] S211: Split the video frame into a left view and a right view frame by frame;

[0069] S212: Perform grayscale processing on the left view and the right view respectively to obtain grayscale images.

[0070] The video frame can be horizontally split into two independent parts, namely the left view and the right view. This split is to process the stereoscopic vision data of multiple cameras, where the left view and the right view correspond to the images captured by the left and right cameras of the stereo camera system respectively. By processing each view separately, the checkerboard features captured by each camera can be analyzed and identified more accurately, which is beneficial to the calibration and depth estimation of multiple cameras.

[0071] Among them, grayscale processing is performed on the left view and the right view respectively, which can remove color information and reduce the complexity of data processing. Grayscale images enable the algorithm to focus more on the brightness changes in the image, which are usually related to corners and are beneficial to subsequent corner detection. By splitting the video frame and performing grayscale processing, the algorithm prepares for subsequent corner detection and image analysis, ensuring that the images captured by each camera in the stereo vision system can be accurately analyzed, which helps to improve the accuracy and efficiency of camera calibration.

[0072] In step S3, the quality of the selected complete image frames is evaluated, and the quality of each frame is quantitatively analyzed by calculating the clarity and unilateral blur. The clarity reflects the overall sharpness of the image, while the unilateral blur detects whether the image is blurred in a specific direction, which is particularly important for evaluating whether the image is partially blurred due to camera shake.

[0073] Further, step S3 includes:

[0074] S31: Obtain the corner points of each complete image frame, and cut out a k*k image sub-region with the corner point coordinates as the center;

[0075] S32: Filter out first relative rows and first relative columns that are opposite to each other in pairs among four boundaries parallel to the image sub-region, and calculate first widths and second widths of black-white transition intervals in the first relative rows and the first relative columns;

[0076] S33: Calculate the clarity and unilateral blurriness of the complete image frame according to the first width and the second width.

[0077] Among them, k is a constant value set in advance by the technicians according to actual needs.

[0078] Among them, the corner points of each complete image frame are obtained, and the k*k image sub-region is extracted with the corner point as the center. The k*k image sub-region usually contains one or more complete grids of the chessboard, which is helpful for the subsequent calculation of the clarity and unilateral blur of the image. By analyzing the image sub-region, the algorithm can more accurately evaluate the local clarity characteristics of the image.

[0079] In each k*k image sub-region, the first relative row and the first relative column parallel to the boundary of the image sub-region are identified, wherein the first relative row and the first relative column represent the edge of the chessboard, and the first width and the second width of the transition area between the black and white grids on these edges are measured to evaluate the local clarity of the image. Ideally, the first width and the second width should be consistent, indicating that the chessboard in the image is clearly distinguishable, but in actual shooting, the movement or shaking of the camera will cause the first width and the second width to be inconsistent.

[0080] Therefore, the clarity and unilateral blur of the complete image frame can be calculated by the first width and the second width. The clarity can be determined by measuring the average value of the first width and the second width or other statistical methods, and the unilateral blur can be evaluated by analyzing the consistency or variation of the first width and the second width.

[0081] Specifically, step S33 includes:

[0082] S331: Calculate the definition of the complete image frame according to the sum of the first width and the second width;

[0083] S332: Calculate the unilateral blur of the frame in the complete image according to the difference between the first width and the second width.

[0084] The average value of the sum of the first width and the second width is the clarity of the complete image frame, and the average value of the difference between the first width and the second width is the unilateral blur of the complete image frame. The calculation of clarity helps determine whether the complete image frame is clear enough and suitable for camera calibration; the unilateral blur helps identify whether the image is blurred in a specific direction, which is crucial for understanding image quality and improving calibration accuracy.

[0085] In step S4, by setting a first preset value, first complete image frames whose unilateral blur is lower than the first preset value are removed. These images may not be suitable for accurate camera calibration due to camera shake or other factors. The remaining second complete image frames are sorted according to their clarity, and the images with the highest clarity are preferentially selected for camera calibration to improve the accuracy and quality of the calibration results. For specific clear images, please refer to Figure 4 .

[0086] Further, step S4 includes:

[0087] S41: filtering out the first complete image frame whose unilateral blurriness is less than a first preset value, and filtering out the third complete image frame whose clarity is less than a second preset value;

[0088] S42: sorting the remaining second complete image frames according to the definition, where the complete image frames also include the third complete image frame.

[0089] By setting two preset thresholds, namely the first preset value and the second preset value, it is ensured that the selected second complete image frame has neither unilateral blur nor sufficient clarity. The first preset value and the second preset value are set in advance by the technician. By setting the first preset value, the first complete image frame that may be caused by camera shaking or unstable shooting conditions is screened out, and the first complete image frame is not suitable for accurate camera calibration; by setting the second preset value, the third complete image frame with lower clarity is screened out, and the third complete image frame has insufficient clarity and cannot provide sufficient information for accurate camera calibration.

[0090] Among them, after screening the clarity and unilateral blur, the remaining second complete image frames are images with higher quality. The screened second complete image frames are sorted, and the images with higher clarity are placed at the front.

[0091] Through the above steps, the computer can automatically filter and sort the video frames to ensure that the most suitable images for camera calibration are selected, thereby improving the efficiency and quality of the calibration process. This method reduces manual intervention and reduces the risk of calibration errors caused by poor image quality.

[0092] Furthermore, step S4 includes:

[0093] S5: Acquire the number of second complete image frames, and when the number of second complete image frames does not meet the preset number of images, continue to read the supplementary video frames in the calibration video;

[0094] S6: According to steps S3 to S4, a fourth complete image frame having a unilateral blur greater than or equal to a first preset value and a clarity greater than or equal to a second preset value in the supplementary video frame is obtained, so that the sum of the second complete image frame and the fourth complete image frame meets the preset image quantity.

[0095] Among them, by counting the total number of second complete image frames, compare whether the current number of second complete image frames has reached the preset number of images for camera calibration. If the current number does not reach the preset number of images, the algorithm needs to continue to read additional video frames from the calibration video, that is, supplementary video frames, to supplement the number of image frames.

[0096] Among them, in the supplementary read video frames, the screening conditions of steps S3 and S4 are continued to be executed to find the fourth complete image frame whose unilateral blur and clarity meet the first preset value and the second preset value respectively, and the original second complete image frame is merged with the newly screened fourth complete image frame to ensure that the overall number meets the requirement of the preset number of images. Through the above steps, it is ensured that the images used for camera calibration not only meet the requirements in quality, but also meet the calibration needs in quantity, which greatly improves the success rate and accuracy of the calibration process, and reduces retries or failures caused by insufficient image quality or quantity. This process is automated and systematic, reduces the interference of human factors, and improves the efficiency and reliability of the entire camera calibration process.

[0097] As can be seen from the above, the image clarity sorting method provided by the present application provides raw materials for subsequent image analysis and processing by reading the video frames in the calibration video; judging the integrity of the video frames frame by frame, and obtaining the complete image frames according to the integrity, which can ensure that the selected complete image frames are suitable for camera calibration, and facilitate the subsequent accurate calculation of camera parameters; by calculating the clarity and unilateral blur of each complete image frame, the quality of the selected complete image frames is evaluated, the clarity reflects the overall sharpness of the image, and the unilateral blur detects whether the image is blurred in a specific direction; the first complete image frame corresponding to the unilateral blur less than the first preset value is screened out, and by setting the first preset value, the first complete image frame that is not suitable for accurate camera calibration due to camera shake or other factors is removed, which is conducive to the subsequent accurate calculation of camera parameters, and the clarity of the remaining second complete image frames is sorted, and the image with the highest clarity is preferentially selected for camera calibration to improve the accuracy and quality of the calibration results. In the above manner, the algorithm can automatically select high-quality checkerboard images from the video, with less calculation, and reduces human intervention, reduces the error rate, and improves the efficiency and accuracy of the entire camera calibration process.

[0098] Please refer to Figure 2 , the present application proposes an image clarity sorting device, the device comprising:

[0099] Image reading module 201: used to read the video frame in the calibration video;

[0100] Image determination module 202: used to determine the integrity of the video frame frame by frame, and obtain a complete image frame according to the integrity;

[0101] Image calculation module 203: used to calculate the clarity and unilateral blur of each complete image frame;

[0102] The image processing module 204 is used to filter out the first complete image frames corresponding to the unilateral blur less than the first preset value, and sort the remaining second complete image frames according to the clarity, wherein the complete image frames include the first complete image frames and the second complete image frames.

[0103] Among them, the image reading module 201 at least includes a video decoder for extracting video frames in the calibration video; a frame extractor for extracting each frame in the calibration video in sequence; a buffer for temporarily storing each extracted frame; an interface, an interface compatible with the calibration video file format; the image judgment module 202 is a program algorithm running on a computer, at least including an integrity detection algorithm for evaluating the integrity of the chessboard in the video frame; a corner detection algorithm, such as Harris or Shi-Tomasi algorithm, for detecting corners in the image; an integrity assessment algorithm for evaluating the integrity of the chessboard based on the results of corner detection; an image screening algorithm for selecting a complete image frame based on the integrity assessment; the image calculation module 203 is a program algorithm running on a computer, at least including a clarity assessment for calculating the clarity of the complete image frame; a blur assessment for calculating the unilateral blur of the complete image frame; the image processing module 204 is a program running on a computer, at least including blur screening for screening out low-quality first complete image frames according to a first preset value; and clarity sorting for sorting the second complete image frames according to the calculated clarity.

[0104] Before reading the calibration video, you need to shoot the calibration video. To shoot the calibration video, you need to make a calibration plate. The calibration plate background is a chessboard with white and black squares arranged in an interlaced manner, with a total of 9*6=54 corner points. When shooting the calibration video, you need to ensure that the calibration plate is within the camera range and keep it fixed. Use a handheld camera to shoot from various positions and angles of the calibration plate to obtain the calibration video.

[0105] The video frames in the calibration video can be read during the shooting process. When the reading starts, a counter can be set to track the number of video frames that have been read, so that when the number of clear images that can be used to calculate the camera parameters is insufficient, supplementary video frames can be read according to the number of video frames tracked by the counter.

[0106] Among them, since a complete chessboard is a prerequisite for accurately calculating camera parameters, the integrity of the chessboard can be judged to ensure that the selected image is suitable for camera calibration.

[0107] Among them, grayscale processing is performed on the video frames one by one, and the color video frames can be converted into grayscale images, which can reduce the amount of data processed and improve the efficiency of subsequent image processing steps. Grayscale images only contain brightness information, which helps to highlight the outlines and edges of the chessboard in the image more clearly, providing a more accurate basis for corner detection.

[0108] Among them, by sliding a window on a grayscale image, the entire image area can be systematically covered to detect possible corners. The calculation of corner strength is usually based on the gradient information of the image, which helps to identify significant geometric change points in the image, i.e., corners. Among them, algorithms for sliding a window on a grayscale image to detect corners and calculate corner strength include but are not limited to: Harris corner detection, the Harris method evaluates the possibility of each pixel becoming a corner by calculating the gradient and second-order moment of the image, and determines the corner strength by using the brightness change of the pixel in the window; Shi-Tomasi corner detection, Shi-Tomasi is an improved algorithm of the Harris method, which determines the corners by minimizing a specific criterion function, and this method is more robust to noise.

[0109] Among them, the corner points with the highest corner point intensity and distribution conforming to the checkerboard rule are screened out from the detected corner points, which means that the computer will identify the corner points that can form a clear and regular checkerboard pattern. The distribution rules of these corner points (such as regular grid-like arrangement) are used to determine whether the video frame contains a complete checkerboard pattern, thereby judging the integrity of the video frame and obtaining a complete image frame. Through the above steps, a complete image frame containing a complete and clear checkerboard can be effectively extracted from the video frame, providing high-quality input for subsequent camera calibration and image processing tasks. This method can automatically improve the accuracy and efficiency of image screening, reduce the interference of human factors, and reduce the demand for computing resources.

[0110] The video frame can be horizontally split into two independent parts, namely the left view and the right view. This split is to process the stereoscopic vision data of multiple cameras, where the left view and the right view correspond to the images captured by the left and right cameras of the stereo camera system respectively. By processing each view separately, the checkerboard features captured by each camera can be analyzed and identified more accurately, which is beneficial to the calibration and depth estimation of multiple cameras.

[0111] Among them, grayscale processing is performed on the left view and the right view respectively, which can remove color information and reduce the complexity of data processing. Grayscale images enable the algorithm to focus more on the brightness changes in the image, which are usually related to corners and are beneficial to subsequent corner detection. By splitting the video frame and performing grayscale processing, the algorithm prepares for subsequent corner detection and image analysis, ensuring that the images captured by each camera in the stereo vision system can be accurately analyzed, which helps to improve the accuracy and efficiency of camera calibration.

[0112] Among them, the corner points of each complete image frame are obtained, and the k*k image sub-region is extracted with the corner point as the center. The k*k image sub-region usually contains one or more complete grids of the chessboard, which is helpful for the subsequent calculation of the clarity and unilateral blur of the image. By analyzing the image sub-region, the algorithm can more accurately evaluate the local clarity characteristics of the image.

[0113] In each k*k image sub-region, the first relative row and the first relative column parallel to the boundary of the image sub-region are identified, wherein the first relative row and the first relative column represent the edge of the chessboard, and the first width and the second width of the transition area between the black and white grids on these edges are measured to evaluate the local clarity of the image. Ideally, the first width and the second width should be consistent, indicating that the chessboard in the image is clearly distinguishable, but in actual shooting, the movement or shaking of the camera will cause the first width and the second width to be inconsistent.

[0114] Therefore, the clarity and unilateral blur of the complete image frame can be calculated by the first width and the second width. The clarity can be determined by measuring the average value of the first width and the second width or other statistical methods, and the unilateral blur can be evaluated by analyzing the consistency or variation of the first width and the second width.

[0115] The average value of the sum of the first width and the second width is the clarity of the complete image frame, and the average value of the difference between the first width and the second width is the unilateral blur of the complete image frame. The calculation of clarity helps determine whether the complete image frame is clear enough and suitable for camera calibration; the unilateral blur helps identify whether the image is blurred in a specific direction, which is crucial for understanding image quality and improving calibration accuracy.

[0116] By setting two preset thresholds, namely the first preset value and the second preset value, it is ensured that the selected second complete image frame has neither unilateral blur nor sufficient clarity. The first preset value and the second preset value are set in advance by the technician. By setting the first preset value, the first complete image frame that may be caused by camera shaking or unstable shooting conditions is screened out, and the first complete image frame is not suitable for accurate camera calibration; by setting the second preset value, the third complete image frame with lower clarity is screened out, and the third complete image frame has insufficient clarity and cannot provide sufficient information for accurate camera calibration.

[0117] Among them, after screening the clarity and unilateral blur, the remaining second complete image frames are images with higher quality. The screened second complete image frames are sorted, and the images with higher clarity are placed at the front.

[0118] Among them, by counting the total number of second complete image frames, compare whether the current number of second complete image frames has reached the preset number of images for camera calibration. If the current number does not reach the preset number of images, the algorithm needs to continue to read additional video frames from the calibration video, that is, supplementary video frames, to supplement the number of image frames.

[0119] Among them, in the supplementary read video frames, the screening conditions of steps S3 and S4 are continued to be executed to find the fourth complete image frame whose unilateral blur and clarity meet the first preset value and the second preset value respectively, and the original second complete image frame is merged with the newly screened fourth complete image frame to ensure that the overall number meets the requirement of the preset number of images. Through the above steps, it is ensured that the images used for camera calibration not only meet the requirements in quality, but also meet the requirements of calibration composition in quantity, which greatly improves the success rate and accuracy of the calibration process, and reduces retries or failures caused by insufficient image quality or quantity. This process is automated and systematic, reduces the interference of human factors, and improves the efficiency and reliability of the entire camera calibration process.

[0120] As can be seen from the above, the image clarity sorting device provided by the present application provides raw materials for subsequent image analysis and processing by reading the video frames in the calibration video; judging the integrity of the video frames frame by frame, and obtaining the complete image frames according to the integrity, so as to ensure that the selected complete image frames are suitable for camera calibration, and facilitate the subsequent accurate calculation of camera parameters; by calculating the clarity and unilateral blur of each complete image frame, the quality of the selected complete image frames is evaluated, the clarity reflects the overall sharpness of the image, and the unilateral blur detects whether the image is blurred in a specific direction; the first complete image frame corresponding to the unilateral blur less than the first preset value is screened out, and by setting the first preset value, the first complete image frame that is not suitable for accurate camera calibration due to camera shake or other factors is removed, which is conducive to the subsequent accurate calculation of camera parameters, and the clarity of the remaining second complete image frames is sorted, and the image with the highest clarity is preferentially selected for camera calibration to improve the accuracy and quality of the calibration results. In the above manner, the algorithm can automatically select high-quality checkerboard images from the video, with less calculation, and reduces human intervention, reduces the error rate, and improves the efficiency and accuracy of the entire camera calibration process.

[0121] Please refer to Figure 3 , Figure 3A structural schematic diagram of an electronic device provided in an embodiment of the present application, the present application provides an electronic device 3, including: a processor 301 and a memory 302, the processor 301 and the memory 302 are interconnected and communicate with each other through a communication bus 303 and / or other forms of connection mechanisms (not marked), the memory 302 stores computer-readable instructions executable by the processor 301, when the electronic device is running, the processor 301 executes the computer-readable instructions to execute the method in any optional implementation of the above embodiment to achieve the following functions: reading video frames in a calibration video; judging the completeness of the video frames frame by frame, and obtaining complete image frames according to the completeness; calculating the clarity and unilateral blur of each complete image frame; screening out the first complete image frame corresponding to the unilateral blur less than the first preset value, and sorting the remaining second complete image frames according to the clarity, the complete image frames including the first complete image frame and the second complete image frame.

[0122] An embodiment of the present application provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the method in any optional implementation of the above embodiment is executed to achieve the following functions: reading video frames in a calibration video; judging the integrity of the video frames frame by frame, and obtaining complete image frames based on the completeness; calculating the clarity and unilateral blur of each complete image frame; filtering out the first complete image frame corresponding to the unilateral blur less than a first preset value, and sorting the remaining second complete image frames according to the clarity, the complete image frames including the first complete image frame and the second complete image frame.

[0123] In the embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some communication interfaces, and the indirect coupling or communication connection of devices or units can be electrical, mechanical or other forms.

[0124] In addition, the units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0125] Furthermore, the functional modules in the various embodiments of the present application may be integrated together to form an independent part, or each module may exist separately, or two or more modules may be integrated to form an independent part.

[0126] In this document, relational terms such as first and second, etc. are used merely to distinguish one entity or operation from another entity or operation, but do not necessarily require or imply any such actual relationship or order between these entities or operations.

[0127] The above description is only an embodiment of the present application and is not intended to limit the scope of protection of the present application. For those skilled in the art, the present application may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the scope of protection of the present application.

Claims

1. A method for sorting image clarity, characterized in that: The method comprises: S1: read the video frame in the calibration video; S2: determining the integrity of the video frame frame by frame, and obtaining a complete image frame according to the integrity; S3: Calculate the clarity and unilateral blur of each complete image frame; S4: Screening out the first complete image frame corresponding to the unilateral blur less than the first preset value, and sorting the remaining second complete image frames according to the clarity, the complete image frames including the first complete image frame and the second complete image frame; Step S3 includes: S31: Obtain the corner points of each complete image frame, and cut out a k*k image sub-region with the corner point coordinates as the center; S32: Filter out first relative rows and first relative columns that are opposite to each other in pairs among four boundaries parallel to the image sub-region, and calculate first widths and second widths of black-white transition intervals in the first relative rows and the first relative columns; S33: Calculate the clarity and unilateral blurriness of the complete image frame according to the first width and the second width; Step S33 includes: S331: Calculating the definition of the complete image frame according to the sum of the first width and the second width; S332: Calculate the unilateral blur of the frame in the complete image according to the difference between the first width and the second width.

2. The image clarity ranking method according to claim 1, characterized in that: Step S2 includes: S21: performing grayscale processing on the video frames frame by frame to obtain a grayscale image; S23: Sliding a window on the grayscale image to detect corner points, and calculating the corner point strength of the corner points; S24: Select a candidate point with the largest intensity and conforming to the checkerboard corner point distribution rule from the corner point intensities, and obtain the video frame corresponding to the candidate point as a complete image frame, wherein the checkerboard corner point distribution rule represents the completeness of the video frame.

3. The image clarity ranking method according to claim 2, characterized in that: Step S21 includes: S211: Splitting the video frame into a left view and a right view frame by frame; S212: Perform grayscale processing on the left view and the right view respectively to obtain grayscale images.

4. The image definition ranking method according to claim 1, characterized in that: Step S4 includes: S41: Screening out the first complete image frame whose unilateral blurriness is less than a first preset value, and screening out the third complete image frame whose clarity is less than a second preset value; S42: Sort the remaining second complete image frames according to definition, wherein the complete image frames also include the third complete image frame.

5. The image definition ranking method according to claim 4, characterized in that: Step S4 includes: S5: Acquire the number of the second complete image frames, and when the number of the second complete image frames does not meet the preset number of images, continue to read the supplementary video frames in the calibration video; S6: According to steps S3 to S4, a fourth complete image frame in which the unilateral blurriness in the supplementary video frame is greater than or equal to a first preset value and the clarity is greater than or equal to a second preset value is obtained, so that the sum of the second complete image frame and the fourth complete image frame meets the preset image quantity.

6. An image definition sorting device, characterized in that: The device is applied to the steps in any one of the methods according to claims 1 to 5, and the device comprises: Image reading module: used to read the video frames in the calibration video; Image judgment module: used to judge the integrity of the video frame frame by frame, and obtain a complete image frame according to the integrity; Image calculation module: used to calculate the clarity and unilateral blur of each complete image frame; Image processing module: used to filter out the first complete image frame corresponding to the unilateral blur less than the first preset value, and sort the remaining second complete image frames according to clarity, the complete image frames including the first complete image frame and the second complete image frame.

7. An electronic device, characterized in that: The method comprises a processor and a memory, wherein the memory stores computer-readable instructions. When the computer-readable instructions are executed by the processor, the steps in the method according to any one of claims 1 to 5 are executed.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 5 are executed.

Images