Sub-pixel precision flange size measurement method and system based on camera calibration

Abstract

The present invention provides a sub-pixel precision flange size measurement method and system based on camera calibration, wherein the method comprises: step 1: determining the calibration process of a CCD camera when performing sub-pixel precision flange size measurement; step 2: performing calibration assistance according to the calibration process; step 3: acquiring a flange image based on a calibrated CCD camera; step 4: acquiring the flange measurement size according to the flange image. The sub-pixel precision flange size measurement method and system based on camera calibration of the present invention determines the calibration process of a CCD camera when performing sub-pixel precision flange size measurement, and performs calibration assistance considering that the calibration process also involves manual operation and is prone to introduce errors. Acquire a flange image based on a calibrated CCD camera, and acquire the flange measurement size according to the flange image. Reduce the error caused in the calibration process, improve the accuracy of camera calibration, and thereby improve the flange size measurement accuracy.

Description

Technical Field

[0001] The present invention relates to the field of measurement technology, and in particular to a flange size measurement method and system with sub-pixel accuracy based on camera calibration. Background Art

[0002] Automobile flanges are an important part of the automobile suspension system, used to connect the wheel axle to the suspension system. The flanges provide firm support and stable fixation for the wheel axle, ensuring that the wheels can rotate smoothly, and effectively absorbing and transmitting various forces and vibrations generated during vehicle driving. However, during the production process, due to reasons such as materials and processes, the flange size deviation is often too large, resulting in insufficient flange size accuracy. At present, companies mainly rely on manual sampling of inspection tools or measuring tools for flange size detection, which not only has low detection efficiency, but also has a high false detection rate, and cannot fully guarantee product quality.

[0003] The invention patent with application number: CN202410017731.0 discloses a Zernike fitting knife edge shadow map wavefront reconstruction surface shape detection method, device and system, the method comprising: obtaining knife edge shadow maps corresponding to multiple X-axis knife edge coordinates; obtaining knife edge shadow maps corresponding to multiple Y-axis knife edge coordinates; for each knife edge shadow map corresponding to the X-axis / Y-axis knife edge coordinate, locating multiple pixel coordinates corresponding to the knife edge shadow boundary in the knife edge shadow map, using the multiple pixel coordinates as multiple index positions of the X-axis / Y-axis cutting distribution matrix, and writing the values ​​of the X-axis / Y-axis knife edge coordinates as matrix elements into the multiple index positions in the X-axis / Y-axis cutting distribution matrix; applying the Zernike fitting differential phase fitting technology to the X-axis cutting distribution matrix and the Y-axis cutting distribution matrix to obtain the surface shape matrix of the surface shape to be measured; judging whether the surface shape to be measured meets the accuracy requirements according to the surface shape matrix of the surface shape to be measured, thereby solving the problem that the surface shape quality detection based on the knife edge shadow meter in the prior art cannot be quantitatively detected and the surface shape has low universality.

[0004] However, when the above-mentioned prior art locates multiple pixel coordinates corresponding to the knife edge shadow boundary in the knife edge shadow image, camera calibration is required, and the calibration process also involves manual operation (such as: selecting a calibration plate, setting the position of the calibration plate, etc.). If the manual operation is not appropriate, it is easy to introduce errors, thereby affecting the detection accuracy.

[0005] In view of this, there is an urgent need for a flange size measurement method and system with sub-pixel accuracy based on camera calibration to at least solve the above-mentioned deficiencies. Summary of the invention

[0006] One of the purposes of the present invention is to provide a sub-pixel precision flange size measurement method and system based on camera calibration, determine the calibration process of the CCD camera when performing sub-pixel precision flange size measurement, and perform calibration assistance, thereby reducing the error caused by the calibration process, improving the accuracy of camera calibration, and thus improving the flange size measurement accuracy.

[0007] The embodiment of the present invention provides a flange size measurement method with sub-pixel accuracy based on camera calibration, comprising:

[0008] Step 1: Determine the calibration process of the CCD camera when measuring the flange size with sub-pixel accuracy;

[0009] Step 2: Perform calibration assistance according to the calibration process;

[0010] Step 3: Acquire the flange image based on the calibrated CCD camera;

[0011] Step 4: Obtain the flange measurement dimensions based on the flange image.

[0012] Preferably, step 1: determining a calibration process of a CCD camera when performing sub-pixel precision flange size measurement, comprises:

[0013] Obtain calibration specification information, parse calibration specification information, and determine the calibration process;

[0014] and / or,

[0015] Obtain the calibration process that target managers pre-enter into the management platform.

[0016] Preferably, step 2: performing calibration assistance according to the calibration process includes:

[0017] Assist in manually setting the calibration plate;

[0018] When the calibration plate is set up, the collected image is obtained, and the collected image is sent to the staff for selection to obtain the selected image;

[0019] Performing automatic corner point detection according to the selected image, obtaining automatic calibrated corner points of the selected image, marking the corner points, and obtaining a marked image;

[0020] Send the marked image to the staff to confirm the corner correction results;

[0021] Adjust the calibration process parameters according to the corner point correction results;

[0022] Based on the calibration result after adjusting the calibration process parameters, the size of the target size object is measured to obtain the verification measurement result;

[0023] If the verification measurement result is verification passed, the calibration is completed;

[0024] If the verification measurement result is verification failure, the problem is attributed according to the historical calibration process, and the calibration process parameters are adjusted based on the problem attribution result until the verification measurement result is verification passing.

[0025] Preferably, assisting in manually setting the calibration plate includes:

[0026] Get the field of view and CCD camera resolution;

[0027] Determine the calibration plate parameters according to the field of view and the resolution of the CCD camera; the calibration plate parameters include: size information and grid point information;

[0028] According to the calibration plate parameters, determining the parameter matching value of the pre-selected calibration plate;

[0029] Determine a first screening value based on the parameter matching value and associate it with a preselected calibration plate;

[0030] Perform a flatness test on the pre-selected calibration plate to obtain a flatness test result;

[0031] Determine a second screening value based on the flatness test result and associate it with the pre-selected calibration plate;

[0032] The first screening value and the second screening value associated with the pre-selected calibration plate are summed to obtain the target value, and the pre-selected calibration plate with the largest target value is used as the calibration plate for auxiliary manual setting.

[0033] Preferably, the calibration plate parameters are determined according to the field of view and the resolution of the CCD camera, including:

[0034] Obtain multiple manual calibration plate selection records;

[0035] Obtain the subsequent calibration accuracy of the manual calibration plate selection record;

[0036] Determining a target manual calibration plate selection record according to subsequent calibration accuracy, and determining a calibration plate selection model based on the target manual calibration plate selection record;

[0037] Select the model based on the calibration plate, and determine the calibration plate parameters according to the field of view and CCD camera resolution.

[0038] Preferably, adjusting the calibration process parameters according to the corner point correction result includes:

[0039] According to the corner point correction results, the corner point outliers are determined;

[0040] According to the relative positions of the corner outliers in the calibration plate, the associated calibration process parameters are determined;

[0041] Simulating adjustment of calibration process parameters according to the associated calibration process parameters and a preset calibration adjustment model to obtain a simulation adjustment result;

[0042] Determine whether the simulation adjustment results can offset the corner abnormalities;

[0043] If the simulation adjustment result can offset the corner abnormal point, the calibration process parameter corresponding to the simulation adjustment parameter is used as the adjusted calibration process parameter;

[0044] If the simulation adjustment result cannot offset the corner abnormality, determine the supplementary mark on the calibration plate according to the corner abnormality;

[0045] After the supplementary calibration is completed, determine the calibration process parameters.

[0046] Preferably, determining the supplementary mark on the calibration plate according to the corner abnormal point includes:

[0047] Traverse the corner outliers in sequence, and take the corner outliers being traversed as the target corner outliers;

[0048] Determine the calibration grid adjacent to the target corner point outlier on the calibration plate and use it as the target calibration grid;

[0049] Obtaining the first distance lengths between the target corner point outlier and the target calibration grid midpoint, determining the longest first distance length among the first distance lengths, and using it as the second distance length;

[0050] Draw an auxiliary supplementary marking circle with the target corner outlier point as the center and the second distance length as the radius;

[0051] Determine the calibration plate features within the auxiliary supplementary marking circle;

[0052] Based on the artificial auxiliary supplementary marking model and according to the characteristics of the calibration plate, determining the first supplementary marking to be selected;

[0053] Obtain the number of marks of the first supplementary mark to be selected;

[0054] If the number of markers is 1, the first candidate supplementary marker is used as the supplementary marker;

[0055] If the number of markers is greater than 1, the first candidate supplementary marker is used as the second candidate supplementary marker;

[0056] Determine the shortest first distance length among the first distance lengths and use it as the third distance length, and use the calibration grid corresponding to the third distance length as the comparison calibration grid;

[0057] Calculate the image similarity between the second candidate supplementary mark and the comparison calibration grid, and associate it with the corresponding second candidate supplementary mark;

[0058] Calculate the distance between the second supplementary mark to be selected and the target corner point abnormal point and use it as the fourth distance, and associate the fourth distance with the corresponding second supplementary mark to be selected;

[0059] Calculate the distance between the second candidate supplementary mark and the comparison mark grid and use it as the fifth distance, and associate the fifth distance with the corresponding second candidate supplementary mark;

[0060] Inputting the image similarity, the fourth distance length and the fifth distance length associated with the second candidate supplementary mark into an artificially preset index normalization template to obtain a normalized value;

[0061] The second candidate supplementary mark with the largest normalized value is used as the supplementary mark of the target corner point outlier;

[0062] When all the outlier points at each corner are traversed, the determination of the supplementary marks on the calibration plate is completed.

[0063] Preferably, step 4: obtaining the flange measurement dimensions according to the flange image, includes:

[0064] Preprocess the flange image and extract the target flange area image;

[0065] Performing a first edge detection on the target flange area image to obtain first positioning data;

[0066] Perform a second edge detection based on the first positioning data to obtain sub-pixel coordinates of the flange edge;

[0067] The sub-pixel coordinates are fitted based on the least squares fitting algorithm to obtain the flange measurement dimensions.

[0068] The sub-pixel precision flange size measurement system based on camera calibration provided by an embodiment of the present invention includes:

[0069] A calibration process determination subsystem is used to determine the calibration process of the CCD camera when measuring the flange size with sub-pixel accuracy;

[0070] A calibration assistance subsystem is used to assist in calibration according to the calibration process;

[0071] The flange image acquisition subsystem is used to acquire the flange image based on the calibrated CCD camera;

[0072] The flange size measurement subsystem is used to obtain the flange measurement size based on the flange image.

[0073] Preferably, the calibration process determines the subsystem, including:

[0074] The first determination module of the calibration process is used to obtain the calibration specification information, parse the calibration specification information, and determine the calibration process;

[0075] and / or,

[0076] The second determination module of the calibration process is used to obtain the calibration process pre-input into the management platform by the target manager.

[0077] Preferably, the calibration auxiliary subsystem includes:

[0078] The calibration plate setting module is used to assist in manually setting the calibration plate;

[0079] The selection image acquisition module is used to acquire the captured image after the calibration plate is set up, and send the captured image to the staff for selection to acquire the selected image;

[0080] The marked image acquisition module is used to perform automatic corner point detection based on the selected image, obtain the automatically calibrated corner points of the selected image, mark the corner points, and obtain the marked image;

[0081] A corner correction result determination module is used to send the marked image to the staff to determine the corner correction result;

[0082] A calibration process parameter adjustment module is used to adjust the calibration process parameters according to the corner point correction results;

[0083] A verification measurement result acquisition module is used to measure the size of the target size object based on the calibration result after adjusting the calibration process parameters, and obtain the verification measurement result;

[0084] A first verification module, used to complete the calibration if the verification measurement result is verification passed;

[0085] The second verification module is used to attribute the problem according to the historical calibration process if the verification measurement result is verification failure, and adjust the calibration process parameters based on the problem attribution result until the verification measurement result is verification passing

[0086] The beneficial effects of the present invention are:

[0087] The present invention determines the calibration process of a CCD camera when measuring the size of a flange with sub-pixel precision and performs calibration assistance, thereby reducing errors caused during the calibration process, improving the accuracy of camera calibration, and further improving the measurement accuracy of the flange size.

[0088] Other features and advantages of the present invention will be described in the following description, and partly become apparent from the description, or understood by practicing the present invention. The purpose and other advantages of the present invention can be realized and obtained by the structures specifically pointed out in the present application documents.

[0089] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0090] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings:

[0091] Figure 1 Schematic diagram of a flange size measurement method with sub-pixel accuracy based on camera calibration in an embodiment of the present invention;

[0092] Figure 2 Schematic diagram of a flange size measurement system with sub-pixel accuracy based on camera calibration in an embodiment of the present invention. DETAILED DESCRIPTION

[0093] The preferred embodiments of the present invention are described below in conjunction with the accompanying drawings. It should be understood that the preferred embodiments described herein are only used to illustrate and explain the present invention, and are not used to limit the present invention.

[0094] The embodiment of the present invention provides a flange size measurement method with sub-pixel accuracy based on camera calibration, such as Figure 1 As shown, including:

[0095] Step 1: Determine the calibration process of the CCD camera when measuring the size of the flange with sub-pixel precision; the calibration process is the whole process of calibrating the size of the CCD camera, such as setting the calibration plate, photographing the calibration plate, using pycharm software to detect corner points, and determining the size calibration results;

[0096] Step 2: Perform calibration assistance according to the calibration process;

[0097] Step 3: Acquire the flange image based on the calibrated CCD camera;

[0098] Step 4: Obtain the flange measurement size according to the flange image. When obtaining the flange measurement size according to the flange image, the flange image is preprocessed to obtain the target flange area, and the sub-pixel coordinates of the flange edge are obtained based on the target flange area based on the improved Canny-Zernike moment sub-pixel edge detection algorithm, and finally the outer diameter and inner diameter of the flange are obtained by fitting the coordinates.

[0099] The working principle and beneficial effects of the above technical solution are:

[0100] First, the whole process of calibrating the CCD camera size when measuring the flange size with sub-pixel precision is determined. Since the calibration process also involves manual operation and is prone to introduce errors, calibration assistance is performed. The flange image is obtained based on the calibrated CCD camera, and the flange measurement size is obtained based on the flange image.

[0101] This application determines the calibration process of the CCD camera when measuring the flange size with sub-pixel accuracy, and provides calibration assistance, thereby reducing the error caused by the calibration process, improving the accuracy of camera calibration, and thus improving the flange size measurement accuracy.

[0102] In one embodiment, step 1: determining a calibration process of a CCD camera when performing sub-pixel precision flange size measurement, includes:

[0103] Acquire calibration specification information, parse the calibration specification information, and determine the calibration process; wherein the calibration specification information is: information related to the calibration specification for sub-pixel precision flange size measurement, such as: a calibration process standard specification manual;

[0104] and / or,

[0105] Obtain the calibration process pre-entered into the management platform by the target manager. The target manager is the staff who manages the flange size measurement task, and the management platform is the flange size measurement operation specification management system.

[0106] The working principle and beneficial effects of the above technical solution are:

[0107] This application introduces two methods to determine the calibration process: the first is to parse and obtain information related to the calibration specification for sub-pixel precision flange size measurement to determine the calibration process; the second is to obtain the calibration process pre-entered into the flange size measurement operation specification management system by the staff who manages the flange size measurement task. The introduction of two methods to determine the calibration process improves the comprehensiveness of the calibration process acquisition.

[0108] In one embodiment, step 2: performing calibration assistance according to the calibration process includes:

[0109] Assisting manual setting of the calibration plate; wherein, assisting manual setting of the calibration plate is: assisting manual selection of a calibration plate with appropriate calibration plate parameters and setting it at a fixed point in a desired shooting posture;

[0110] When the calibration plate is set, the collected image is obtained, and the collected image is sent to the staff for selection to obtain the selected image; wherein the selected image is: an image that meets the quality requirements selected by the staff from the collected image;

[0111] Performing automatic corner point detection according to the selected image, obtaining the automatic calibration corner points of the selected image, marking the corner points, and obtaining the marked image; wherein the automatic corner point detection is: using pycharm software to write an automatic corner point detection script to perform corner point detection;

[0112] The marked image is sent to the staff to determine the corner point correction result; wherein the corner point correction result is: the result obtained by the staff after correcting the marked corner points in the marked image, such as: marking the wrong marked corner points with a removal mark, and marking the missing marked corner points with a supplement mark;

[0113] According to the corner point correction result, the calibration process parameters are adjusted; wherein the calibration process parameters are: a parameterized representation of the calibration process behavior;

[0114] Based on the calibration result after adjusting the calibration process parameters, the size of the target size object is measured to obtain a verification measurement result; wherein the target size object is: an object of known size for verifying whether the camera calibration is accurate; the verification measurement result is: verification passed and verification failed, the verification passed means: the size measurement result of the target size object measured based on the calibration result is consistent with the known size of the target size object, and the verification failed means: the size measurement result of the target size object measured based on the calibration result is inconsistent with the known size of the target size object;

[0115] If the verification measurement result is verification passed, the calibration is completed;

[0116] If the verification measurement result is verification failure, the problem is attributed according to the historical calibration process, and the current calibration process parameters are adjusted based on the problem attribution result until the verification measurement result is verification passing.

[0117] The working principle and beneficial effects of the above technical solution are:

[0118] This application assists in manually selecting a calibration plate with appropriate calibration plate parameters and sets it at a fixed point at the required shooting posture. After the setting is completed, the calibration plate image (captured image) captured by the CCD camera is obtained, and the captured image is sent to the staff to select a high-quality selected image for subsequent corner point detection. The selected image is first automatically corner-detected to obtain a marked image with the automatically calibrated corner points marked.

[0119] Send the marked image to the staff for verification to determine the corner correction result. According to the corner correction result, adjust the calibration process parameters. According to the calibration result after adjusting the calibration process parameters and the measured image of the target size object, determine the measured size of the target size object (verification measurement result). If the measured size is consistent with the size of the target size object, the calibration process is standardized and accurate; if the verification measurement result is verification failure, attribute the problem according to the historical calibration process, and adjust the calibration process parameters according to the attribution result until the verification measurement result is verification passing. Improve the standardization and accuracy of the calibration process.

[0120] In one embodiment, assisting manual setting of the calibration plate includes:

[0121] Obtain the field of view and the resolution of the CCD camera; the field of view is: the size of the area that the CCD camera used in the system can cover;

[0122] Determine the calibration plate parameters according to the field of view and the resolution of the CCD camera; the calibration plate parameters include: size information and grid point information; the size information includes: size; the grid point information includes: grid point color and grid point layout;

[0123] Determine the parameter matching value of the pre-selected calibration plate according to the calibration plate parameters; wherein the parameter matching value is: a quantitative value of the parameter matching degree between the calibration plate parameters and the pre-selected calibration plate parameters;

[0124] Determine a first screening value according to the parameter matching value and associate it with the preselected calibration plate; wherein the larger the parameter matching value, the larger the corresponding first screening value;

[0125] Performing a flatness test on the pre-selected calibration plate to obtain a flatness test result; wherein the flatness test is to test whether the surface of the pre-selected calibration plate is flat;

[0126] According to the flatness test result, a second screening value is determined and associated with the pre-selected calibration plate; wherein the flatter the flatness test result, the larger the corresponding second screening value;

[0127] The first screening value and the second screening value associated with the pre-selected calibration plate are summed to obtain the target value, and the pre-selected calibration plate with the largest target value is used as the calibration plate for auxiliary manual setting.

[0128] The working principle and beneficial effects of the above technical solution are:

[0129] This application determines the calibration plate parameters according to the acquired field of view range and CCD camera resolution. After the calibration plate parameters are formulated, the calibration plate parameters and the relevant parameters of the pre-selected calibration plate are matched to determine the parameter matching value. The higher the parameter matching value, the higher the corresponding first screening value. The first screening value is associated with the pre-selected calibration plate. The flatness detection of the calibration plate is introduced, and the second screening value is determined according to the flatness detection result. The flatter the flatness detection result, the larger the second screening value, and the second screening value is associated with the pre-selected calibration plate. The first screening value and the second screening value associated with the pre-selected calibration plate are summed to obtain the target value, and the pre-selected calibration plate with the largest target value is used as the calibration plate for auxiliary manual setting. The rationality of calibration plate selection is improved.

[0130] In one embodiment, the calibration plate parameters are determined according to the field of view and the resolution of the CCD camera, including:

[0131] Acquire multiple manual calibration plate selection records; wherein the manual calibration plate selection record is: a process record of manually determining which calibration plate to use based on historical calibration plate selection requirements (for example, which field of view range, which CCD camera resolution calibration situation);

[0132] Obtaining the subsequent calibration accuracy of the manual calibration plate selection record; wherein the subsequent calibration accuracy is: the accuracy of the calibration plate selected in the manual calibration plate selection record when used for subsequent calibration;

[0133] Determine a target manual calibration plate selection record according to a subsequent calibration accuracy, and determine a calibration plate selection model based on the target manual calibration plate selection record; wherein the target manual calibration plate selection record is: a manual calibration plate selection record with a subsequent calibration accuracy greater than or equal to 0.95; the calibration plate selection model is: an AI model for determining a calibration plate selection strategy according to a field of view range and a CCD camera resolution;

[0134] Select the model based on the calibration plate, and determine the calibration plate parameters according to the field of view and CCD camera resolution.

[0135] The working principle and beneficial effects of the above technical solution are:

[0136] When determining the calibration plate parameters, the present application introduces multiple manual calibration plate selection records, obtains the subsequent calibration accuracy of the manual calibration plate selection records, and determines the target manual calibration plate selection records whose subsequent calibration accuracy is greater than or equal to the preset value. The calibration plate selection model is trained based on the target manual calibration plate selection records. The field of view range and CCD camera resolution are input into the calibration plate selection model to obtain the calibration plate parameters, thereby improving the suitability of the calibration plate parameter formulation.

[0137] In one embodiment, adjusting calibration process parameters according to the corner point correction result includes:

[0138] According to the corner point correction result, the corner point abnormal point is determined; wherein the corner point abnormal point is: the corner point with an incorrect annotation among the manually determined automatically annotated corner points;

[0139] Determine the associated calibration process parameters according to the relative positions of the corner point abnormal points in the calibration plate; wherein the associated calibration process parameters are: parameters that need to be adjusted during the calibration process determined according to the relative positions of the corner point abnormal points in the calibration plate;

[0140] Simulating the adjustment of the calibration process parameters according to the associated calibration process parameters and the preset calibration adjustment model to obtain the simulated adjustment result; wherein the preset calibration adjustment model is: a simulation model for adjusting the camera calibration process; constructing the calibration adjustment model according to the current calibration situation, and then simulating the adjustment of the calibration process parameters according to the associated calibration process parameters and the corresponding model parameters to obtain the simulated adjustment result;

[0141] Determine whether the simulation adjustment results can offset the corner abnormalities;

[0142] If the simulation adjustment result can offset the corner abnormal point, the calibration process parameter corresponding to the simulation adjustment parameter is used as the adjusted calibration process parameter;

[0143] If the simulation adjustment result cannot offset the corner abnormal point, determine the supplementary mark on the calibration plate according to the corner abnormal point; if the simulation adjustment cannot offset the corner abnormal point, it is necessary to add additional marks (i.e., supplementary marks, such as augmented reality QR codes) on the calibration plate to provide more information to help identify the corner point;

[0144] After the supplementary calibration is completed, determine the calibration process parameters.

[0145] The working principle and beneficial effects of the above technical solution are:

[0146] The present application analyzes the corner point correction results to determine the corner point outliers. According to the relative positions of the corner point outliers in the calibration plate, the associated calibration process parameters that need to be adjusted during the calibration process are determined. A calibration adjustment model is introduced, and the calibration process parameters are simulated and adjusted according to the associated calibration process parameters to obtain the simulation adjustment results. If there is a simulation verification measurement result in the simulation adjustment result that does not correspond to the simulated corner point outliers, the adjusted calibration process parameters are determined according to the corresponding simulation adjustment parameters. If the simulation adjustment result cannot offset the corner point outliers, the supplementary marks on the calibration plate are determined according to the corner point outliers, and the supplementary mark behavior is used as the calibration process parameter, thereby improving the suitability of the calibration process adjustment.

[0147] In one embodiment, determining the supplementary mark on the calibration plate according to the corner abnormal point includes:

[0148] Traverse the corner outliers in sequence, and take the corner outliers being traversed as the target corner outliers;

[0149] Determine the calibration grid adjacent to the target corner point outlier on the calibration plate and use it as the target calibration grid;

[0150] Obtain the first distance length between the target corner point outlier and the midpoint of the target calibration grid, determine the longest first distance length among the first distance lengths, and use it as the second distance length; wherein the first distance length is: the distance length between the target corner point outlier and the midpoint of the target calibration grid;

[0151] Draw an auxiliary supplementary marking circle with the target corner outlier point as the center and the second distance length as the radius;

[0152] Determine the calibration plate features within the auxiliary supplementary mark circle; wherein the calibration plate features are: regional feature representations of the calibration plate corresponding to the area within the auxiliary supplementary mark circle, such as: calibration plate color, calibration grid distribution, etc.;

[0153] Based on the manual auxiliary supplementary marking model, the first candidate supplementary marking is determined according to the features of the calibration plate; wherein the manual auxiliary supplementary marking model is an AI model that manually determines the content of the supplementary marking according to the features of the calibration plate around the corner abnormal point. Since the form and position of the supplementary marking can be various, the model may output multiple output supplementary markings, namely, the first candidate supplementary marking;

[0154] Obtaining the number of marks of the first supplementary mark to be selected; wherein the number of marks is: the number of the first supplementary mark to be selected;

[0155] If the number of markers is 1, the first candidate supplementary marker is used as the supplementary marker;

[0156] If the number of markers is greater than 1, the first candidate supplementary marker is used as the second candidate supplementary marker;

[0157] Determine the shortest first distance length among the first distance lengths and use it as the third distance length, and use the calibration grid corresponding to the third distance length as the comparison calibration grid;

[0158] Calculate the image similarity between the second candidate supplementary mark and the comparison calibration grid, and associate with the corresponding second candidate supplementary mark; wherein the image similarity is: the image feature similarity between the second candidate supplementary mark and the comparison calibration grid;

[0159] Calculate the distance between the second supplementary mark to be selected and the target corner point abnormal point and use it as the fourth distance, and associate the fourth distance with the corresponding second supplementary mark to be selected;

[0160] Calculate the distance between the second candidate supplementary mark and the comparison mark grid and use it as the fifth distance, and associate the fifth distance with the corresponding second candidate supplementary mark;

[0161] Inputting the image similarity, the fourth distance length and the fifth distance length associated with the second candidate supplementary mark into an artificially preset index normalization template to obtain a normalized value; wherein the normalization template normalizes the image similarity, the fourth distance length and the fifth distance length according to different normalization rules, wherein the longer the fifth distance length is, the higher the normalized value is; the higher the image similarity is and the shorter the fourth distance length is, the lower the normalized value is;

[0162] The second candidate supplementary mark with the largest normalized value is used as the supplementary mark of the target corner point outlier;

[0163] When all the outlier points at each corner are traversed, the determination of the supplementary marks on the calibration plate is completed.

[0164] The working principle and beneficial effects of the above technical solution are:

[0165] When there are corner outliers during calibration, special calibration information (supplementary marks) can be introduced for auxiliary calibration. When determining the supplementary marks, it is necessary to consider the calibration grids around the target corner outliers, and coordinate the supplementary marks according to the surrounding calibration grids. Therefore, the calibration grid adjacent to the target corner outlier on the calibration plate is determined and used as the target calibration grid. With the target corner outlier as the center and the second distance between the target corner outlier and the target calibration grid as the radius, an auxiliary supplementary mark circle is drawn. Determine the calibration plate features within the auxiliary supplementary mark circle, and determine the first candidate supplementary mark based on the introduced artificial auxiliary supplementary mark model. When there is only one first candidate supplementary mark, it is directly used as a supplementary mark. When the number of marks is greater than 1, it is necessary to select the most suitable supplementary mark among the second candidate supplementary marks.

[0166] When selecting, determine the comparison calibration grid closest to the target corner outlier. Calculate the image similarity between the second candidate supplementary mark and the comparison calibration grid. The smaller the image similarity, the greater the difference between the corresponding second candidate supplementary mark and the comparison calibration grid, and the easier it is to identify the target corner outlier. In addition, the fourth distance length between the second candidate supplementary mark and the target corner outlier should be as close as possible, and the fifth distance length between the second candidate supplementary mark and the comparison calibration grid should be as far as possible, so that the auxiliary comparison calibration grid provides more calibration information. Introduce an indicator normalization template to determine the normalization value, and use the second candidate supplementary mark with the largest normalization value as the supplementary mark for the target corner outlier, thereby improving the suitability of the supplementary mark.

[0167] In one embodiment, step 4: obtaining flange measurement dimensions according to the flange image, includes:

[0168] Preprocessing the flange image and extracting the target flange area image; wherein the target flange area image is: after image processing (such as grayscale conversion, filtering and noise reduction, and binary segmentation) is performed on the flange image, a local image containing the flange is determined;

[0169] Performing a first edge detection on the target flange area image to obtain first positioning data; wherein the first edge detection is: performing edge detection on the image based on an improved Canny adaptive threshold selection algorithm; the first positioning data is: a rough positioning result of an improved Canny operator;

[0170] Performing a second edge detection according to the first positioning data to obtain the sub-pixel coordinates of the flange edge; wherein the second edge detection is: performing edge detection on the image based on an improved Zernike moment sub-pixel edge detection algorithm;

[0171] The sub-pixel coordinates are fitted based on the least squares fitting algorithm to obtain the flange measurement dimensions.

[0172] The working principle and beneficial effects of the above technical solution are:

[0173] This application first uses grayscale conversion, filtering and noise reduction, and binarization segmentation for preprocessing and extracts the target flange area at the same time, uses the improved Canny adaptive threshold selection algorithm to detect the edge of the image, and proposes a new edge judgment condition based on the Zernike algorithm. Then, the improved Zernike moment sub-pixel edge detection algorithm is used to obtain the sub-pixel level coordinates of the flange edge based on the improved Canny operator coarse positioning, and finally uses the least squares fitting algorithm to obtain the outer circle diameter and inner hole diameter of the flange;

[0174] As shown in Tables 1 to 6, the improved algorithm is compared with the traditional algorithm and the manual measurement value. The improved algorithm has a lower relative error value, and the actual value obtained is closer to the manual measurement value. The measurement accuracy is significantly higher than the traditional Zern i ke moment algorithm. The achieved measurement accuracy can meet the accuracy requirements in the production process of industrial parts.

[0175]

[0176] Table 1 Flange outer diameter measurement results

[0177]

[0178] Table 2 Measurement results of flange center hole diameter

[0179]

[0180] Table 3 Flange inner hole 1 diameter measurement results

[0181]

[0182] Table 4 Measurement results of flange inner hole 2 diameter

[0183]

[0184] Table 5 Measurement results of flange inner hole 3 diameter

[0185]

[0186] Table 6 Measurement results of flange inner hole 4 diameter

[0187] This application introduces an improved Canny adaptive threshold selection algorithm and an improved Zernike moment sub-pixel edge detection algorithm to determine the sub-pixel level coordinates, and fits the sub-pixel level coordinates to obtain the flange measurement size, which greatly improves the measurement accuracy of the flange size.

[0188] The embodiment of the present invention provides a flange size measurement system with sub-pixel accuracy based on camera calibration, such as Figure 2 As shown, including:

[0189] The calibration process determination subsystem 1 is used to determine the calibration process of the CCD camera when performing sub-pixel precision flange size measurement;

[0190] The calibration assistance subsystem 2 is used to assist in calibration according to the calibration process;

[0191] The flange image acquisition subsystem 3 is used to acquire the flange image based on the calibrated CCD camera;

[0192] The flange size measurement subsystem 4 is used to obtain the flange measurement size according to the flange image.

[0193] In one embodiment, the calibration process determines the subsystem, including:

[0194] The first determination module of the calibration process is used to obtain the calibration specification information, parse the calibration specification information, and determine the calibration process;

[0195] and / or,

[0196] The second determination module of the calibration process is used to obtain the calibration process pre-input into the management platform by the target manager.

[0197] In one embodiment, the calibration assistance subsystem includes:

[0198] The calibration plate setting module is used to assist in manually setting the calibration plate;

[0199] The selection image acquisition module is used to acquire the captured image after the calibration plate is set up, and send the captured image to the staff for selection to acquire the selected image;

[0200] The marked image acquisition module is used to perform automatic corner point detection based on the selected image, obtain the automatically calibrated corner points of the selected image, mark the corner points, and obtain the marked image;

[0201] A corner correction result determination module is used to send the marked image to the staff to determine the corner correction result;

[0202] A calibration process parameter adjustment module is used to adjust the calibration process parameters according to the corner point correction results;

[0203] A verification measurement result acquisition module is used to measure the size of the target size object based on the calibration result after adjusting the calibration process parameters, and obtain the verification measurement result;

[0204] A first verification module, used to complete the calibration if the verification measurement result is verification passed;

[0205] The second verification module is used to attribute the problem according to the historical calibration process if the verification measurement result is verification failure, and adjust the calibration process parameters based on the problem attribution result until the verification measurement result is verification success.

[0206] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. A sub-pixel precision flange size measurement method based on camera calibration, characterized in that: include: Step 1: Determine the calibration process of the CCD camera when measuring the flange size with sub-pixel accuracy; Step 2: Perform calibration assistance according to the calibration process; Step 3: Acquire the flange image based on the calibrated CCD camera; Step 4: Obtain the flange measurement dimensions based on the flange image; Step 2: Perform calibration assistance according to the calibration process, including: Assist in manually setting the calibration plate; When the calibration plate is set up, the collected image is obtained, and the collected image is sent to the staff for selection to obtain the selected image; Performing automatic corner point detection according to the selected image, obtaining automatic calibrated corner points of the selected image, marking the corner points, and obtaining a marked image; Send the marked image to the staff to confirm the corner correction results; Adjust the calibration process parameters according to the corner point correction results; Based on the calibration result after adjusting the calibration process parameters, the size of the target size object is measured to obtain the verification measurement result; If the verification measurement result is verification passed, the calibration is completed; If the verification measurement result is verification failure, the problem is attributed according to the historical calibration process, and the calibration process parameters are adjusted based on the problem attribution result until the verification measurement result is verification passing; Assists in manually setting the calibration plate, including: Get the field of view and CCD camera resolution; Determine the calibration plate parameters according to the field of view and the resolution of the CCD camera; the calibration plate parameters include: size information and grid point information; According to the calibration plate parameters, determining the parameter matching value of the pre-selected calibration plate; Determine a first screening value based on the parameter matching value and associate it with a preselected calibration plate; Perform a flatness test on the pre-selected calibration plate to obtain a flatness test result; Determine a second screening value based on the flatness test result and associate it with the pre-selected calibration plate; The first screening value and the second screening value associated with the pre-selected calibration plate are summed to obtain the target value, and the pre-selected calibration plate with the largest target value is used as the calibration plate for auxiliary manual setting; Determine the calibration board parameters based on the field of view and CCD camera resolution, including: Obtain multiple manual calibration plate selection records; Obtain the subsequent calibration accuracy of the manual calibration plate selection record; Determining a target manual calibration plate selection record according to subsequent calibration accuracy, and determining a calibration plate selection model based on the target manual calibration plate selection record; Select the model based on the calibration plate, and determine the calibration plate parameters according to the field of view and CCD camera resolution.

2. The method for measuring flange size with sub-pixel accuracy based on camera calibration according to claim 1, characterized in that: Step 1: Determine the calibration process of the CCD camera for sub-pixel precision flange size measurement, including: Obtain calibration specification information, parse calibration specification information, and determine the calibration process; and / or, Obtain the calibration process that target managers pre-enter into the management platform.

3. The method for measuring flange size with sub-pixel accuracy based on camera calibration as claimed in claim 1, characterized in that: According to the corner point correction results, adjust the calibration process parameters, including: According to the corner point correction results, the corner point outliers are determined; According to the relative positions of the corner outliers in the calibration plate, the associated calibration process parameters are determined; Simulating adjustment of calibration process parameters according to the associated calibration process parameters and a preset calibration adjustment model to obtain a simulation adjustment result; Determine whether the simulation adjustment results can offset the corner abnormalities; If the simulation adjustment result can offset the corner abnormal point, the calibration process parameter corresponding to the simulation adjustment parameter is used as the adjusted calibration process parameter; If the simulation adjustment result cannot offset the corner abnormality, determine the supplementary mark on the calibration plate according to the corner abnormality; After the supplementary calibration is completed, determine the calibration process parameters.

4. The method for measuring flange size with sub-pixel accuracy based on camera calibration as claimed in claim 1, characterized in that: Step 4: Obtain flange measurement dimensions based on the flange image, including: Preprocess the flange image and extract the target flange area image; Performing a first edge detection on the target flange area image to obtain first positioning data; Perform a second edge detection based on the first positioning data to obtain sub-pixel coordinates of the flange edge; The sub-pixel coordinates are fitted based on the least squares fitting algorithm to obtain the flange measurement dimensions.

5. Sub-pixel precision flange size measurement system based on camera calibration, characterized in that: include: A calibration process determination subsystem is used to determine the calibration process of the CCD camera when measuring the flange size with sub-pixel accuracy; A calibration assistance subsystem is used to assist in calibration according to the calibration process; The flange image acquisition subsystem is used to acquire the flange image based on the calibrated CCD camera; The flange size measurement subsystem is used to obtain the flange measurement size based on the flange image; The calibration assistance subsystem performs calibration assistance according to the calibration process and performs the following operations: Assist in manually setting the calibration plate; When the calibration plate is set up, the collected image is obtained, and the collected image is sent to the staff for selection to obtain the selected image; Performing automatic corner point detection according to the selected image, obtaining automatic calibrated corner points of the selected image, marking the corner points, and obtaining a marked image; Send the marked image to the staff to confirm the corner correction results; Adjust the calibration process parameters according to the corner point correction results; Based on the calibration result after adjusting the calibration process parameters, the size of the target size object is measured to obtain the verification measurement result; If the verification measurement result is verification passed, the calibration is completed; If the verification measurement result is verification failure, the problem is attributed according to the historical calibration process, and the calibration process parameters are adjusted based on the problem attribution result until the verification measurement result is verification passing; Assists in manually setting the calibration plate, including: Get the field of view and CCD camera resolution; Determine the calibration plate parameters according to the field of view and the resolution of the CCD camera; the calibration plate parameters include: size information and grid point information; According to the calibration plate parameters, determining the parameter matching value of the pre-selected calibration plate; Determine a first screening value based on the parameter matching value and associate it with a preselected calibration plate; Perform a flatness test on the pre-selected calibration plate to obtain a flatness test result; Determine a second screening value based on the flatness test result and associate it with the pre-selected calibration plate; The first screening value and the second screening value associated with the pre-selected calibration plate are summed to obtain the target value, and the pre-selected calibration plate with the largest target value is used as the calibration plate for auxiliary manual setting; Determine the calibration board parameters based on the field of view and CCD camera resolution, including: Obtain multiple manual calibration plate selection records; Obtain the subsequent calibration accuracy of the manual calibration plate selection record; Determining a target manual calibration plate selection record according to subsequent calibration accuracy, and determining a calibration plate selection model based on the target manual calibration plate selection record; Select the model based on the calibration plate, and determine the calibration plate parameters according to the field of view and CCD camera resolution.

6. The sub-pixel precision flange size measurement system based on camera calibration as claimed in claim 5, characterized in that: The calibration process determines the subsystems, including: The first determination module of the calibration process is used to obtain the calibration specification information, parse the calibration specification information, and determine the calibration process; and / or, The second determination module of the calibration process is used to obtain the calibration process pre-input into the management platform by the target manager.

Images