Method and system for automatic optical measurement of nut size and geometric tolerance

Abstract

This invention relates to the field of nut measurement technology, and particularly to an automated optical measurement method and system for nut dimensions and geometric tolerances. The method includes: real-time analysis of each continuously acquired image frame during the measurement process; extracting image feature parameters from each image frame to construct an image frame feature vector; acquiring optical measurement parameters corresponding to each image frame; and performing multi-dimensional correlation between the optical measurement parameters and the image frame feature vector to form an optical-image joint feature vector. This invention employs a comprehensive strategy based on optical measurement parameters, image features, temporal analysis, and abnormal frame correction, solving the problem that traditional optical measurement systems cannot dynamically adapt to complex changes and abnormal data. Through this method, the system can monitor, identify, and correct abnormal data in real time in a dynamic measurement environment, thereby significantly improving the accuracy of dimensional assessment and geometric tolerance judgment, and ensuring the reliability of measurement results.

Description

Technical Field

[0001] This invention relates to the field of nut measurement technology, specifically to an automated optical measurement method and system for nut dimensions and geometric tolerances. Background Technology

[0002] Currently, automated optical measurement systems typically rely on time-series modeling methods to process continuous image data, especially during dynamic measurements. Trend analysis of image sequences is used to determine geometric changes in components. While this method effectively improves measurement efficiency, it can encounter anomalous frames in certain situations. This is particularly true when signal loss, sensor errors, or external interference cause abnormal data to fail to be identified in a timely manner. These anomalous frames often affect the overall trend judgment, leading to inaccurate measurement results. Without suitable algorithms to identify and correct these anomalous frames, the system may incorrectly treat them as the dominant data for the measurement trend, thus affecting the final dimensional evaluation and geometric tolerance determination.

[0003] Traditional measurement systems typically rely on simple data filtering or error correction methods, which cannot cope with data mismatch problems in complex and dynamic changes. In particular, the system fails to dynamically adjust according to the actual situation during the measurement process, resulting in unnecessary errors in the measurement results. Summary of the Invention

[0004] To address the aforementioned issues, this invention provides an automated optical measurement method and system for nut dimensions and geometric tolerances. By performing image feature extraction, optical measurement parameter correlation, temporal similarity analysis, abnormal image frame determination, and dynamic correction of abnormal image frames on continuously acquired nut images, an abnormal image frame identification and correction processing chain for continuous measurement is formed. This incorporates changes in imaging state, image structure, and temporal continuity into the measurement determination process, reducing error propagation caused by abnormal image frames directly participating in dimensional parameter extraction and geometric tolerance calculation.

[0005] This invention employs the following technical solution: an automated optical measurement method for nut dimensions and geometric tolerances, comprising:

[0006] During the measurement process, each continuously acquired image frame is analyzed in real time, and image frame feature vectors are constructed by extracting the image feature parameters of each image frame.

[0007] The optical measurement parameters corresponding to each frame of the image are collected, and the optical measurement parameters are correlated with the image frame feature vector in multiple dimensions to form an optical-image joint feature vector.

[0008] By using temporal correlation analysis, a temporal dependency graph is established, and the similarity measure between each frame and its preceding and following neighboring frames is calculated through the correlation matrix between image frames to obtain the total similarity measure of each frame.

[0009] The optical-image joint feature vector and total similarity metric of each frame are input into the pre-built abnormal frame prediction model, and the result of determining whether each frame is a normal image frame or an abnormal image frame is output.

[0010] For abnormal image frames, identify the abnormal type of the image frame, and perform dynamic correction based on the abnormal type of the image frame and information from adjacent frames. Based on the corrected normal image frames, complete the extraction of the nut's dimensional parameters and the calculation of its geometric tolerances.

[0011] As a further description of the above technical solution: the image feature parameters include the mean pixel distribution, grayscale histogram vector, edge contour change rate, and local texture consistency coefficient of each frame image.

[0012] As a further description of the above technical solution: the method for obtaining the edge contour change rate includes:

[0013] Edge detection is performed on the preprocessed image to extract the thread edge pixels. The edge pixels are then located according to a preset coordinate reference. In two consecutively acquired adjacent frames, the positions of the corresponding matching edge points are used to calculate the spatial offset of each corresponding edge point. At the same time, the rate of change of the number of pixels that constitute the thread edge in the current frame is counted. The average edge offset and the rate of change of the number are weighted and summed to obtain the edge contour change rate that represents the degree of change of the thread contour over time.

[0014] As a further description of the above technical solution: the method for obtaining the local texture consistency coefficient includes:

[0015] Perform edge detection on the current image frame to obtain a set of thread contour edge points. With each edge point as the center, construct a neighborhood region of a preset width along the contour normal direction.

[0016] Within the neighborhood area, it is divided into K local analysis windows according to a preset size. Each local analysis window is used to independently extract texture features and reflect the imaging stability state within the area.

[0017] For each local analysis window, the gradient direction angle of the pixels within the window is calculated, and the window texture consistency coefficient is calculated based on the degree of distribution concentration of the gradient direction angle within the local analysis window.

[0018] The local texture consistency coefficient is obtained by summing the window texture consistency coefficients obtained from each analysis window and dividing by the number of windows.

[0019] As a further description of the above technical solution: the optical measurement parameters include light intensity distribution characteristic parameters, exposure time, and shutter speed.

[0020] As a further description of the above technical solution: the method for obtaining the total similarity metric for each frame includes:

[0021] Each frame of image data is connected to the adjacent frames before and after it by establishing an edge connection to form a graph structure, where the nodes of the graph represent image frames, and the edges represent the temporal dependencies between image frames.

[0022] For each pair of adjacent frames, calculate the similarity measure of their image frame feature vectors to obtain an element of the similarity matrix;

[0023] For each frame of image, calculate its similarity measure with the preceding and following adjacent frames to form a similarity matrix, and then calculate the total similarity measure of the frame by taking the average value.

[0024] As a further description of the above technical solution: the method for forming the optical-image joint feature vector includes:

[0025] When acquiring each frame of image, the corresponding optical measurement parameters are also obtained, including the light intensity distribution characteristics of the entire imaging area, the exposure time and shutter speed of each frame of image are recorded.

[0026] The optical measurement parameters are normalized. According to the time frame correspondence, the normalized optical measurement parameters are concatenated and combined with the image frame feature vector of the corresponding frame image to form the optical-image joint feature vector corresponding to the frame image.

[0027] As a further description of the above technical solution: the method for obtaining the image frame anomaly type includes:

[0028] The optical-image joint feature vector of frames identified as anomalous is separated and analyzed to obtain the optical measurement parameter vector and the image feature vector, respectively.

[0029] Calculate the deviation of the optical measurement parameters of this frame from the mean and variance of the normal frames before and after it. If the deviation exceeds the set threshold, mark the optical anomalous component.

[0030] Based on the image feature vector, the degree of change of the abnormal frame image feature vector relative to the structural features of the preceding and following normal frames is calculated. When the degree of change exceeds the corresponding image structural anomaly determination threshold, the corresponding image feature is marked as an image abnormal component.

[0031] Based on the labeling results of optical and image anomaly components, the anomaly type of the abnormal frame is determined to obtain the image frame anomaly type.

[0032] As a further description of the above technical solution: the method for determining the anomaly type of an image frame based on the labeling results of optical anomaly components and image anomaly components includes:

[0033] When only optical anomalous components exist, it is determined to be an optical anomalous frame; when only image anomalous components exist, it is determined to be an image anomalous frame; when both optical and image anomalous components exist, it is determined to be an optical-image composite anomalous frame.

[0034] As a further description of the above technical solution: the method for dynamic correction based on image frame anomaly type combined with adjacent frame information includes:

[0035] For abnormal frames identified as optically abnormal, a reference mapping for light intensity distribution feature parameters is constructed based on the light intensity distribution feature parameters of consecutive normal frames. The light intensity distribution feature parameters of the abnormal frames are mapped to the stable parameter range corresponding to the reference mapping. On this basis, light intensity redistribution and grayscale mapping correction are performed on the abnormal frame image to restore imaging consistency without changing the geometric structure.

[0036] As a further description of the above technical solution: the method for dynamic correction based on image frame anomaly type combined with adjacent frame information also includes:

[0037] For abnormal frames identified as image anomalies, the pixel coordinate information of the corresponding regions in the preceding and following normal frames is extracted. Based on the coordinate information, the mean position of each pixel is calculated, and a set of mean coordinate points is constructed. The pixel coordinates of the preceding and following frames are connected with the mean coordinate points to form a continuous pixel trajectory. In the abnormal frame, the pixel coordinates of the corresponding region are obtained. For pixels located on the trajectory, no correction is made. For pixels not located on the trajectory, the shortest distance point from the pixel to the trajectory is calculated, and the coordinates of the shortest distance point are used as the corrected pixel position to restore the temporal and spatial continuity of the pixels.

[0038] As a further description of the above technical solution: the method for dynamic correction based on image frame anomaly type combined with adjacent frame information also includes:

[0039] For abnormal frames identified as optical-image composite anomalies, optical correction is performed first, followed by image anomaly correction.

[0040] The corrected abnormal frames are regenerated into optical-image joint feature vectors, and the total similarity metric is obtained and re-input into the abnormal frame prediction model for verification. If the model determines that it is normal, it can be used. If the model still determines that it is abnormal, iterative correction can be performed with a preset number of iterations. If the model still determines that it is abnormal after the iteration is completed, it is marked as an unrecoverable abnormal frame.

[0041] An automated optical measurement system for nut dimensions and geometric tolerances, used to implement the aforementioned automated optical measurement method for nut dimensions and geometric tolerances, the system comprising:

[0042] The image feature extraction module performs real-time analysis on each continuously acquired image frame during the measurement process, and constructs an image frame feature vector by extracting the image feature parameters of each image frame.

[0043] The optical measurement parameter acquisition module acquires the optical measurement parameters corresponding to each frame of the image.

[0044] The optical-image joint module performs multi-dimensional correlation between optical measurement parameters and image frame feature vectors to form an optical-image joint feature vector.

[0045] The temporal correlation analysis module utilizes temporal correlation analysis to establish a temporal dependency graph. It calculates the similarity measure between each frame and its preceding and following neighboring frames using a correlation matrix, thereby obtaining the total similarity measure for each frame.

[0046] The abnormal frame determination module inputs the optical-image joint feature vector and total similarity measure of each frame into the pre-built abnormal frame prediction model, and outputs the determination result of normal or abnormal image frames for each frame.

[0047] The abnormal image correction module identifies the abnormal type of the image frame, performs dynamic correction based on the abnormal type and information from adjacent frames, and extracts the nut's dimensional parameters and calculates its geometric tolerances based on the corrected normal image frame.

[0048] The beneficial effects of this invention are as follows:

[0049] The automated optical measurement method for nut dimensions and geometric tolerances provided by this invention employs a comprehensive strategy based on optical measurement parameters, image features, time series analysis, and abnormal frame correction. This solves the problem that traditional optical measurement systems cannot dynamically adapt to complex changes and abnormal data. Through this method, the system can monitor, identify, and correct abnormal data in real time in dynamic measurement environments, thereby significantly improving the accuracy of dimensional evaluation and geometric tolerance judgment, and ensuring the reliability of measurement results. Attached Figure Description

[0050] The present invention will be further explained below with reference to the accompanying drawings and embodiments:

[0051] Figure 1 This is a flowchart of the automated optical measurement method for nut dimensions and geometric tolerances provided in Embodiment 1 of the present invention;

[0052] Figure 2 A flowchart illustrating the method for obtaining the edge contour change rate provided in Embodiment 1 of the present invention;

[0053] Figure 3 This is a flowchart of the image anomaly frame correction method provided in Embodiment 1 of the present invention;

[0054] Figure 4 This is a module connection diagram of the automated optical measurement system for nut dimensions and geometric tolerances provided in Embodiment 2 of the present invention. Detailed Implementation

[0055] To make the technical means, creative features, objectives, and effects of this invention readily understandable, the invention is further described below with reference to specific illustrations. It should be noted that, unless otherwise specified, the embodiments and features described in these embodiments can be combined with each other.

[0056] Example 1

[0057] Please see Figures 1-3 This invention provides a technical solution: an automated optical measurement method for nut dimensions and geometric tolerances, comprising:

[0058] During the measurement process, each continuously acquired image frame is analyzed in real time, and image frame feature vectors are constructed by extracting the image feature parameters of each image frame.

[0059] The image feature parameters include the mean pixel distribution, grayscale histogram vector, edge contour change rate, and local texture consistency coefficient for each frame of the image.

[0060] Methods for obtaining the pixel distribution mean and gray-level histogram vector include:

[0061] Each frame of the original image is converted to grayscale, and the image is cropped based on the preset measurement area to obtain the grayscale image of the measurement area.

[0062] The distribution of gray values ​​of all pixels within the measurement area is statistically analyzed, the gray values ​​are summed, and the mean of the pixel distribution is calculated.

[0063] The pixel gray values ​​of the grayscale image of the measurement area are divided into multiple fixed grayscale intervals according to a preset grayscale division strategy; the number of pixels in each grayscale interval is counted and the proportion of each pixel in the total number of pixels in the image area is calculated; the pixel proportions corresponding to each grayscale interval are arranged in order of grayscale interval to form a grayscale histogram vector that represents the grayscale distribution characteristics of the image area.

[0064] It should be noted that the grayscale range division example is as follows:

[0065] Interval 1 corresponds to the grayscale range [0,51]; Interval 2 corresponds to the grayscale range (51,102]; Interval 3 corresponds to the grayscale range (102,153]; Interval 4 corresponds to the grayscale range (153,204]; Interval 5 corresponds to the grayscale range (204,255).

[0066] The method for obtaining the edge contour change rate includes:

[0067] Edge detection is performed on the preprocessed image to extract the thread edge pixels. The edge pixels are then located according to a preset coordinate reference. In two consecutively acquired adjacent frames, the positions of the corresponding matching edge points are used to calculate the spatial offset of each corresponding edge point. At the same time, the rate of change of the number of pixels that constitute the thread edge in the current frame is counted. The average edge offset and the rate of change of the number are weighted and summed to obtain the edge contour change rate that characterizes the degree of change of the thread contour over time.

[0068] Example of calculating the rate of change of edge contour:

[0069] Let the image of frame t be denoted as The set of threads edge pixels extracted in frame t is denoted as ; ; Represents the image in frame t. The spatial position of each thread edge pixel in the image coordinate system;

[0070] Let the set of spiral edge pixels corresponding to frame t-1 be denoted as ; ;in, In the (t-1)th frame of the image, the first... The spatial position of each thread edge pixel in the image coordinate system;

[0071] The number of edge pixels are as follows: ; ; Indicates the first Set of pixels representing the thread edge in a frame image The number of edge pixels included; Indicates the first Set of pixels representing the thread edge in a frame image The number of edge pixels included, of which, Represents taking a set The number of elements in the text, that is, the total number of elements;

[0072] The corresponding matching number The formula for calculating the position offset of an edge point in an adjacent frame is: In the formula, Indicates the first The first frame of the image The spatial position of each thread edge pixel in the image coordinate system For the corresponding matched first The position offset of each edge point in adjacent frames The norm of a vector is used in this embodiment to characterize the Euclidean length of the difference vector between the positions of two corresponding edge points.

[0073] For all successfully matched edge points, calculate the average edge offset: In the formula, This represents the number of edge points that were successfully matched in two adjacent frames. This represents the average edge offset.

[0074] The rate of change of quantity The calculation formula is: ;

[0075] The expression for the weighted summation of edge position offset and quantity change rate is as follows:

[0076] In the formula, β is the edge position offset weighting coefficient, and β is the edge pixel number change weighting coefficient. It should be noted that... The edge contour change rate, and the edge position offset weighting coefficient. The weighting coefficient β for the change in the number of edge pixels is determined based on the contribution of the two types of features to the thread profile anomaly recognition results. Specifically, multiple sets of continuous nut image samples, including normal imaging state, slight profile disturbance state, and obvious abnormal state, can be collected in advance. The average edge offset and the rate of change in the number of edge pixels in each frame are calculated, and both are normalized to a unified dimension. Based on this, the sensitivity of the average edge offset to the change in edge spatial position and the sensitivity of the rate of change in the number of edge pixels to the change in edge integrity are analyzed. Combining the influence of these two factors on the accuracy of abnormal frame recognition, the stability of size extraction, or the error in form and position tolerance calculation, the corresponding weight values ​​are determined.

[0077] The method for obtaining the local texture consistency coefficient includes:

[0078] Perform edge detection on the current image frame to obtain the set of edge points of the thread contour. ; with each edge point Centered on the contour, a neighborhood region of a predetermined width is constructed along the contour normal direction, wherein, ; Indicates the first The set of thread edge contour points extracted from the frame image, the first The first frame of the image The pixel coordinates of the thread edge points are ;

[0079] It should be noted that the preset width depends on the image resolution, the required texture analysis precision, and the geometric characteristics of the target object. Optionally, the preset width can be set as a proportion of the target object's width, such as 3 times the pixel resolution.

[0080] Within the neighborhood area, it is divided into K local analysis windows according to a preset size. Each local analysis window is used to independently extract texture features and reflect the imaging stability state within the area.

[0081] It should be noted that the preset size is set based on factors including image resolution and the scale of local texture changes in the thread contour. Specifically, when the image resolution is high and the local texture changes are fine, the size is set to a smaller value to improve the ability to distinguish local texture changes; when the image resolution is low or the texture area is relatively flat, the size is set to a larger value to ensure that each window contains a sufficient number of effective pixels, thereby improving the stability of the gradient direction statistics results.

[0082] Optionally, ; Indicates the first A set of local analysis windows constructed in a frame image. Represents the image in frame t. A local analysis window;

[0083] For each local analysis window The gradient direction angle of each pixel within the window is calculated, and the window texture consistency coefficient is calculated based on the degree of concentration of the gradient direction angle within the local analysis window.

[0084] Example: The formula for calculating the pixel gradient direction angle is:

[0085] In the formula, Represents pixels gradient direction angle at that location, , Representing pixels Gray-level gradient components in the horizontal and vertical directions;

[0086] No. Frame, First The formula for calculating the window texture consistency coefficient of texture direction distribution within a local analysis window is as follows: In the formula, For the first Frame, First The window texture consistency coefficient of texture direction distribution within a local analysis window; Represents the image of frame t. The number of valid pixels included in the statistics within each local analysis window;

[0087] It should be noted that, The larger the value, the more concentrated the direction and the higher the consistency of the local texture.

[0088] The local texture consistency coefficient is obtained by summing the window texture consistency coefficients obtained from each analysis window and dividing by the number of windows.

[0089] The optical measurement parameters corresponding to each frame of the image are collected, and the optical measurement parameters are correlated with the image frame feature vector in multiple dimensions to form an optical-image joint feature vector.

[0090] In this embodiment, by acquiring image frames in real time and extracting image feature parameters, a detailed image frame feature vector can be generated. This vector is then correlated with optical measurement parameters in multiple dimensions to form an optical-image joint feature vector. The generation of this joint feature vector effectively enhances the system's comprehensive understanding of the measurement data, enabling each image frame to accurately reflect its corresponding optical measurement parameter information. This improves the system's ability to model the complex relationship between optical characteristics and image information.

[0091] The optical measurement parameters include light intensity distribution characteristic parameters, exposure time, and shutter speed.

[0092] The method for forming the optical-image joint feature vector includes:

[0093] For each frame of image During data acquisition, corresponding optical measurement parameters are also acquired, including the light intensity distribution characteristics of the entire imaging area, the exposure time and shutter speed of each frame of the image.

[0094] The optical measurement parameters are normalized, and according to the time frame correspondence, the normalized optical measurement parameters are combined with the image frame feature vector of the corresponding frame image to form the optical-image joint feature vector corresponding to the frame image.

[0095] Example of optical-image joint feature matrix: In the formula, For the first The optical-image joint feature vector of a frame. ∈ (1, 2, ..., ), This represents the total number of frames in continuously acquired images. For the first Frame image frame feature vector, The first Normalized light intensity distribution characteristic parameters, exposure time, and shutter speed of frame images;

[0096] It should be noted that the collected optical measurement parameters are mapped using a combination of maximum value normalization and reference value proportional normalization. The light intensity distribution is mapped to a unified range by dividing the pixel value of each frame by the maximum light intensity value of that frame. The exposure time and shutter speed are obtained by taking the ratio with the preset reference value. The logic for setting the preset reference value is as follows: the exposure time and shutter speed parameters that can obtain stable image quality and small measurement error under normal measurement conditions are used as the benchmark values.

[0097] The method for obtaining the light intensity distribution characteristic parameters includes:

[0098] The grayscale values ​​of the acquired raw images Normalization is performed to obtain a normalized image. ;

[0099] in, ; Let represent the maximum grayscale value of this frame of the image; where, This represents the grayscale value of the image frame.

[0100] Calculate the light intensity distribution characteristic parameters of the entire imaging area for the normalized image;

[0101] The formula for calculating the characteristic parameters of light intensity distribution is: ;

[0102] In the formula, These are the characteristic parameters of light intensity distribution. Average light intensity; This represents the total number of pixels in the imaging area. The light intensity distribution characteristic parameter indicates that the distribution of pixel grayscale values ​​within the imaging area varies significantly, resulting in uneven brightness and the potential for locally overly bright or dark areas.

[0103] By using temporal correlation analysis, a temporal dependency graph is established, and the similarity measure between each frame and its preceding and following neighboring frames is calculated through the correlation matrix between image frames to obtain the total similarity measure of each frame.

[0104] The method for obtaining the total similarity metric for each frame includes:

[0105] Each frame of image data is connected to the adjacent frames before and after it by establishing an edge connection to form a graph structure, where the nodes of the graph represent image frames, and the edges represent the temporal dependencies between image frames.

[0106] For each pair of adjacent frames, calculate the similarity measure of their image frame feature vectors to obtain an element of the similarity matrix;

[0107] For each frame of image, calculate its similarity measure with the preceding and following adjacent frames to form a similarity matrix, and then calculate the total similarity measure of the frame by taking the average value.

[0108] Example: The formula for calculating the similarity metric between frame t and frame (t-1) is:

[0109] In the formula, Let be the image frame feature vector of the t-th frame. Let be the image frame feature vector of the (t-1)th frame. Let be the similarity measure between frame t and frame (t-1). Let be the length of the image frame feature vector of the t-th frame;

[0110] The formula for calculating the total similarity metric of the t-th frame image is:

[0111] In the formula, Let be the total similarity measure of the images in frame t. Similarity measurement between frame t and frame t+1;

[0112] In this embodiment, through temporal correlation analysis, the present invention can calculate the similarity metric between each frame and its preceding and following adjacent frames in real time, thereby providing a strong basis for abnormal frame prediction. By inputting the optical-image joint feature vector of each frame image and the temporal similarity metric into the abnormal frame prediction model, abnormal image frames can be identified efficiently and accurately. This method not only improves the accuracy of abnormal frame identification, but also provides a reliable basis for subsequent abnormal image correction.

[0113] The optical-image joint feature vector and total similarity metric of each frame are input into the pre-built abnormal frame prediction model, and the result of determining whether each frame is a normal image frame or an abnormal image frame is output.

[0114] The training method for the abnormal frame prediction model includes:

[0115] Raw data collection was carried out in the target monitoring scenario. Multiple sets of complete image sequence frames were continuously acquired through optical imaging equipment. The acquisition process covered the working conditions of normal equipment operation, image occlusion, image distortion, target missing, and abnormal lighting, so that the collected samples contained a sufficient number of normal image frames and various abnormal image frames. All collected image frames were manually labeled and verified. Image frames that met the normal monitoring conditions were marked as 0, and image frames with abnormalities were uniformly marked as 1. The optical-image joint feature vector and total similarity measure of each image frame and the corresponding annotation were obtained to construct a dataset.

[0116] Gradient boosting tree classifier was selected as the abnormal frame prediction model, and the specific training method is as follows:

[0117] Before model training, initial configuration should be completed, and initial hyperparameters should be set. The initial hyperparameters specifically include: number of decision trees 100-150, maximum depth of a single tree 4-6, minimum number of samples for node splitting 8-12, maximum number of features considered during splitting 3, splitting criterion is Gini impurity, learning rate is 0.05-0.1, and regularization coefficient L2 is 0.1-0.2;

[0118] During the model training phase, a binary cross-entropy loss function is used to measure the deviation between the model's predicted probability and the true label of the image frame. The optical-image joint feature vector and the total similarity measure are used as the model input features, and the corresponding image frame encoded labels are used as the prediction targets. Model training is carried out based on training data. The construction of each new tree is aimed at fitting the classification residual of the training set loss function. The optimal splitting feature is selected by the Gini impurity criterion, such as prioritizing the optical-image joint feature vector as the splitting feature, and the samples are divided into different child nodes until the stopping condition is met, such as reaching the preset maximum depth or the number of child node samples is less than the minimum number of samples.

[0119] The gradient descent method is used to optimize the weights of the leaf nodes of each new tree. The contribution weight of the new tree to the final judgment result is adjusted by the learning rate to avoid a single tree dominating the prediction process and to ensure the stability of the model's judgment.

[0120] In the hyperparameter optimization phase, the Bayesian optimization method is adopted to search for the optimal combination of hyperparameters within the preset optimization range, with the optimization objective being to maximize the F1 score on the validation set.

[0121] An early stopping mechanism is introduced during training. The collected training data is divided into training, validation, and test sets in a 7:2:1 ratio. The validation set F1 score is calculated every 20 trees. Training stops when the validation set F1 score improves by less than 0.01 over three consecutive iterations to prevent overfitting. After training, the model parameters with the highest validation set F1 score are saved to ensure the model has stable decision-making capabilities for image frames with different feature combinations.

[0122] During the model evaluation phase, the trained model is validated using a test set, and precision, recall, and F1 score are calculated. If the precision is ≥90%, the recall is ≥95%, and the F1 score is ≥92%, the model performance evaluation is considered satisfactory and it can be deployed for practical application, used to determine whether an image frame is normal or abnormal in each frame.

[0123] For abnormal image frames, identify the abnormal type of the image frame, and perform dynamic correction based on the abnormal type of the image frame and the information of adjacent frames. Based on the corrected normal image frames, complete the extraction of the nut's size parameters and the calculation of its geometric tolerances.

[0124] The method for obtaining the image frame anomaly type includes:

[0125] The optical-image joint feature vector of frames identified as anomalous is separated and analyzed to obtain the optical measurement parameter vector and the image feature vector, respectively.

[0126] The deviation of the optical measurement parameters of the current frame from the mean and variance of the preceding and following normal frames is calculated. If the deviation exceeds a set threshold, an optically anomalous component is marked. The specific method for setting the threshold is as follows: A large number of historical normal image sequences are pre-collected under normal operating conditions of the same measurement system. For each historical normal frame, several consecutive normal images before and after it are used as a reference frame set. The deviation of each optical measurement parameter relative to the mean and standard deviation of the reference frames is calculated, forming a historical deviation sample set. The mean and standard deviation of the historical deviation sample set are calculated, and the anomaly judgment threshold for the corresponding optical measurement parameter is determined according to the formula "threshold = mean + kkk times the standard deviation", where kkk is 2 to 3. When the deviation of the optical measurement parameter corresponding to the current frame exceeds the anomaly judgment threshold, the parameter is marked as an optically anomalous component.

[0127] Based on the image feature vector, the degree of change of the abnormal frame image feature vector relative to the structural features of the preceding and following normal frames is calculated. When the degree of change exceeds the corresponding image structural anomaly determination threshold, the corresponding image feature is marked as an abnormal image component. The anomaly determination threshold is set using a rolling window approach, calculating the local mean and local fluctuation amplitude using the image features of several normal frames before and after the current frame, and setting the threshold to 2-3 times the local fluctuation range. This allows the threshold to be dynamically adjusted with minor changes in the system state, improving the reliability of abnormal frame identification.

[0128] Based on the labeling results of optical and image anomalous components, the anomalous type of the anomalous frame is determined: when only optical anomalous components exist, it is determined to be an optical anomalous frame; when only image anomalous components exist, it is determined to be an image anomalous frame; when both optical and image anomalous components exist, it is determined to be an optical-image composite anomalous frame.

[0129] Methods for dynamic correction based on image frame anomaly types and adjacent frame information include:

[0130] For abnormal frames identified as optically abnormal, a reference mapping for light intensity distribution feature parameters is constructed based on the light intensity distribution feature parameters of consecutive normal frames. The light intensity distribution feature parameters of the abnormal frame are mapped to the stable parameter range corresponding to the reference mapping. On this basis, light intensity redistribution and grayscale mapping correction are performed on the abnormal frame image to restore imaging consistency without changing the geometric structure. The stable parameter range refers to the allowable fluctuation range used to describe the light intensity distribution feature parameters during optical measurement.

[0131] In some implementations, the implementation steps include: performing light intensity redistribution and grayscale mapping correction on abnormal frame images as follows: based on historical measurement data, determining the stable fluctuation range of the light intensity distribution characteristic parameters, denoted as: In the formula, and These represent the minimum and maximum permissible values ​​of the light intensity distribution characteristic parameters under stable imaging conditions, respectively. These are characteristic parameters of light intensity distribution;

[0132] The light intensity distribution feature parameters of the abnormal frames are mapped to the stable parameter range to obtain the corrected light intensity distribution feature parameters. ;

[0133]

[0134] in, This indicates a mapping operator within parameter limits; This represents the characteristic parameter of light intensity distribution in the t-th frame image.

[0135] Let the pixel value of the original image in frame t be... Based on the corrected light intensity distribution characteristic parameters The image undergoes intensity redistribution processing:

[0136] In the formula, The intermediate corrected image of frame t after intensity redistribution processing. Pixel value at;

[0137] Based on the light intensity redistribution, gray-level mapping is performed on the image to make its gray-level distribution consistent with the stable reference mapping. The gray-level mapping function is defined as follows:

[0138] ; where, mapping function Based on the grayscale histograms of the preceding and following normal frames, the grayscale distribution of the abnormal frame is aligned with the reference grayscale distribution. Let be the pixel value at (x,y) of the final corrected image of frame t after grayscale mapping.

[0139] For anomalous frames identified as image anomalies, the pixel coordinates of corresponding regions in the preceding and following normal frames are first extracted. The mean position of each pixel is then calculated based on these coordinates, constructing a set of mean coordinate points. Next, the pixel coordinates of the preceding and following frames are connected to the mean coordinate points to form a continuous pixel trajectory, representing the spatial continuity of pixels over time. In the anomalous frame, the pixel coordinates of the corresponding region are obtained. Pixels located on the trajectory are not corrected; for pixels not located on the trajectory, the shortest distance point from the pixel to the trajectory is calculated, and the coordinates of this shortest distance point are used as the corrected pixel position to restore the temporal spatial continuity of the pixels.

[0140] In this embodiment, by using the average pixel coordinates and route trajectory of the preceding and following normal frames for guidance, only abnormal pixels that deviate from the trajectory are corrected, avoiding excessive intervention in normal pixels and thus ensuring the integrity of local structural information. The corrected image maintains consistency with the preceding and following frames in terms of edge contours, texture, and grayscale distribution, improving the stability and reliability of optical measurement parameter evaluation during dynamic measurement.

[0141] For abnormal frames identified as optical-image composite anomalies, optical correction is performed first, followed by image anomaly correction.

[0142] The corrected abnormal frames are regenerated into optical-image joint feature vectors, and the total similarity metric is obtained and re-input into the abnormal frame prediction model for verification. If the model determines that it is normal, it can be used. If the model still determines that it is abnormal, iterative correction can be performed with a preset number of iterations. If the model still determines that it is abnormal after the iteration is completed, it is marked as an unrecoverable abnormal frame.

[0143] In this embodiment, when an abnormal image frame is determined, for optically abnormal frames, an optical measurement parameter correction reference mapping is constructed to map the optical measurement parameters of the abnormal frame to a stable parameter range. Based on this, the image is subjected to light intensity redistribution and grayscale mapping correction to restore the imaging consistency of the image. For image abnormal frames, the pixel coordinate information of the preceding and following normal frames is extracted, the mean coordinate points are constructed, and a pixel trajectory is formed to restore the temporal and spatial continuity of the image. This correction method not only ensures the consistency of image data, but also avoids measurement errors caused by a single abnormal data source.

[0144] In the correction of optical-image composite anomalous frames, this invention proposes a dual correction strategy of first performing optical correction and then image correction. Through iterative correction, it ensures that even under complex anomalous conditions, the system can still obtain accurate measurement results. If the corrected anomalous frame passes model verification, the system can restore its normal state in real time, thereby improving the reliability and accuracy of the measurement results.

[0145] Example 2

[0146] Please see Figure 4 This invention provides a technical solution: an automated optical measurement system for nut dimensions and geometric tolerances, used to implement the aforementioned automated optical measurement method for nut dimensions and geometric tolerances. The system includes:

[0147] The image feature extraction module performs real-time analysis on each continuously acquired image frame during the measurement process, and constructs an image frame feature vector by extracting the image feature parameters of each image frame.

[0148] The optical measurement parameter acquisition module acquires the optical measurement parameters corresponding to each frame of the image.

[0149] The optical-image joint module performs multi-dimensional correlation between optical measurement parameters and image frame feature vectors to form an optical-image joint feature vector.

[0150] The temporal correlation analysis module utilizes temporal correlation analysis to establish a temporal dependency graph. It calculates the similarity measure between each frame and its preceding and following neighboring frames using a correlation matrix, thereby obtaining the total similarity measure for each frame.

[0151] The abnormal frame determination module inputs the optical-image joint feature vector and total similarity measure of each frame into the pre-built abnormal frame prediction model, and outputs the determination result of normal or abnormal image frames for each frame.

[0152] The abnormal image correction module identifies the abnormal type of the image frame, performs dynamic correction based on the abnormal type and information from adjacent frames, and extracts the nut's dimensional parameters and calculates its geometric tolerances based on the corrected normal image frame.

[0153] Specifically, the method for extracting the dimensional parameters of the nut and calculating the geometric tolerances includes: based on the corrected normal image frame, the image is first preprocessed, including grayscale normalization, noise suppression and light intensity equalization, to ensure uniform image brightness distribution and remove local light spots or interference noise.

[0154] For each nut target area, contour segmentation is performed. The nut contour is extracted by edge detection or image segmentation methods to make the contour clear and identifiable, and to form a set of contour points that can be used for subsequent measurements.

[0155] On the set of contour points, key geometric feature points are identified, including the center of the inner hole, the vertex of the outer edge, and the start and end points of the thread. Using the contour points and feature points, the main dimensional parameters of the nut are calculated.

[0156] The outer and inner diameters are obtained by fitting a circle or polygon with the contour points to obtain the minimum outer circle diameter and the maximum inner circle diameter. The thickness is calculated by the vertical distance between the contour points on the upper and lower surfaces to obtain the average thickness. The thread parameters are obtained by fitting the thread shape with the thread contour points to obtain the pitch and tooth height.

[0157] For the form and position tolerances of nuts, firstly, by fitting the outer circle, inner circle, and thread cylindrical surface, the deviation sequence of each profile point from the fitted circle or cylindrical surface is calculated. The maximum deviation is used as the roundness or cylindricity tolerance. Flatness and parallelism are obtained by performing plane fitting on the upper and lower profile points of the nut. The maximum deviation from the point to the plane is used as the flatness, and the deviation of the surface normal vector fitting is used as the parallelism. Eccentricity is calculated by the difference in position between the inner hole and the outer edge center. Symmetry is measured by measuring the deviation of corresponding points on both sides of the profile rotation symmetry axis. Other geometric features, such as thread groove depth and thread angle, are obtained by fitting the corresponding feature lines or surfaces to the profile points and extracting the deviation values ​​as form and position tolerances.

[0158] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely illustrative of the principles of the invention. Various changes and modifications can be made to the invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of the present invention as claimed. The scope of protection of the present invention is defined by the appended claims and their equivalents.

Claims

1. A method for automated optical measurement of nut size and geometric tolerances, characterized in that, include: During the measurement process, each continuously acquired image frame is analyzed in real time, and image frame feature vectors are constructed by extracting the image feature parameters of each image frame. The optical measurement parameters corresponding to each frame of the image are collected, and the optical measurement parameters are correlated with the image frame feature vector in multiple dimensions to form an optical-image joint feature vector. By using temporal correlation analysis, a temporal dependency graph is established, and the similarity measure between each frame and its preceding and following neighboring frames is calculated through the correlation matrix between image frames to obtain the total similarity measure of each frame. The optical-image joint feature vector and total similarity metric of each frame are input into the pre-built abnormal frame prediction model, and the result of determining whether each frame is a normal image frame or an abnormal image frame is output. For abnormal image frames, identify the abnormal type of the image frame, and perform dynamic correction based on the abnormal type of the image frame and information from adjacent frames. Based on the corrected normal image frames, complete the extraction of the nut's dimensional parameters and the calculation of its geometric tolerances.

2. The method of claim 1, wherein the method further comprises: The image feature parameters include the mean pixel distribution, grayscale histogram vector, edge contour change rate, and local texture consistency coefficient for each frame of the image.

3. The method of claim 2, wherein the method further comprises: The method for obtaining the edge contour change rate includes: Edge detection is performed on the preprocessed image to extract the thread edge pixels. The edge pixels are then located according to a preset coordinate reference. In two consecutively acquired adjacent frames, the positions of the corresponding matching edge points are used to calculate the spatial offset of each corresponding edge point. At the same time, the rate of change of the number of pixels that constitute the thread edge in the current frame is counted. The average edge offset and the rate of change of the number are weighted and summed to obtain the edge contour change rate that represents the degree of change of the thread contour over time.

4. The method of claim 2, wherein the method further comprises: The method for obtaining the local texture consistency coefficient includes: Perform edge detection on the current image frame to obtain a set of thread contour edge points. With each edge point as the center, construct a neighborhood region of a preset width along the contour normal direction. Within the neighborhood area, it is divided into K local analysis windows according to a preset size. Each local analysis window is used to independently extract texture features and reflect the imaging stability state within the area. For each local analysis window, the gradient direction angle of the pixels within the window is calculated, and the window texture consistency coefficient is calculated based on the degree of distribution concentration of the gradient direction angle within the local analysis window. The local texture consistency coefficient is obtained by summing the window texture consistency coefficients obtained from each analysis window and dividing by the number of windows.

5. The method of claim 1, wherein the method further comprises: The optical measurement parameters include light intensity distribution characteristic parameters, exposure time, and shutter speed.

6. The automated optical measurement method for nut dimensions and geometric tolerances according to claim 1, characterized in that, The method for obtaining the total similarity metric for each frame includes: Each frame of image data is connected to the adjacent frames before and after it by establishing an edge connection to form a graph structure, where the nodes of the graph represent image frames, and the edges represent the temporal dependencies between image frames. For each pair of adjacent frames, calculate the similarity measure of their image frame feature vectors to obtain an element of the similarity matrix; For each frame of image, calculate its similarity measure with the preceding and following adjacent frames to form a similarity matrix, and then calculate the total similarity measure of the frame by taking the average value.

7. The automated optical measurement method for nut dimensions and geometric tolerances according to claim 1, characterized in that, The method for forming the optical-image joint feature vector includes: When acquiring each frame of image, the corresponding optical measurement parameters are also obtained, including the light intensity distribution characteristics of the entire imaging area, the exposure time and shutter speed of each frame of image are recorded. The optical measurement parameters are normalized. According to the time frame correspondence, the normalized optical measurement parameters are concatenated and combined with the image frame feature vector of the corresponding frame image to form the optical-image joint feature vector corresponding to the frame image.

8. The automated optical measurement method for nut dimensions and geometric tolerances according to claim 1, characterized in that, The method for obtaining the image frame anomaly type includes: The optical-image joint feature vector of frames identified as anomalous is separated and analyzed to obtain the optical measurement parameter vector and the image feature vector, respectively. Calculate the deviation of the optical measurement parameters of this frame from the mean and variance of the normal frames before and after it. If the deviation exceeds the set threshold, mark the optical anomalous component. Based on the image feature vector, the degree of change of the abnormal frame image feature vector relative to the structural features of the preceding and following normal frames is calculated. When the degree of change exceeds the corresponding image structural anomaly determination threshold, the corresponding image feature is marked as an image abnormal component. Based on the labeling results of optical and image anomalous components, the anomalous type of the anomalous frame is determined to obtain the image frame anomalous type.

9. The automated optical measurement method for nut dimensions and geometric tolerances according to claim 8, characterized in that, Methods for determining the anomaly type of an image frame based on the labeling results of optical and image anomaly components include: When only optical anomalous components exist, it is determined to be an optical anomalous frame; when only image anomalous components exist, it is determined to be an image anomalous frame; when both optical and image anomalous components exist, it is determined to be an optical-image composite anomalous frame.

10. The automated optical measurement method for nut dimensions and geometric tolerances according to claim 1, characterized in that, The method for dynamic correction based on image frame anomaly type and adjacent frame information includes: For abnormal frames identified as optically abnormal, a reference mapping for light intensity distribution feature parameters is constructed based on the light intensity distribution feature parameters of consecutive normal frames. The light intensity distribution feature parameters of the abnormal frames are mapped to the stable parameter range corresponding to the reference mapping. On this basis, light intensity redistribution and grayscale mapping correction are performed on the abnormal frame image to restore imaging consistency without changing the geometric structure.

11. The automated optical measurement method for nut dimensions and geometric tolerances according to claim 10, characterized in that, The method for dynamic correction based on image frame anomaly type and adjacent frame information also includes: For abnormal frames identified as image anomalies, the pixel coordinate information of the corresponding regions in the consecutive normal frames before and after them is extracted. The mean position of each pixel is calculated based on the coordinate information, and a set of mean coordinate points is constructed. The pixel coordinates of the consecutive frames are connected with the mean coordinate points to form a continuous pixel trajectory. In the abnormal frame, the pixel coordinates of the corresponding region are obtained. For pixels located on the trajectory, no correction is made. For pixels not located on the trajectory, the shortest distance point from the pixel to the trajectory is calculated, and the coordinates of the shortest distance point are used as the corrected pixel position.

12. The automated optical measurement method for nut dimensions and geometric tolerances according to claim 11, characterized in that, The method for dynamic correction based on image frame anomaly type and adjacent frame information also includes: For abnormal frames identified as optical-image composite anomalies, optical correction is performed first, followed by image anomaly correction. The corrected abnormal frames are regenerated into optical-image joint feature vectors, and the total similarity metric is obtained and re-input into the abnormal frame prediction model for verification. If the model determines that it is normal, it can be used. If the model still determines that it is abnormal, iterative correction can be performed with a preset number of iterations. If the model still determines that it is abnormal after the iteration is completed, it is marked as an unrecoverable abnormal frame.

13. An automated optical measurement system for nut dimensions and geometric tolerances, used to implement the automated optical measurement method for nut dimensions and geometric tolerances as described in any one of claims 1-12, characterized in that, The system includes: The image feature extraction module performs real-time analysis on each continuously acquired image frame during the measurement process, and constructs an image frame feature vector by extracting the image feature parameters of each image frame. The optical measurement parameter acquisition module acquires the optical measurement parameters corresponding to each frame of the image. The optical-image joint module performs multi-dimensional correlation between optical measurement parameters and image frame feature vectors to form an optical-image joint feature vector. The temporal correlation analysis module utilizes temporal correlation analysis to establish a temporal dependency graph. It calculates the similarity measure between each frame and its preceding and following neighboring frames using a correlation matrix, thereby obtaining the total similarity measure for each frame. The abnormal frame determination module inputs the optical-image joint feature vector and total similarity measure of each frame into the pre-built abnormal frame prediction model, and outputs the determination result of normal or abnormal image frames for each frame. The abnormal image correction module identifies the abnormal type of the image frame, performs dynamic correction based on the abnormal type and information from adjacent frames, and extracts the nut's dimensional parameters and calculates its geometric tolerances based on the corrected normal image frame.

Images