Multi-mode double-link thread measurement method

Abstract

The invention discloses a multi-modal double-link thread measurement method. The method comprises the following steps: acquiring a thread multi-modal image; filtering and segmenting the acquired image, and extracting a target area through local segmentation; geometric parameters of the threads are measured by extracting artificial features; the geometric parameters of the depth feature classification thread are learned through AI; and performing dynamic fusion on geometric parameters obtained through the artificial features and the depth features to obtain final thread parameters. According to the method for measuring the geometric parameters of the multi-mode and double-link threads, the geometric parameters of the threads are measured in a non-contact mode, and abrasion to the threads is avoided. Meanwhile, unstable influences such as scrap iron and burrs on the surface of the thread are effectively treated in a mode of combining manual characteristics and depth characteristics, and the stability of a measurement result is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of thread measurement, and in particular to a multi-modal dual-link thread measurement method. Background Art

[0002] In industrial production, threaded workpieces are a commonly used mechanical structure that has the functions of sealing, mechanical connection and kinetic energy transmission. When the geometric parameters of the threaded workpiece are out of tolerance, such as excessive pitch or tooth angle deviation, it may lead to problems such as reduced sealing, fracture of the connection and reduced transmission efficiency. In high-demand fields such as aerospace, oil and gas pipelines and military defense, the use of threaded workpieces is huge and their quality directly affects the safety of life and major property. Therefore, there are strict standards and specifications for the manufacture, processing and use of threaded workpieces. It is necessary to precisely measure the threaded workpiece before installation and use to ensure that the geometric parameters and model specifications of the thread meet the production and use requirements. The main geometric parameters of the thread are as follows: Figure 1 As shown in the figure, the major diameter d is the maximum diameter of the thread, that is, the diameter of the ideal cylinder tangent to the thread crest; the pitch ρ is the distance between the corresponding characteristic points of two adjacent thread profiles; the tooth profile angle α is the angle between the two sides of the thread profile; the tooth crest P is the maximum diameter of the thread, that is, the diameter of the ideal cylinder tangent to the thread crest; the pitch ρ is the distance between the corresponding characteristic points of the ... t :Top of the thread protrusion; bottom of the thread P d : The bottom surface of two adjacent thread profiles.

[0003] The measurement methods of thread geometric parameters are mainly divided into two types: contact measurement and non-contact measurement; common contact measurement methods include three-coordinate machines and thread integrated measuring machines. Although the contact method can measure accurate data, the contact measurement equipment is expensive, cumbersome to operate, and easily causes wear to the thread, making it unsuitable for batch measurement in industry. For this reason, some non-contact thread measurement methods have been proposed. Non-contact measurement mainly uses machine vision to collect thread images and then uses image algorithms to calculate the geometric parameters of the thread. For example, the authorization announcement date is 2023-04-07, the announcement number is CN110211047B, and the name is a thread measurement method with image rotation [1]; the authorization announcement date is 2021-05-25, the authorization announcement number is CN110849287B, and the name is a thread measurement method with tooth angle compensation [2]. While non-contact measurement methods based on machine vision do not wear threads, offer high inspection efficiency, and are cost-effective, they still face unresolved challenges. The camera's inherent noise and the spiral structure of threads can lead to a loss of spatial information when generating images. Furthermore, traditional geometric imaging algorithms struggle to handle unstable factors like burrs and iron filings on the thread surface, resulting in significant measurement errors.

[0004] The above patent [1] makes the axis of the thread parallel to the image coordinate axis by performing operations such as binarization, pixel calculation, translation, and rotation on the thread image, thereby improving the accuracy of thread parameter measurement. Patent [2] proposes a tooth angle compensation algorithm, which first collects the slope of the thread image, then calculates the compensation amount of the left and right tooth angles, and finally calculates the average compensation amount of the tooth angle to solve the error caused by image distortion. Most solutions solve the compensation amount of thread parameters at the algorithm level, and do not fundamentally solve the distortion problem. In addition, most machine vision-based thread measurement methods measure threads in an ideal clean state, without considering unstable factors such as iron filings and burrs on the thread surface. Summary of the Invention

[0005] To solve the above technical problems, the purpose of the present invention is to provide a multimodal dual-link thread measurement method, which uses a macro camera and a short-focus camera to collect two different modal images of the thread shaft and head. The multimodal images can provide richer spatial data and compensate for the influence of image distortion. After that, a dual-link measurement method is designed by combining a deep learning network and a geometric image algorithm. The deep learning network can learn the depth characteristics of the thread parameters and effectively deal with the unstable factors of the thread surface. The geometric algorithm can ensure the accuracy of the measurement. Finally, by dynamically fusing the measurement results of the two links, the final measurement result is balanced between accuracy and stability. The problem of measurement error caused by image distortion is solved, and the influence of unstable factors on the thread surface is solved.

[0006] The purpose of the present invention is achieved through the following technical solutions:

[0007] A multi-modal dual-link thread measurement method, comprising:

[0008] A. Collect multimodal images of threads;

[0009] B. Filter and segment the acquired image, and extract the target area through local segmentation;

[0010] C. Measuring the geometric parameters of the thread by extracting artificial features;

[0011] D. Classify the geometric parameters of the thread through AI learning deep features;

[0012] E. Dynamically fuse the geometric parameters obtained by artificial features and deep features to obtain the final thread parameters.

[0013] Compared with the prior art, one or more embodiments of the present invention may have the following advantages:

[0014] The multimodal, dual-link thread geometry measurement method proposed in this paper measures thread geometry in a non-contact manner, avoiding wear on the thread. Furthermore, by combining manual and depth features, it effectively addresses destabilizing influences such as iron filings and burrs on the thread surface, improving the stability of measurement results and providing technical support for industrialized batch measurement. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 It is an illustration of the main geometric parameters of the thread;

[0016] Figure 2 It is the overall process of the multi-modal dual-link thread measurement method;

[0017] Figure 3 It is the image of the shank and head of the thread;

[0018] Figure 4 This is a schematic diagram of the corner point results;

[0019] Figure 5 It is a deep feature learning network graph. DETAILED DESCRIPTION

[0020] In order to make the objectives, technical solutions and advantages of the present invention more clear, the present invention will be described in further detail below with reference to embodiments and accompanying drawings.

[0021] like Figure 2 The figure shows the overall process of the multi-modal dual-link thread measurement method, including:

[0022] Image acquisition: Most image thread parameter measurement methods generally use ordinary cameras to capture images of the thread shaft. However, when the images captured by ordinary cameras are magnified at high magnification, they will be distorted and introduce measurement errors. In addition, since the thread is a spiral ascending structure, some spatial information will be lost when generating the image, resulting in measurement errors. In order to reduce the error caused by the distortion caused by the camera and the thread structure, the present invention uses macro and short-focus cameras to capture images of the thread shaft and head respectively. The image capture diagram is shown in the figure below. Figure 3 As shown in Figure 2, multimodal images can provide richer spatial information and reduce the impact of image distortion. Macro cameras, with their 1:1 magnification capability, can optimize aberrations, distortion, and dispersion in close-range photography, ensuring edge-to-edge image quality.

[0023] Image filtering: Image filtering is to suppress image noise while preserving image features. The filtering algorithm of the present invention is as follows:

[0024] Step 1: Calculate the median value of each pixel in the r×r neighborhood:

[0025]

[0026] Step 2: Generate a new image based on the median value:

[0027]

[0028] Image segmentation: Image segmentation is to separate the area where the thread is located from the background to reduce the interference of background noise. This invention extracts the target area by improving the local segmentation method. The detailed steps are as follows:

[0029] Step 1: Calculate the average grayscale value in the r×r neighborhood, where g(x, y) represents the grayscale value of the pixel (x, y) in the image matrix:

[0030]

[0031] Step 2: Calculate the standard deviation within the r×r neighborhood:

[0032]

[0033] Step 3: Calculate the threshold of the pixel point (x, y), where R is the dynamic range of the standard deviation, and in this embodiment, R=128; k is the correction coefficient, and in this embodiment, k=0.68;

[0034]

[0035] Step ④: Adjust the r value and repeat steps ① to ③. In this embodiment, the r value is [24, 22, 20, 18, 16, 14, 12, 10, 8, 6, 4, 2] to calculate 12 groups of thresholds.

[0036] Step 5: Calculate the average threshold:

[0037]

[0038] Step 4: Reset the grayscale value of each pixel:

[0039]

[0040] Edge detection and geometric measurement: This link measures the geometric parameters of the thread by extracting artificial features. The specific steps are as follows:

[0041] Step 1: Use the Sobel operator to extract the edge of the thread and obtain the image img1;

[0042] Step ②: Use Harris corner detection algorithm to extract the corner points of the thread on img1, and obtain the corner point set P = {p1, p2, ..., p n};

[0043] Step ③: Classify the tooth vertices and tooth bottom points on the upper and lower sides according to the coordinate relationship of the corner points on the set P, such as Figure 4 As shown, the classification process is as follows: find the average ordinate of all corner points The corner point with a ordinate greater than the average ordinate is the upper corner point Otherwise it is the lower corner point Calculate the average ordinate of the upper and lower corner points respectively. The upper corner point with a greater ordinate than the average ordinate is the tooth vertex. Otherwise it is the bottom point of the tooth The lower corner point with a larger vertical coordinate than the average is the tooth bottom point Otherwise it is the tooth apex

[0044] Step ④: Solve the fitting tangent of the upper and lower tooth vertices. The solution is to use the least square method The value of is minimized, where e, x, and y are the sum of squared errors and the horizontal and vertical coordinates of the tooth vertex, respectively, and the optimal slope m and intercept b of the fitting tangent are obtained.

[0045] Step ⑤: Calculate the major diameter d of the thread using the following formula, where n1 and n2 are the number of upper and lower tooth vertices, respectively. The average distance from each upper and lower tooth vertex to the lower and upper fitting tangents is the fitted major diameter.

[0046]

[0047] Step 6: Calculate the thread pitch p using the following formula, where |·| represents the distance between two points, and n1, n2, n3, and n4 are the number of apex and bottom points on the upper and lower sides, respectively. The formula works by calculating the distances between the apex and bottom points on the upper and lower sides, and then averaging the distances to obtain the fitted pitch p:

[0048]

[0049] Step 7: Calculate the thread profile angle α using the following formula, where <·> represents the angle formed by the three corner points, which can be calculated using the vector dot product formula:

[0050]

[0051] Feature extraction and AI measurement: This link uses AI to learn deep features to classify the geometric parameters of the thread. The specific steps are as follows.

[0052] Step 1: Connect the threaded rod and head images in the channel dimension to form a multidimensional matrix img.

[0053] Step 2: Input the multidimensional matrix img into the deep feature learning network. The network structure is as follows Figure 5As shown in the figure, the network consists of three modules, namely feature learning module (left frame), feature evaluation module (upper right frame), and feature classification module (lower right frame).

[0054] Step 3: Extract thread features through a feature learning module. This module consists of a four-layer architecture, each containing two convolutional layers (Conv) and one maximum pooling layer (Max Pool). The parameters following the convolutional layer represent size × size × number of cores, and the parameters following the maximum pooling layer represent size × size. This double-layer convolution plus a single pooling layer design enables learning complex, nonlinear key features. The convolutional parameters of the four-layer architecture are 9×9×16, 7×7×32, 5×5×64, and 3×3×128, respectively, with the size gradually decreasing and the number of cores gradually increasing. The parameters of the maximum pooling layer are 8×8, 8×8, 4×4, and 4×4, respectively. Its primary function is to gradually extract and retain stable features, from coarse to fine.

[0055] Step 4: The extracted features are evaluated for quality using the feature evaluation module, which consists of one encoding layer (Encode), one large fully connected layer (FC), and three small fully connected layers. The encoding layer integrates the extracted features into a one-dimensional vector. The extracted features are multidimensional matrices of size a×a×n, where a×a represents the matrix size and n represents the dimension. The integration is performed by concatenating the matrices of each dimension from left to right, top to bottom, and end to end to form a one-dimensional vector. Redundant elements of the one-dimensional vector are then randomly sampled to achieve feature pruning. The re-encoded features are then fed into a large fully connected layer with parameters of 1024×100, meaning the input is a 1024×1 one-dimensional vector and the output is a 100×1 vector. After further optimizing the features through the large-scale fully connected layer, the optimized features are input into three small-scale fully connected layers with a parameter of 100×1. The three small connected layers evaluate the quality of the optimized features and finally output the quality scores (0 to 1) of the thread pitch, major diameter and tooth angle.

[0056] Step 5: Classify the extracted features through the feature classification block. The structure of the classification module is similar to that of the evaluation module. The only difference is that the parameters of the last three small-size fully connected layers are 100×8, and their output is an 8-bit binary number ranging from [0000 0000] to [1111 1111], representing pitch 0 to 2.56 mm, major diameter 0 to 25.6 mm, and tooth angle 0 to 256°, respectively.

[0057] Dynamic fusion: According to the score of the feature quality evaluation module, the results of the two links of manual features and deep features are fused. The detailed process is shown in the following formula, where δ, ε, and θ are the output scores of the evaluation module respectively, and represent the weights of the pitch, major warp, and tooth angle results in the deep feature link. p1, d1, and α1 represent the classification results of the deep feature link, and p2, d2, and α2 represent the measurement results of the manual feature link.

[0058]

[0059] Although the embodiments disclosed herein are as described above, the contents described herein are merely embodiments for facilitating understanding of the present invention and are not intended to limit the present invention. Any person skilled in the art may make any modifications and variations in the form and details of the embodiments without departing from the spirit and scope of the present invention. However, the scope of patent protection of the present invention shall remain subject to the scope defined by the appended claims.

Claims

1. A multi-modal dual-link thread measurement method, characterized in that: The following steps are involved: A. Collect multimodal images of threads; B. Filter and segment the acquired image, and extract the target area through local segmentation; C. Measuring the geometric parameters of the thread by extracting artificial features; D. Classify the geometric parameters of the thread through AI learning deep features; E. Dynamically fuse the geometric parameters obtained by artificial features and deep features to obtain the final thread parameters.

2. The multi-modal dual-link thread measurement method according to claim 1, characterized in that: In the above-mentioned A, the images of the stem and head of the thread are collected respectively by macro and short-focus cameras.

3. The multi-modal dual-link thread measurement method according to claim 1, characterized in that: The filtering of the image in B is to retain the image features and suppress the image noise. The filtering algorithms used include: 1) Calculate the median value of each pixel in the r×r neighborhood of the image: 2) Generate a new image based on the median value: Among them, a refers to the grayscale value of the pixel in the image, and mid refers to the median value of the corresponding pixel.

4. The multi-modal dual-link thread measurement method according to claim 1, characterized in that: The image segmentation in B extracts the target area by using a local segmentation method, which specifically includes the following steps: Step 1) Calculate the average grayscale value in the r×r neighborhood: Step 2) Calculate the standard deviation within the r×r neighborhood: Step 3) Calculate the threshold of the pixel (x, y), where R is the dynamic range of the standard deviation, R = 128; k is the correction coefficient, k = 0.68 Step 4) Adjust the r value and repeat steps 1) to 3) to calculate 12 groups of thresholds with the r value set to [24, 22, 20, 18, 16, 14, 12, 10, 8, 6, 4, 2]. Step 5) Calculate the average threshold Step 6) Reset the grayscale value of each pixel: In the formula is the average threshold of pixel (x, y).

5. The multi-modal dual-link thread measurement method according to claim 1, characterized in that: The C specifically includes the following steps: Step 1) Use the Sobel operator to extract the edge of the thread and obtain the image img1; Step 2) Use Harris corner detection algorithm to extract the corner points of the thread on img1, and obtain the corner point set P = {p1, p2, ..., p n }; Step 3) Classify the tooth apex and tooth bottom points on the upper and lower sides of the set P according to the coordinate relationship of the corner points; Step 4) Find the fitting tangent line of the upper and lower tooth vertices: Let the least square method The value of is minimized, where e, x, and y are the sum of squared errors and the horizontal and vertical coordinates of the tooth vertex, respectively, and the optimal slope m and intercept b of the fitting tangent are obtained; Step 5) Calculate the average distance from each tooth vertex on the upper and lower sides to the lower and upper fitting tangent lines, which is the thread fitting major diameter d: Where n1 and n2 are the number of upper and lower lateral tooth vertices, respectively; Step 6) Calculate the distances between the top and bottom points of the upper and lower teeth respectively, and then find the average value, which is the fitting pitch p: Where |·| represents the distance between two points, n1, n2, n3, and n4 are the number of tooth apex and tooth bottom points on the upper and lower sides respectively; Step 7) Calculate the thread angle α using the vector dot product formula: Where <·> represents the angle formed by the three corner points.

6. The multi-modal dual-link thread measurement method according to claim 5, characterized in that: The method of classifying the tooth apex and tooth bottom points on the upper and lower sides according to the coordinate relationship of the corner points includes: Find the average ordinate of all corner points The corner point with a ordinate greater than the average ordinate is the upper corner point Otherwise it is the lower corner point Calculate the average ordinate of the upper and lower corner points respectively. The upper corner point with a greater ordinate than the average ordinate is the tooth vertex. Otherwise it is the bottom point of the tooth The lower corner point with a larger vertical coordinate than the average is the tooth bottom point Otherwise it is the tooth apex 7. The multi-modal dual-link thread measurement method according to claim 1, characterized in that: The D specifically includes the following steps: Step 1) Connect the thread shaft and head images in the channel dimension to form a multidimensional matrix img; Step 2) inputting the multidimensional matrix img into a deep feature learning network, wherein the deep feature learning network includes a feature learning module, a feature evaluation module and a feature classification module; Step 3) extracting thread features through a feature learning module; Step 4) Performing quality evaluation on the extracted features through the feature evaluation module; Step 5) Classify the extracted features through the feature classification block.

8. The multi-modal dual-link thread measurement method according to claim 7, characterized in that: In the step 4): The feature evaluation module consists of one encoding layer, one large fully connected layer, and three small fully connected layers. The encoding layer is used to integrate the extracted features into a one-dimensional vector. The extracted features are multidimensional matrices with a size of a×a×n, where a×a is the matrix size and n is the dimension. The integration method is to connect the matrices of each dimension from left to right, from top to bottom, and from beginning to end to form a one-dimensional vector, and then randomly sample and remove redundant elements of the one-dimensional vector. The re-encoded features are input into a large-size fully connected layer with a parameter of 1024×100, indicating that the input is a 1024×1 one-dimensional vector and the output is a 100×1 vector. After further optimizing the features through the large-size fully connected layer, the optimized features are input into three small-size fully connected layers with a parameter of 100×1. The three small connected layers evaluate the quality of the optimized features and finally output a quality score of 0 to 1 for the pitch, major diameter and tooth angle of the thread.

9. The multi-modal dual-link thread measurement method according to claim 8, characterized in that: According to the score of the feature quality evaluation module, the geometric parameters obtained from the two links of manual features and deep features are fused. The fusion formula is: Where δ, ε, and θ are the output scores of the evaluation module respectively, representing the weights of the pitch, major warp, and tooth angle results in the deep feature link, p1, d1, and α1 represent the classification results of the deep feature link, and p2, d2, and α2 represent the measurement results of the manual feature link.

Images