Flange size measurement method and system based on sub-pixel accuracy of machine vision

Abstract

The present invention provides a flange size measurement method and system with sub-pixel precision based on machine vision, wherein the method comprises: calibrating the size of a CCD camera; collecting a flange image of a flange to be measured by the calibrated CCD camera; preprocessing the flange image; performing edge detection on the preprocessed flange image to obtain sub-pixel coordinates of the flange edge to be measured; and determining the size of the flange to be measured based on the sub-pixel coordinates of the flange edge to be measured. The flange size measurement method and system with sub-pixel precision based on machine vision of the present invention performs edge detection on the preprocessed flange image to obtain sub-pixel coordinates of the flange edge to be measured, and determines the size of the flange to be measured based on the sub-pixel coordinates of the flange edge to be measured, thereby improving the detection accuracy and detection efficiency of the flange size detection, reducing the false detection rate, and ensuring the product quality of the flange.

Description

Technical Field

[0001] The present invention relates to the technical field of machine vision, and in particular to a flange size measurement method and system with sub-pixel accuracy based on machine vision. Background Art

[0002] At present, automobile flanges are an important part of the automobile suspension system, used to connect the wheel axle to the suspension system. The flanges provide firm support and stable fixation for the wheel axle, ensuring that the wheels can rotate smoothly, and effectively absorbing and transmitting various forces and vibrations generated during vehicle driving. However, during the production process, due to reasons such as materials and processes, the flange size deviation is often too large, resulting in insufficient flange size accuracy. At present, companies mainly rely on manual sampling of inspection tools or measuring tools for flange size detection, which not only has low detection efficiency, but also has a high false detection rate, and cannot fully guarantee product quality.

[0003] Therefore, a solution is urgently needed. Summary of the invention

[0004] One of the purposes of the present invention is to provide a flange size measurement method with sub-pixel accuracy based on machine vision, perform edge detection on the preprocessed flange image, obtain the sub-pixel level coordinates of the edge of the flange to be measured, and determine the size of the flange to be measured based on the sub-pixel level coordinates of the edge of the flange to be measured, thereby improving the detection accuracy and efficiency of the flange size detection, reducing the false detection rate, and ensuring the product quality of the flange.

[0005] The flange size measurement method with sub-pixel accuracy based on machine vision provided by an embodiment of the present invention includes:

[0006] Calibrate the size of the CCD camera;

[0007] The flange image of the flange to be measured is collected by a CCD camera after dimension calibration;

[0008] Preprocessing the flange image;

[0009] Perform edge detection on the preprocessed flange image to obtain the sub-pixel coordinates of the flange edge to be measured;

[0010] The size of the flange to be measured is determined based on the sub-pixel coordinates of the edge of the flange to be measured.

[0011] Preferably, the flange image is preprocessed, including:

[0012] Grayscale processing is performed on the flange image; the grayscale processing method at least includes: weighted average method;

[0013] Performing filtering and denoising on the flange image after grayscale processing; the filtering and denoising processing methods at least include: Gaussian filtering, median filtering, and mean filtering;

[0014] The flange image after filtering and denoising is subjected to image segmentation processing; the image segmentation processing method at least includes: segmentation based on threshold value.

[0015] Preferably, edge detection is performed on the preprocessed flange image to obtain sub-pixel coordinates of the flange edge to be measured, including:

[0016] Based on the improved Canny-Zernike moment sub-pixel edge detection algorithm, edge detection is performed on the preprocessed flange image to obtain the sub-pixel coordinates of the flange edge to be measured.

[0017] Preferably, determining the size of the flange to be measured based on the sub-pixel coordinates of the edge of the flange to be measured includes:

[0018] The size of the flange to be measured is fitted according to the sub-pixel coordinates of the edge of the flange to be measured based on the least squares method.

[0019] Preferably, the flange size measurement method with sub-pixel accuracy based on machine vision also includes:

[0020] When the number of dimension measurement abnormal events occurring in the first time period is greater than or equal to the number threshold, the detection multimodal data is acquired; the dimension measurement abnormal event includes: the deviation between the dimension of the flange to be measured and the predetermined dimension is greater than or equal to the deviation threshold; the detection multimodal data includes: dimension detection traceability information of the CCD camera in the first time period, and the product production traceability information of the flange to be measured in the production dimension measurement abnormal event;

[0021] Based on the first feature processing template, feature processing is performed on the detection multimodal data to obtain a first feature description vector;

[0022] Determine an attribution strategy based on the first feature description vector;

[0023] Based on the attribution strategy, the attribution results are determined according to the detection of multimodal data;

[0024] When the attribution result is size detection, stop continuing to measure the size of the remaining flanges to be detected in the second time period; the second time period is after the first time period and adjacent to the first time period;

[0025] When the attribution result is product production, stop continuing to measure the dimensions of the remaining flanges to be inspected from the same production batch as the flange to be measured in the production dimension measurement abnormality event within the second time period, and issue a recall warning for the flanges that have been measured for the dimensions of the same production batch as the flange to be measured in the production dimension measurement abnormality event.

[0026] Preferably, determining the attribution strategy based on the first feature description vector includes:

[0027] Determining an analysis value corresponding to the first feature description vector from a situation analysis library;

[0028] When the analysis value is less than or equal to the analysis threshold, determining the attribution strategy corresponding to the first feature description vector from the attribution strategy library;

[0029] When the analysis value is greater than the analysis threshold, determining the crawling strategy corresponding to the first feature description vector from the crawling strategy library;

[0030] Crawling big data knowledge based on crawling strategies;

[0031] Based on big data knowledge, build a big data knowledge graph;

[0032] Determine attribution strategy based on big data knowledge graph.

[0033] Preferably, crawling big data knowledge based on crawling strategy includes:

[0034] Analyze the access scenarios, content search rules, and content extraction rules of the crawling strategy;

[0035] Based on the factor generation template, preliminary factors are generated according to the content search rules and content extraction rules;

[0036] Pre-access the access scene, and obtain the pre-access content from the access scene based on the pre-access factor;

[0037] Based on the second feature processing template, feature processing is performed on the prepared content to obtain a second feature description vector;

[0038] Determining the permitted value and additional content corresponding to the second feature description vector from the permitted analysis library;

[0039] When the permission value is greater than or equal to the permission threshold, the access scene is accessed, and the first target content is searched from the access scene based on the content search rule during the access;

[0040] When the additional content is empty, extracting big data knowledge from the first target content based on the content extraction rule;

[0041] When the additional content includes at least one set of one-to-one corresponding association relationship and content condition, searching for second target content having any association relationship with the first target content from the access scenario, and determining whether the second target content meets the content condition corresponding to any association relationship;

[0042] When it meets the requirements, big data knowledge is extracted from the first target content based on the content extraction rules.

[0043] Preferably, based on big data knowledge, a big data knowledge graph is constructed, including:

[0044] Analyze the knowledge types of big data knowledge;

[0045] Determine the mapping node corresponding to the knowledge type from the mapping node table;

[0046] Set the big data knowledge at the mapping nodes in the initial graph;

[0047] The initial graph after all mapping nodes that need to set big data knowledge are set with corresponding big data knowledge is used as the big data knowledge graph.

[0048] Preferably, the attribution strategy is determined based on the big data knowledge graph, including:

[0049] Determine the support corresponding to multiple hypothetical attribution results from the big data knowledge graph; the support is the ratio of the total number of big data knowledge in the big data knowledge graph that matches the hypothetical attribution results to the total number of big data knowledge in the big data knowledge graph;

[0050] Based on the hypothesis attribution results corresponding to the maximum support, determine the search rules for strategy formulation;

[0051] Based on the search rules for the basis for policy formulation, search for the basis for policy formulation in the big data knowledge graph;

[0052] Develop an attribution strategy based on the strategy formulation basis.

[0053] The flange size measurement system with sub-pixel accuracy based on machine vision provided by an embodiment of the present invention comprises:

[0054] Camera calibration module, used to calibrate the size of the CCD camera;

[0055] An image acquisition module is used to acquire the flange image of the flange to be measured through a size-calibrated CCD camera;

[0056] A preprocessing module, used for preprocessing the flange image;

[0057] An edge detection module is used to perform edge detection on the preprocessed flange image to obtain the sub-pixel coordinates of the flange edge to be measured;

[0058] The size determination module is used to determine the size of the flange to be measured based on the sub-pixel level coordinates of the edge of the flange to be measured.

[0059] Other features and advantages of the present invention will be described in the following description, and partly become apparent from the description, or understood by practicing the present invention. The purpose and other advantages of the present invention can be realized and obtained by the structures particularly pointed out in the written description, claims, and drawings.

[0060] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings:

[0062] Figure 1 Schematic diagram of a flange size measurement method with sub-pixel accuracy based on machine vision in an embodiment of the present invention;

[0063] Figure 2 is a schematic diagram of an ideal edge step model in an embodiment of the present invention;

[0064] Figure 3 The original image and grayscale image of the outer circle and inner hole image of the flange in the embodiment of the present invention;

[0065] Figure 4 Schematic diagram of image comparison after filtering in an embodiment of the present invention;

[0066] Figure 5 is a grayscale histogram in an embodiment of the present invention;

[0067] Figure 6 is a binary segmentation map in an embodiment of the present invention;

[0068] Figure 7 The extraction diagrams of the flange edge extraction performed by the traditional algorithm and the improved algorithm in the embodiment of the present invention respectively;

[0069] Figure 8 Schematic diagram of a flange size measurement system with sub-pixel accuracy based on machine vision in an embodiment of the present invention. DETAILED DESCRIPTION

[0070] The preferred embodiments of the present invention are described below in conjunction with the accompanying drawings. It should be understood that the preferred embodiments described herein are only used to illustrate and explain the present invention, and are not used to limit the present invention.

[0071] The embodiment of the present invention provides a flange size measurement method with sub-pixel accuracy based on machine vision, such as Figure 1 As shown, including:

[0072] S1. Calibrate the size of the CCD camera;

[0073] S2, collecting the flange image of the flange to be measured by using a CCD camera after size calibration;

[0074] S3, preprocessing the flange image;

[0075] S4, performing edge detection on the preprocessed flange image to obtain sub-pixel coordinates of the flange edge to be measured;

[0076] S5. Determine the size of the flange to be measured based on the sub-pixel coordinates of the edge of the flange to be measured.

[0077] Preprocess the flange image, including:

[0078] Grayscale processing is performed on the flange image; the grayscale processing method at least includes: weighted average method;

[0079] Performing filtering and denoising on the flange image after grayscale processing; the filtering and denoising processing methods at least include: Gaussian filtering, median filtering, and mean filtering;

[0080] The flange image after filtering and denoising is subjected to image segmentation processing; the image segmentation processing method at least includes: segmentation based on threshold value.

[0081] Perform edge detection on the preprocessed flange image to obtain the sub-pixel coordinates of the flange edge to be measured, including:

[0082] Based on the improved Canny-Zernike moment sub-pixel edge detection algorithm, edge detection is performed on the preprocessed flange image to obtain the sub-pixel coordinates of the flange edge to be measured.

[0083] The size of the flange to be measured is fitted according to the sub-pixel coordinates of the edge of the flange to be measured based on the least squares method.

[0084] The working principle and beneficial effects of the above technical solution are:

[0085] Edge detection is performed on the preprocessed flange image to obtain the sub-pixel coordinates of the edge of the flange to be measured. Based on the sub-pixel coordinates of the edge of the flange to be measured, the size of the flange to be measured is determined, which improves the detection accuracy and efficiency of the flange size detection, reduces the false detection rate, and ensures the product quality of the flange.

[0086] Zernike moments are orthogonal functions based on Zernike polynomials. Zernike orthogonal moments are widely used. They are rotationally invariant and have low sensitivity to noise. By calculating the step grayscale value, background grayscale value, vertical distance from the center of the circle to the edge of the image, and the angle between the X-axis and the vertical line of the image edge of the Zernike moment edge model, the sub-pixel coordinates can be obtained to obtain the precise image edge.

[0087] The nth-order and mth-order Zernike moment Z of a continuous image f(x, y) nm The definition is shown in formula (1):

[0088]

[0089] Where: x and y satisfy x+y≦l; is the nth order and mth degree Zernike moment polynomial V nm The complex conjugate of (ρ,θ), where ρ is the length of the vector from the origin to the point (x, y), and θ is the angle between the vector and the counterclockwise direction of the X-axis.

[0090] Under discrete conditions, the Zernike moment is as shown in formula (2):

[0091]

[0092] The ideal edge step model is as follows Figure 2 shown. Figure 2 In the equation: (x0, y0) is the origin coordinate; L is the ideal edge of the image; k is the step amplitude between the background and the target area; h is the gray value of the image background; l is the vertical distance between the center of the circle and L; α is the angle between line l1 and the X-axis.

[0093] If the edge is rotated by -α, the Zernike moment after rotation and the Zernike moment Z before rotation nm The relationship is shown in formula (3):

[0094] Z′ nm =Z nm e -pmα (3)

[0095] Where: p is the imaginary unit.

[0096] Using Z 11 The imaginary part Im[Z 11 ] and the real part Re[Z 11 ], the rotation angle, that is, the angle α, can be obtained, and the result is shown in formula (4):

[0097]

[0098] Derived Z′00 , Z′ 11 and Z′ 20 , as shown in formula (5):

[0099]

[0100] The parameters h, k and l can be obtained from the formula, as shown in formula (6):

[0101]

[0102] Finally, the sub-pixel edge detection formula is obtained, as shown in formula (7):

[0103]

[0104] Where: (x s ,y s ) are sub-pixel edge coordinates.

[0105] Then, when the unit circle is sampled using N×N templates in the discrete case, the calculation formula for the Zernike moment sub-pixel coordinates is as shown in formula (8):

[0106]

[0107] The Canny algorithm is a widely used edge detection algorithm, which mainly consists of four steps: convolution smoothing the image with a two-dimensional Gaussian template; calculating the gradient amplitude and direction of each pixel; non-maximum suppression of the gradient amplitude; double threshold algorithm detection and connection of edges. The traditional Canny algorithm has many shortcomings: it ignores edge protection when using Gaussian filtering; using a 2x2 neighborhood to calculate the gradient is easily affected by noise, resulting in the loss of the target edge; and the manually set double threshold has low adaptability.

[0108] The traditional Canny operator was improved in the process of rough extraction of pixel-level edges. The improved algorithm optimized the image input process. At the same time, improvements were made to the deficiencies in the traditional algorithm image denoising process, gradient amplitude calculation, and threshold selection method. Ultimately, the obtained pixel-level edge points had better continuity and the edge information was more accurate, which provided a basis for the subsequent fine positioning of the edges and reduced the workload of subsequent sub-pixel precise positioning.

[0109] During the image acquisition process, noise will be generated due to factors such as the external environment. Since Gaussian filtering will ignore the protection of edges, in order to ensure the integrity of the edges, this system uses bilateral filtering to denoise the image. Bilateral filtering is a nonlinear filtering method that adds pixel value weights to Gaussian filtering, while considering spatial information and grayscale similarity, maintaining the edge information of the image target and achieving noise reduction effects.

[0110] Bilateral filtering is shown in formula (9), where i and j are the coordinate points of the current convolved pixel, and k and l are the coordinate points of the domain pixel.

[0111]

[0112] In the edge detection algorithm, the size of the gradient can well reflect the pixel changes of the image. Based on the traditional Sobel operator, this application adds 45° and 135° direction templates, and transforms the direction template into four directions: horizontal and vertical directions, 45° and 135° directions. The first-order partial derivatives in the x, y, 45° and 135° directions are calculated in the pixel 8 neighborhood, so the improved template is more accurate in edge positioning and can better suppress noise than the traditional template. The specific template is as follows:

[0113]

[0114] In formula (10), Hx, Hy, H45°, and H135° represent the horizontal, vertical, 45°, and 135° directions of the gradient template, respectively. The original image after filtering is convolved with the multi-directional Canny template to obtain the gradient components in four directions. The gradient calculation process is as follows:

[0115]

[0116]

[0117] Formula (11) is the partial derivative of the gradient template in each direction. In formula (12), θ is the gradient direction, and f x (i, j) and f y (i, j) represent the horizontal direction and the vertical direction respectively. The gradient weighting coefficients are represented by α and β respectively. The gradients in each direction are Mx°, My°, M45° and M135° respectively. Finally, the final gradient M can be obtained.

[0118] The edge extracted only by the gradient value is still very fuzzy. The purpose of non-maximum suppression is to refine the edge. The basic method is to compare the gradient intensity of the current pixel with the gradient values ​​on both sides of the gradient direction. If it is an extreme point, the pixel is retained as an edge point; if not, it is a non-edge point.

[0119] This application uses a more accurate method to determine whether a pixel can form an edge. In addition to considering the gradient amplitude of the pixel, the direction of the gradient is also considered. Specifically, if the gradient amplitude of the current pixel in the x direction is greater than the gradient amplitude in the y direction, then the gradient in the x direction is used for judgment; otherwise, the gradient in the y direction is used for judgment. By considering the direction of the gradient, we can better capture the edge of the image, thereby improving the accuracy of edge detection.

[0120] After completing the maximum suppression, the image still contains some noise and false edges. The double threshold method is used in the Canny algorithm to reduce false edges. In order to solve the problem of the difficulty of selecting double thresholds, we improved the threshold selection method of the traditional Canny algorithm and used the Otsu method to obtain the adaptive double threshold to optimize the shortcomings of the traditional manual experience threshold selection. The specific algorithm is as follows:

[0121] Assume there is a threshold T h Divide all pixels of the image into two categories, C1 and C2. The grayscale means of C1 and C2 are m1 and m2 respectively. The global average grayscale value of the image is m. The probability of pixels being classified into C1 and C2 is p1 and p2 respectively. Then:

[0122]

[0123] σ 2 =p1(m1-m) 2 +p2(m2-m) 2 (14)

[0124] When σ 2 When the maximum value is reached, the optimal threshold value T is obtained. h As the high threshold of the dual threshold, select 0.5×T h , as the lower threshold of the dual threshold.

[0125] When calculating the Zernike moment, the traditional algorithm identifies the centroid of the image as the coordinate origin and projects the entire image into the unit circle. The improved Canny-Zernike sub-pixel edge detection algorithm does not need to traverse the entire image. Therefore, the sub-pixel edge detection method can improve the detection speed.

[0126] The value of the Zernike moment obtained by using Zernike templates of different orders directly affects the accuracy of image edge detection in the sub-pixel detection process. Generally speaking, the larger the template size, the higher the detection accuracy will be, but the time will also be longer. Taking into account the needs of the paper, a 7ⅹ7 Zernike moment parameter template will be used to further improve the detection accuracy of the algorithm.

[0127] The improved algorithm is based on the Ghosal algorithm. 00 , Z 11 , Z 20 Expand to Z 31 , Z 40 Coefficient template, at the same time, adopt the new edge judgment condition k≥k t ∩∣l2-l1∣≦l t Instead of the traditional edge judgment condition k≥k t ∩l2-l1≦l tThe specific algorithm is as follows:

[0128]

[0129] From equations (12), (15) and (16), we can deduce:

[0130]

[0131] The specific implementation steps of the improved algorithm are as follows:

[0132] Step 1: Use the traditional Zernike moment detection algorithm to obtain five different orders of Zernike moments Z 00 , Z 11 , Z 20 , Z 31 , Z 40 ;

[0133] Step 2: According to the rotational invariance property of Zernike orthogonal moments, calculate Z′ 00 , Z′ 11 , Z′ 20 , Z′ 31 , Z′ 40 ;

[0134] Step 3: According to Z′ 00 , Z′ 11 , Z′ 20 , Z′ 31 , Z′ 40 Derive l1, l2 and k, and calculate l;

[0135] Step 4: Substitute l into the formula of k and h to calculate k;

[0136] Step 5: Determine whether the parameters of each edge pixel meet the judgment condition, that is, k ≥ k t and

[0137] Step 6: Use a new sub-pixel edge judgment criterion k ≥ k t ∩∣l2-l1∣≦l t Then, the pixel-level rough edge points obtained by using the improved Canny operator are identified. Those that meet the requirements are edge pixels, and those that do not meet the requirements are not.

[0138] Dimension measurement requires converting the measured pixel size into the actual size of the workpiece, so the conversion relationship between pixel size and actual size is required. In order to ensure the accuracy of the measurement results, a 6mmx6mm black and white chessboard calibration plate is used to calibrate the camera. The camera is used to collect 20 calibration plate images at different angles in the measurement area, and two corner points are found in each image. The actual distance and pixel distance between the two points are calculated respectively. The calibration formula is:

[0139]

[0140] Where L is the actual size between two points on the calibration plate; l is the pixel size between two points on the calibration plate.

[0141] The average value of the 20 groups of K values ​​is the calibration coefficient K, which actually means that the physical size of a pixel is 0.08421 mm / pixel.

[0142] The hardware part of the detection system is mainly composed of computers, industrial cameras, lenses, backlights, ring light sources and test benches. The detection process is: use an industrial camera to capture the image of the flange end face, and the captured image is transmitted to the computer through the camera device driver. The camera device driver can be a USB interface or other communication interface, which is used to transmit static image data and video stream frame data to the computer. The computer uses pycharm software and various algorithms to process the image (such as grayscale, filtering and noise reduction, image segmentation, edge detection, etc.), and then fits the processed image to obtain the measurement result.

[0143] Since the camera collects a three-channel color image, it contains a large amount of features and calculations, so direct analysis takes a long time. The grayscale image will become a single-channel image, which can increase the calculation speed. Here, the weighted average method is used to achieve grayscale. The original image and grayscale image of the outer circle and inner hole of the blue plate are as follows: Figure 3 As shown, Figure 3 In the figure, the left side is the original image and the right side is the grayscale image.

[0144] Since the noise mixed into the original image generation process will reduce the image quality, resulting in unclear image details, which directly affects the effects of image segmentation, edge detection, etc., it is necessary to adopt appropriate methods to eliminate noise as much as possible to maximize the image quality. Among the commonly used filtering methods, median filtering has the characteristics of protecting the edge information of the image while removing the useless image noise contained in the image. This application performs three different filtering processes on the flange image, and the effects are compared. Figure 4 As shown, in Figure 4 In the figure, from left to right are the images after Gaussian filtering, median filtering, and mean filtering.

[0145] Using image segmentation to obtain the region of interest can simplify the subsequent image processing process. The threshold-based segmentation method has a significant effect on images with a strong contrast between the foreground and the background. The flange in this application has a clear contrast with the background, so a threshold-based segmentation method is used to achieve image segmentation: Threshold-based segmentation is a method of binary segmentation, that is, directly giving a threshold for binary segmentation.

[0146] Use the histogram to display the grayscale distribution of the image. Figure 5 The grayscale histogram shown in Figure 2 shows that the segmentation effect is better when T = 88 is selected. Figure 6 shown.

[0147] In order to make a qualitative analysis of the traditional algorithm and the improved algorithm intuitively, the traditional and improved Zernike algorithms are used to Figure 6 Sub-pixel edge point extraction of the inner and outer diameter contours of the parts. Figure 7 The left picture in the middle is the flange edge extraction diagram using the traditional algorithm, and the right picture is the flange outer diameter edge extraction diagram using the improved algorithm. Figure 7 It can be concluded that the algorithm before improvement can fuzzily detect the edge contour of the flange, but compared with the improved algorithm, some key point information is lost, there is more edge noise, and contour point breaks appear on some edges. Therefore, the improved algorithm is more capable of dimensional measurement and has higher extraction accuracy.

[0148] In one embodiment, the flange size measurement method with sub-pixel accuracy based on machine vision further includes:

[0149] When the number of dimension measurement abnormal events occurring in the first time period is greater than or equal to the number threshold, the detection multimodal data is acquired; the dimension measurement abnormal event includes: the deviation between the dimension of the flange to be measured and the predetermined dimension is greater than or equal to the deviation threshold; the detection multimodal data includes: dimension detection traceability information of the CCD camera in the first time period, and the product production traceability information of the flange to be measured in the production dimension measurement abnormal event;

[0150] Based on the first feature processing template, feature processing is performed on the detection multimodal data to obtain a first feature description vector;

[0151] Determine an attribution strategy based on the first feature description vector;

[0152] Based on the attribution strategy, the attribution results are determined according to the detection of multimodal data;

[0153] When the attribution result is size detection, stop continuing to measure the size of the remaining flanges to be detected in the second time period; the second time period is after the first time period and adjacent to the first time period;

[0154] When the attribution result is product production, stop continuing to measure the dimensions of the remaining flanges to be inspected from the same production batch as the flange to be measured in the production dimension measurement abnormality event within the second time period, and issue a recall warning for the flanges that have been measured for the dimensions of the same production batch as the flange to be measured in the production dimension measurement abnormality event.

[0155] The first time period is an arbitrary time period of, for example, 100 seconds, for measuring the dimensions of multiple flanges to be measured that are waiting in line for inspection; the second time period may be, for example, 50 seconds; the quantity threshold may be, for example, 5; the flanges to be measured all have predetermined dimensions, which are determined by the production standard of the flanges to be measured; the deviation between the dimensions of the flanges to be measured and the predetermined dimensions is the ratio of the dimension difference to the predetermined dimensions; the deviation threshold may be, for example, 0.001%; ​​when the number of dimension measurement abnormality events occurring in the first time period is greater than or equal to the quantity threshold, it indicates that a batch abnormality has occurred in flange inspection and attribution is required; the dimension inspection traceability information is the working status information of the CCD camera in the first time period, such as: working temperature, white balance parameters, gamma correction parameters, etc.; the product production traceability information is the investment information of the production equipment of the flanges to be measured in the dimension measurement abnormality event. The service life, the last maintenance time, the historical abnormality records, etc. are input; the first feature processing template is used to extract the information features of the detection multimodal data, and the first feature description vector is constructed in the form of a vector; based on the first feature description vector, the attribution strategy is determined, and the attribution strategy indicates how to attribute the batch abnormalities of flange detection according to the detection multimodal data. Based on the attribution strategy, the attribution result is determined according to the detection multimodal data. The attribution result includes dimension detection and product production. If it is dimension detection, it means that it is necessary to stop continuing to measure the dimension of the remaining flanges to be detected. Otherwise, stop continuing to measure the dimension of the remaining flanges to be detected from the same production batch as the flanges to be measured in the production dimension measurement abnormality event, and issue a recall warning for the flanges that have been measured in the same production batch as the flanges to be measured in the production dimension measurement abnormality event.

[0156] The present application has realized the size of the flange without manual measurement. However, in the process of the system automatically measuring the size of the flange, the measurement may be abnormal. At this time, it is necessary to immediately attribute and take corresponding processing measures. However, in the specific implementation, if a measurement abnormality is encountered, no matter what the reason is, it is necessary to manually pause the system to continue the size measurement of the flange, which seriously affects the detection progress of the flange. In addition, the staff is required to investigate the cause, and the manpower cost is large. The embodiment of the present invention can solve these problems. When the number of size measurement abnormal events in the first time period is greater than or equal to the quantity threshold, the detection multimodal data is obtained, the first feature description vector of the detection multimodal data is constructed, and the attribution strategy is determined based on the first feature description vector. The attribution strategy is used to quickly attribute and take corresponding measures. There is no need to suspend the system for a long time to continue the size measurement of the flange, which greatly reduces the impact on the detection progress of the flange. In addition, there is no need for the staff to investigate the cause, which reduces the manpower cost and improves the applicability of the system.

[0157] In one embodiment, determining an attribution strategy based on the first feature description vector includes:

[0158] Determining an analysis value corresponding to the first feature description vector from a situation analysis library;

[0159] When the analysis value is less than or equal to the analysis threshold, determining the attribution strategy corresponding to the first feature description vector from the attribution strategy library;

[0160] When the analysis value is greater than the analysis threshold, determining the crawling strategy corresponding to the first feature description vector from the crawling strategy library;

[0161] Crawling big data knowledge based on crawling strategies;

[0162] Based on big data knowledge, build a big data knowledge graph;

[0163] Determine attribution strategy based on big data knowledge graph.

[0164] Based on big data knowledge, build a big data knowledge graph, including:

[0165] Analyze the knowledge types of big data knowledge;

[0166] Determine the mapping node corresponding to the knowledge type from the mapping node table;

[0167] Set the big data knowledge at the mapping nodes in the initial graph;

[0168] The initial graph after all mapping nodes that need to set big data knowledge are set with corresponding big data knowledge is used as the big data knowledge graph.

[0169] The attribution strategy is determined based on the big data knowledge graph, including:

[0170] Determine the support corresponding to multiple hypothetical attribution results from the big data knowledge graph; the support is the ratio of the total number of big data knowledge in the big data knowledge graph that matches the hypothetical attribution results to the total number of big data knowledge in the big data knowledge graph;

[0171] Based on the hypothesis attribution results corresponding to the maximum support, determine the search rules for strategy formulation;

[0172] Based on the search rules for the basis for policy formulation, search for the basis for policy formulation in the big data knowledge graph;

[0173] Develop an attribution strategy based on the strategy formulation basis.

[0174] There are analysis values ​​corresponding to different first feature description vectors in the situation analysis library. The analysis values ​​represent the complexity of attribution of the detection multimodal data response. The larger the analysis value, the greater the complexity. The technicians can pre-set the analysis values ​​corresponding to different first feature description vectors to establish a situation analysis library. For example: if there are many types of data in the detection multimodal data constructed into the first feature description vector, and there are also multiple values ​​of the same type of data, it means that the complexity of attribution is relatively large, and a larger analysis value is set for the corresponding first feature description vector; the analysis threshold can be, for example: 8; when the analysis value is less than or equal to the analysis threshold, it means that the complexity of attribution is relatively small, and the attribution strategy corresponding to the first feature description vector is directly determined based on the attribution strategy library, and the attribution strategy library does not contain The attribution strategy corresponding to the first feature description vector can be set in advance by the technician. The first feature description vector reflects the current situation that needs to be attribution, and the attribution strategy indicates how to attribute the situation. When the analysis value is greater than the analysis threshold, it means that the complexity of attribution is relatively large. The crawling strategy corresponding to the first feature description vector is determined from the crawling strategy library. The crawling strategy is a strategy for crawling big data knowledge that can be used to determine the attribution strategy. There are crawling strategies corresponding to different first feature description vectors in the crawling strategy library. The technician can set it in advance. The first feature description vector reflects the current situation that needs to be attribution, and the crawling strategy knowledge indicates what knowledge needs to be collected for the situation. Based on the big data knowledge, a big data knowledge graph is constructed. Based on the big data knowledge graph, the attribution strategy is determined.

[0175] There are a large number of main nodes at different levels in the initial graph, and the nodes at each level are also connected to multiple child nodes. The main node sets the attribution conclusion, and multiple attribution processes for drawing the attribution conclusion are set on the multiple child nodes connected to the main node; therefore, the knowledge types are divided into attribution conclusions and attribution processes, and there are mapping nodes corresponding to different knowledge types in the mapping node table, and the mapping nodes are main nodes and child nodes; the initial graph after all the mapping nodes that need to set big data knowledge are set with the corresponding big data knowledge is used as the big data knowledge graph.

[0176] The hypothesized attribution results are divided into three types: 1. Dimension detection; 2. Product production; 3. Combination of dimension detection and product production; determine the support corresponding to multiple hypothesized attribution results from the big data knowledge graph, the support is the ratio of the total number of big data knowledge in the big data knowledge graph that matches the hypothesized attribution results to the total number of big data knowledge in the big data knowledge graph, the total number of big data knowledge in the big data knowledge graph only counts the number of main nodes with attribution conclusions set in the big data knowledge graph; the big data knowledge that matches the hypothesized attribution results is the attribution conclusion that matches the hypothesized attribution results; based on the hypothesized attribution results corresponding to the maximum support, determine the search rules for strategy formulation, the search rules for strategy formulation are to search for the same attribution process on the sub-nodes connected to the main node where the attribution conclusion that matches the hypothesized attribution results corresponding to the maximum support is located, as the basis for strategy formulation; based on these same attribution processes, an attribution strategy can be formulated.

[0177] The embodiment of the present invention introduces a situation analysis library to determine the analysis value, which is used to quickly determine the complexity of attribution. When the complexity is small, the attribution strategy is directly determined based on the local attribution strategy library. When the complexity is large, a big data knowledge graph is constructed to comprehensively, comprehensively and accurately determine the attribution strategy, which greatly improves the applicability of the system. When constructing a big data knowledge graph, an initial graph is introduced, and mapping nodes are determined based on the knowledge type of big data knowledge. The mapping setting of big data knowledge is performed in the initial graph, which improves the efficiency of constructing the big data knowledge graph. When using the big data knowledge graph to determine the attribution strategy, support is introduced, and based on the hypothetical attribution result corresponding to the maximum support, a reverse search is performed on the basis for formulating the strategy, and the attribution strategy is determined based on the basis for formulating the strategy, which greatly improves the accuracy and efficiency of determining the attribution strategy.

[0178] In one embodiment, crawling big data knowledge based on a crawling strategy includes:

[0179] Analyze the access scenarios, content search rules, and content extraction rules of the crawling strategy;

[0180] Based on the factor generation template, preliminary factors are generated according to the content search rules and content extraction rules;

[0181] Pre-access the access scene, and obtain the pre-access content from the access scene based on the pre-access factor;

[0182] Based on the second feature processing template, feature processing is performed on the prepared content to obtain a second feature description vector;

[0183] Determining the permitted value and additional content corresponding to the second feature description vector from the permitted analysis library;

[0184] When the permission value is greater than or equal to the permission threshold, the access scene is accessed, and the first target content is searched from the access scene based on the content search rule during the access;

[0185] When the additional content is empty, extracting big data knowledge from the first target content based on the content extraction rule;

[0186] When the additional content includes at least one set of one-to-one corresponding association relationship and content condition, searching for second target content having any association relationship with the first target content from the access scenario, and determining whether the second target content meets the content condition corresponding to any association relationship;

[0187] When it meets the requirements, big data knowledge is extracted from the first target content based on the content extraction rules.

[0188] The access scenario is the scenario that needs to be accessed when executing the crawling strategy, such as: CCD camera research sharing forum, flange production technology exchange group, etc.; the content search rule is the rule for searching for content of big data knowledge to be extracted that is valuable for attribution, and the content extraction rule is the rule for extracting big data knowledge from the search content; before officially accessing the access scenario, it can be pre-accessed. During the pre-access, preparatory content is obtained from the access scenario based on the preparatory factor. Pre-access refers to scene access that occupies fewer access resources. The preparatory factor is formed by the feature sorting of the content search rule and the content extraction rule. When obtaining the preparatory content, the content related to the preparatory factor in the access scenario is obtained as the preparatory content; similarly to the method of using the first feature processing template, a second feature description vector of the preparatory content is constructed. The second feature description vector reflects the situation that the access scenario contains the big data knowledge that the crawling strategy wants to crawl. The permitted analysis library has allowed values ​​and additional content corresponding to different second feature description vectors. The second feature description vector reflects the access. The more the scenario contains the big data knowledge that the crawling strategy wants to crawl, the more worthy it is to formally visit and crawl the big data knowledge, the larger the corresponding permission value, the additional content can be empty, and it can also contain at least one set of one-to-one corresponding association relationships and content conditions. When it contains at least one set of one-to-one corresponding association relationships and content conditions, the second target content in the access scenario that has any association relationship with the first target content needs to meet the corresponding content conditions to extract big data knowledge. For example: the first target content is the process of publisher A introducing the causes of flange production anomalies in the flange production technology exchange group, and the second target content with an association relationship is the number of historical publications of publisher A, the completeness of personal authentication information, etc. The content conditions that need to be met are that the number of historical publications of publisher A is ≥20 and the completeness of personal authentication information is ≥85%. Only in this way can it be shown that the credibility of the process of publisher A introducing the causes of flange production anomalies in the flange production technology exchange group is high, and big data knowledge can be extracted.

[0189] The embodiment of the present invention crawls big data knowledge based on a crawling strategy, first pre-visits the access scenario, introduces a preparatory factor to obtain preparatory content, determines the permitted value and additional content based on the permitted analysis library, and performs subsequent direct access and extraction of big data knowledge based on the permitted value and additional content, thereby improving the accuracy of big data knowledge crawling, ensuring the knowledge quality of the crawled big data knowledge, and indirectly improving the accuracy of attribution strategy determination after the big data knowledge is used to construct a big data knowledge graph. In addition, the pre-visit can greatly reduce the unnecessary occupation of the system's access resources and improve the applicability of the system.

[0190] The embodiment of the present invention provides a flange size measurement system with sub-pixel accuracy based on machine vision, such as Figure 8 As shown, including:

[0191] The camera calibration module 1 is used to calibrate the size of the CCD camera;

[0192] An image acquisition module 2 is used to acquire a flange image of the flange to be measured by using a CCD camera after size calibration;

[0193] A preprocessing module 3 is used to preprocess the flange image;

[0194] The edge detection module 4 is used to perform edge detection on the preprocessed flange image to obtain the sub-pixel coordinates of the flange edge to be measured;

[0195] The size determination module 5 is used to determine the size of the flange to be measured based on the sub-pixel level coordinates of the edge of the flange to be measured.

[0196] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. A flange size measurement method with sub-pixel accuracy based on machine vision, characterized in that: include: Calibrate the size of the CCD camera; The flange image of the flange to be measured is collected by a CCD camera after dimension calibration; Preprocessing the flange image; Perform edge detection on the preprocessed flange image to obtain the sub-pixel coordinates of the flange edge to be measured; Determine the size of the flange to be measured based on the sub-pixel coordinates of the edge of the flange to be measured; When the number of abnormal size measurement events occurring within the first time period is greater than or equal to a number threshold, acquiring detection multimodal data; The dimension measurement abnormal event includes: the deviation between the dimension of the flange to be measured and the predetermined dimension is greater than or equal to the deviation threshold; the detection multimodal data includes: dimension detection traceability information of the CCD camera in the first time period, and production traceability information of the flange to be measured in the production dimension measurement abnormal event; Based on the first feature processing template, feature processing is performed on the detection multimodal data to obtain a first feature description vector; Based on the first feature description vector, an attribution strategy is determined; wherein the attribution strategy indicates how to attribute batch anomalies of flange detection according to the detection multimodal data; Based on the attribution strategy, the attribution results are determined according to the multimodal data of the detection; the attribution results include size detection and product production; When the attribution result is size detection, stop continuing to measure the size of the remaining flanges to be detected in the second time period; the second time period is after the first time period and adjacent to the first time period; When the attribution result is product production, stop continuing to measure the dimensions of the remaining flanges to be inspected from the same production batch as the flange to be measured in the production dimension measurement abnormality event within the second time period, and issue a recall warning for the flanges that have been measured for the dimensions of the same production batch as the flange to be measured in the production dimension measurement abnormality event.

2. The flange size measurement method with sub-pixel accuracy based on machine vision according to claim 1, characterized in that: Preprocess the flange image, including: Grayscale processing is performed on the flange image; the grayscale processing method at least includes: weighted average method; Performing filtering and denoising on the flange image after grayscale processing; the filtering and denoising processing methods at least include: Gaussian filtering, median filtering, and mean filtering; The flange image after filtering and denoising is subjected to image segmentation processing; the image segmentation processing method at least includes: segmentation based on threshold value.

3. The flange size measurement method with sub-pixel accuracy based on machine vision as claimed in claim 1, characterized in that: Perform edge detection on the preprocessed flange image to obtain the sub-pixel coordinates of the flange edge to be measured, including: Based on the improved Canny-Zernike moment sub-pixel edge detection algorithm, edge detection is performed on the preprocessed flange image to obtain the sub-pixel coordinates of the flange edge to be measured.

4. The flange size measurement method with sub-pixel accuracy based on machine vision as claimed in claim 1, characterized in that: Based on the sub-pixel coordinates of the edge of the flange to be measured, the size of the flange to be measured is determined, including: The size of the flange to be measured is fitted according to the sub-pixel coordinates of the edge of the flange to be measured based on the least squares method.

5. The flange size measurement method with sub-pixel accuracy based on machine vision according to claim 1, characterized in that: Based on the first feature description vector, an attribution strategy is determined, including: Determining an analysis value corresponding to the first feature description vector from a situation analysis library; When the analysis value is less than or equal to the analysis threshold, determining the attribution strategy corresponding to the first feature description vector from the attribution strategy library; When the analysis value is greater than the analysis threshold, determining the crawling strategy corresponding to the first feature description vector from the crawling strategy library; Crawling big data knowledge based on crawling strategies; Based on big data knowledge, build a big data knowledge graph; Determine attribution strategy based on big data knowledge graph.

6. The flange size measurement method with sub-pixel accuracy based on machine vision as claimed in claim 5, characterized in that: The crawling strategy-based crawling of big data knowledge includes: Analyze the access scenarios, content search rules, and content extraction rules of the crawling strategy; Based on the factor generation template, preliminary factors are generated according to the content search rules and content extraction rules; Pre-access the access scene, and obtain the pre-access content from the access scene based on the pre-access factor; Based on the second feature processing template, feature processing is performed on the prepared content to obtain a second feature description vector; Determining the permitted value and additional content corresponding to the second feature description vector from the permitted analysis library; When the permission value is greater than or equal to the permission threshold, the access scene is accessed, and the first target content is searched from the access scene based on the content search rule during the access; When the additional content is empty, extracting big data knowledge from the first target content based on the content extraction rule; When the additional content includes at least one set of one-to-one corresponding association relationship and content condition, searching for second target content having any association relationship with the first target content from the access scenario, and determining whether the second target content meets the content condition corresponding to any association relationship; When it meets the requirements, big data knowledge is extracted from the first target content based on the content extraction rules.

7. The flange size measurement method with sub-pixel accuracy based on machine vision as claimed in claim 5, characterized in that: Based on big data knowledge, build a big data knowledge graph, including: Analyze the knowledge types of big data knowledge; Determine the mapping node corresponding to the knowledge type from the mapping node table; Set the big data knowledge at the mapping nodes in the initial graph; The initial graph after all mapping nodes that need to set big data knowledge are set with corresponding big data knowledge is used as the big data knowledge graph.

8. The flange size measurement method with sub-pixel accuracy based on machine vision as claimed in claim 7, characterized in that: The attribution strategy is determined based on the big data knowledge graph, including: Determine the support corresponding to multiple hypothetical attribution results from the big data knowledge graph; the support is the ratio of the total number of big data knowledge in the big data knowledge graph that matches the hypothetical attribution results to the total number of big data knowledge in the big data knowledge graph; Based on the hypothesis attribution results corresponding to the maximum support, determine the search rules for strategy formulation; Based on the search rules for the basis for policy formulation, search for the basis for policy formulation in the big data knowledge graph; Develop an attribution strategy based on the strategy formulation basis.

9. The flange size measurement system with sub-pixel accuracy based on machine vision is characterized by: include: Camera calibration module, used to calibrate the size of the CCD camera; An image acquisition module is used to acquire the flange image of the flange to be measured through a size-calibrated CCD camera; A preprocessing module, used for preprocessing the flange image; An edge detection module is used to perform edge detection on the preprocessed flange image to obtain the sub-pixel coordinates of the flange edge to be measured; A size determination module is used to determine the size of the flange to be measured based on the sub-pixel coordinates of the edge of the flange to be measured; Also includes an early warning module for: When the number of abnormal size measurement events occurring within the first time period is greater than or equal to a number threshold, acquiring detection multimodal data; The dimension measurement abnormal event includes: the deviation between the dimension of the flange to be measured and the predetermined dimension is greater than or equal to the deviation threshold; the detection multimodal data includes: dimension detection traceability information of the CCD camera in the first time period, and production traceability information of the flange to be measured in the production dimension measurement abnormal event; Based on the first feature processing template, feature processing is performed on the detection multimodal data to obtain a first feature description vector; Based on the first feature description vector, an attribution strategy is determined; wherein the attribution strategy indicates how to attribute batch anomalies of flange detection according to the detection multimodal data; Based on the attribution strategy, the attribution results are determined according to the multimodal data of the detection; the attribution results include size detection and product production; When the attribution result is size detection, stop continuing to measure the size of the remaining flanges to be detected in the second time period; the second time period is after the first time period and adjacent to the first time period; When the attribution result is product production, stop continuing to measure the dimensions of the remaining flanges to be inspected from the same production batch as the flange to be measured in the production dimension measurement abnormality event within the second time period, and issue a recall warning for the flanges that have been measured for the dimensions of the same production batch as the flange to be measured in the production dimension measurement abnormality event.

Images