Industrial flexible object detection method based on machine vision

Through machine vision technology and related algorithms, accurate measurement of the width of industrial flexible objects is achieved, solving the problems of high detection error rate and low real-time performance in the prior art, and improving the accuracy and efficiency of detection.

CN114494169BActive Publication Date: 2024-11-08NANJING UNIV OF POSTS & TELECOMM
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202210053072.7
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2022-01-18
Publication Date
2024-11-08
Estimated Expiration
2042-01-18

AI Technical Summary

Technical Problem

The prior art is difficult to effectively detect and measure industrial flexible objects with irregular appearance profiles, resulting in high detection error rate and low real-time performance.

Method used

Using machine vision-based detection method, the accuracy of the width of flexible objects is achieved by collecting original color images, data augmentation processing, connection area analysis, centerline extraction and pixel distance mapping, combined with Zhang-Suen refinement algorithm and Hopeline detection. Measurement.

Benefits of technology

It reduces the detection error rate, improves the real-time and accuracy of detection, and is suitable for real-time width measurement of flexible objects on the assembly line.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN114494169B_ABST
    Figure CN114494169B_ABST
Patent Text Reader

Abstract

The invention provides an industrial flexible object detection method based on machine vision, which mainly comprises the following steps: step 1, collecting an original color image of a target as an original data set; step 2, performing data enhancement processing on the initial data set to obtain a black-and-white binary image for distinguishing a target area from a white background; step 3, drawing a connected area of ​​the target, and counting the arc length of each connected area, and retaining a part of the connected area whose area is greater than a fixed value S, or whose arc length is greater than a fixed value L; step 4, drawing an edge contour curve according to the connected area, and extracting a center line of the connected area and parallel straight lines on the target edge contour; step 5, using the center line and the parallel lines to draw a line segment that can represent the width of the target; and step 6, mapping the length of the line segment representing the width of the target from a pixel distance to an actual distance.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The invention relates to an industrial flexible object detection method based on machine vision, belonging to the technical field of industrial detection. Background Art

[0002] The rapid development of computer technology has injected greater vitality into industrial production and life. In particular, the application of computer vision technology has greatly accelerated the production efficiency and capacity of some industrial fields. Computer vision technology has always been a very eye-catching research hotspot. In principle, the machine vision system mainly consists of three parts: image acquisition, image processing and analysis, and output or display. Including digital image processing technology, mechanical engineering technology, control technology, light source lighting technology, optical imaging technology, sensor technology, analog and digital video technology, computer software and hardware technology, human-computer interface technology, etc. Only the coordinated application of these technologies can constitute a complete machine vision application system.

[0003] Based on these fields, we can roughly summarize the five typical applications of the machine vision industry, which can also basically summarize the role that machine vision technology can play in industrial production. They are image recognition application, image detection application, visual positioning application, object measurement application and object sorting application.

[0004] Image recognition applications use machine vision to process, analyze and understand images to identify targets and objects of various different modes. A typical application example is the recognition of QR codes. Image detection applications: Almost all products need to be detected, but manual detection has many disadvantages. Manual detection has low accuracy, and the accuracy cannot be guaranteed if it works for a long time. In addition, the detection speed is slow, which can easily affect the efficiency of the entire production process. Machine vision is also widely used in image detection, such as color positioning and color comparison inspection in the printing process, printing quality inspection of beverage bottle caps in the packaging process, barcode and character recognition on product packaging, etc. Visual positioning applications: Visual positioning requires the machine vision system to quickly and accurately find the measured parts and confirm their positions. Object measurement applications: The biggest feature of machine vision industrial applications is its non-contact measurement technology, which also has high precision and high speed performance, but non-contact and wear-free, eliminating the hidden danger of secondary damage caused by contact measurement. Object sorting applications: Object sorting applications are based on a link after identification and detection. The image is processed by the machine vision system to achieve sorting. In machine vision industrial applications, it is often used for food sorting, automatic sorting of parts surface defects, cotton fiber sorting, etc.

[0005] The more mature cases in the field of object measurement applications are basically the detection of rigid objects. The shape and size of rigid objects are generally fixed, and the degree of deformation under the action of external forces is minimal, which will not have a significant impact on the detection process and detection results. For example, gear and automobile parts detection, etc. However, the width index detection of flexible targets with irregular appearance contours has always been relatively blank, and there are relatively few application invention patents in this area.

[0006] In view of this, it is necessary to propose an industrial flexible object detection method based on machine vision to solve the above problems. Summary of the invention

[0007] The purpose of the present invention is to provide an industrial flexible object detection method based on machine vision to reduce the error rate of detection and improve the real-time performance of detection.

[0008] To achieve the above object, the present invention provides an industrial flexible object detection method based on machine vision, and the industrial flexible object detection method mainly includes:

[0009] Step 1, collecting the target original color image as the original data set;

[0010] Step 2: Perform data enhancement processing on the initial data set to obtain a black-and-white binary image to distinguish the target area from the white background;

[0011] Step 3, draw the connected area of ​​the target, and count the arc length of each connected area, and retain the part with an area greater than a fixed value S or an arc length greater than a fixed value L;

[0012] Step 4, draw the edge contour curve according to the connected area, and extract the parallel straight lines on the center line of the connected area and the target edge contour;

[0013] Step 5, using the center line and the parallel lines to draw a line segment that can represent the target width;

[0014] Step 6: Map the pixel distance to the actual distance for the line segment length representing the target width.

[0015] As a further improvement of the present invention, in step 1, the initial data set is acquired by real-time capture by a digital imaging device.

[0016] As a further improvement of the present invention, in step 2, the data enhancement processing includes median filtering and graying, wherein the median filtering is used to eliminate salt and pepper noise; and the graying uses a weighted average graying method.

[0017] As a further improvement of the present invention, the calculation formula of the weighted average grayscale method is:

[0018] Gray(i,j)=wR*R(i,j)+wG*G(i,j)+wB*B(i,j), where wR, wG, and wB represent the weighted averages of green, red, and blue, respectively.

[0019] As a further improvement of the present invention, wR=0.229, wG=0.578, and wB=0.114.

[0020] As a further improvement of the present invention, in step 3, the connected region of the image is a region consisting of pixels having the same pixel value and being adjacent to each other.

[0021] As a further improvement of the present invention, step 4 comprises:

[0022] Step 4.1, the algorithm for extracting the center line of the connected region is the Zhang-Suen thinning algorithm; from the periphery of the target to the center of the target, the target is continuously eroded and thinned using the features of a 3*3 pixel window centered on the pixel to be detected until it is eroded to a single layer of pixel width to obtain the center line of the connected region;

[0023] Step 4.2, applying Hopse line detection, maps the curves or lines with the same shape in the image space to a point in the Hopse space, so as to transform the problem of detecting arbitrary shapes into the problem of counting the coincidence of points in the Hopse space.

[0024] As a further improvement of the present invention, step 5 comprises:

[0025] Step 5.1, traverse the edge points of the target contour in turn, find the connecting line between the current position and all the edge contour pixel points that are not at this position, and record the length;

[0026] Step 5.2, determine whether it passes through the center line of the connected area, if it passes, it will be retained, if it does not pass, it will be discarded; find the minimum length of all retained ones as the target width of the current pixel position.

[0027] As a further improvement of the present invention, in step 6, the pixel distance to actual distance mapping method adopts Zhang Zhengyou calibration method; a monocular camera is used for shooting, and with the help of a pinhole imaging model, the positional relationship between a certain point in space and the imaging point in the camera is constructed. The unit distance in the actual distance is millimeters, and the unit distance in the pixel coordinate system is pixels.

[0028] The beneficial effects of the present invention are as follows: the industrial flexible object detection method based on machine vision of the present invention utilizes mature computer vision technology and combines relevant improved algorithms to realize width detection of industrial flexible objects. The method has the characteristics of low error rate and high real-time performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1It is a flow chart of the industrial flexible object detection method based on machine vision of the present invention. DETAILED DESCRIPTION

[0030] In order to make the purpose, technical solutions and advantages of the present invention more clear, the present invention is described in detail below with reference to the accompanying drawings and specific embodiments.

[0031] The existing method for detecting industrial flexible targets is usually the projection method, which usually selects product patterns with relatively regular appearance contours produced by cutting on the assembly line and pastes them on the sample holder, and uses a magnifying glass scale method / digital projector to measure the target. For each target, the width measurement values ​​of multiple positions are selected, and the average value of the width measurement points of all pattern products is used as the final width measurement value of the target. This method is time-consuming and has low real-time performance. In addition, during the selection and clamping of the target product, the external force will have a secondary impact on the width index of the target, causing a certain error in the final result.

[0032] The present invention adopts a computer vision detection method with high real-time performance. It can take pictures of products on the assembly line in real time and perform data enhancement and black-and-white binary processing, etc. It uses the centerline method and Hope parallel line method to extract feature representation, and combines the distance mapping between the world coordinate system and the camera coordinate system to calculate the width value. After many experimental results, it is found that the proposed innovative method has accurate measurement results and timely feedback, which greatly increases the detection accuracy and efficiency.

[0033] like Figure 1 As shown, the present invention discloses an industrial flexible object detection method based on machine vision, which mainly includes:

[0034] Step 1, collecting the target original color image as the original data set;

[0035] Step 2: Perform data enhancement processing on the initial data set to obtain a black-and-white binary image to distinguish the target area from the white background;

[0036] Step 3, draw the connected area of ​​the target, and count the arc length of each connected area, and retain the part with an area greater than a fixed value S or an arc length greater than a fixed value L;

[0037] Step 4, draw the edge contour curve according to the connected area, and extract the parallel straight lines on the center line of the connected area and the target edge contour;

[0038] Step 5, using the center line and the parallel lines to draw a line segment representing the target width;

[0039] Step 6: Map the pixel distance to the actual distance for the line segment length representing the target width.

[0040] Furthermore, the initial data set in step 1 is obtained by real-time capture of the data set by a digital imaging device.

[0041] Furthermore, the data enhancement processing in step 2 includes median filtering and graying, and median filtering is used to eliminate salt and pepper noise. Graying uses a weighted average graying method, that is, Gray(i,j)=wR*R(i,j)+wG*G(i,j)+wB*B(i,j), where wR, wG, and wB represent the weighted averages of green, red, and blue, respectively. In the present invention, wR=0.229, wG=0.578, and wB=0.114 are taken.

[0042] Furthermore, the connected region of the image in step 3 refers to a region composed of pixels having the same pixel value and adjacent positions, and the present invention uses an 8-neighborhood method.

[0043] Furthermore, the algorithm for extracting the contour centerline in step 4 is the Zhang-Suen thinning algorithm. This method uses the characteristics of a 3*3 pixel window centered on the pixel to be detected to continuously erode and thin the target from the periphery to the center of the target until it is eroded to a single layer of pixel width, and the centerline of the connected area can be obtained.

[0044] For the image in step 4, Hop line detection is applied. The basic idea of ​​Hop line detection is to use the transformation between image space and Hop space to map the curve or straight line with the same shape in the image space to a point in the Hop space, thereby converting the problem of detecting arbitrary shapes into the problem of statistically counting the coincidence of points in the Hop space. In the present invention, when applying the Hop transform, the distance step in pixels is selected as 1, the angle step in radians is selected as Π / 180, the threshold parameter of the cumulative value is set to 5, the minimum line segment length threshold is 3, and if the length of the break interval pixels on the same straight line is 2 or more, it is considered to be a different line segment.

[0045] Furthermore, in step 5, the slope k of each line segment is calculated i , and the midpoint of each line segment (x cen ,y cen ). Traverse all parallel line segments, connect the midpoint of the current line segment to the midpoints of the remaining line segments, and determine whether the connecting line is a perpendicular line between the two straight lines. If so, keep it, otherwise discard it, and iterate to select the perpendicular line with the shortest distance.

[0046] Furthermore, in step 5, the target contour edge points are traversed in turn to find the current position (x, y) and all the edge contour pixel points (x i ,y j ) i , record l i Length. Determine l iWhether it passes through the center line of the connected area, if it passes, it will be retained, if it does not pass, it will be discarded. i Find the minimum length as the target width at the current pixel position.

[0047] Furthermore, in step 6, the pixel distance to actual distance mapping method adopts Zhang Zhengyou calibration method. In the present invention, a monocular camera is used for shooting, and with the help of a pinhole imaging model, the position relationship between a certain point in space and the imaging point in the camera is constructed. In the actual distance, the unit distance is millimeter; the unit distance in the pixel coordinate system is pixel. Finally, the basic formula for coordinate system conversion is:

[0048] sm=A[R t]M

[0049] Among them, s is an arbitrary scale factor; m is the pixel coordinates of the corner points of the calibration chessboard; M is the world coordinates of the corner points of the chessboard. A is the camera intrinsic parameter matrix; [R t] is the extrinsic parameter matrix, R is the rotation matrix, and t is the translation matrix, which correspond to the rotation and translation of the calibration plate respectively.

[0050] The camera intrinsic parameter matrix is:

[0051]

[0052] Among them, u0 is the abscissa of the principal point; v0 is the ordinate of the principal point; γ is the skew error parameter of the two coordinate axes u and v caused by industrial manufacturing; it can be approximately regarded as 0; f x is the horizontal effective focal length; f y is the vertical effective focal length; the above parameters are all in pixels.

[0053] In the present invention, the reprojection error is used as the evaluation index of calibration. The reprojection error refers to the error between the two-dimensional point obtained by reprojecting the three-dimensional point of the world coordinate system to the image coordinate system based on the camera intrinsic parameter matrix and the original two-dimensional point. The calculation formula is as follows:

[0054]

[0055] Among them, error is the reprojection error; for the corner points of the chessboard, x′ is the two-dimensional point coordinate after reprojection, x is the original two-dimensional point coordinate; total_points is the number of all corner points on the chessboard.

[0056] In summary, the industrial flexible object detection method based on machine vision of the present invention utilizes mature computer vision technology and combines relevant improved algorithms to realize width detection of industrial flexible objects. The method has the characteristics of low error rate and high real-time performance.

[0057] The above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solution of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical solution of the present invention.

Claims

1. A method for detecting industrial flexible objects based on machine vision, characterized in that: The industrial flexible object detection method mainly includes: Step 1, collecting the target original color image as the original data set; Step 2: Perform data enhancement processing on the initial data set to obtain a black-and-white binary image to distinguish the target area from the white background; Step 3, draw the connected area of ​​the target, and count the arc length of each connected area, and retain the part with an area greater than a fixed value S or an arc length greater than a fixed value L; Step 4, draw the edge contour curve according to the connected area, and extract the parallel straight lines on the center line of the connected area and the target edge contour; Step 5, using the center line and the parallel lines to draw a line segment that can represent the target width; Step 6, mapping the pixel distance to the actual distance of the line segment length representing the target width; Step 4 includes: Step 4.1, the algorithm for extracting the center line of the connected region is the Zhang-Suen thinning algorithm; from the periphery of the target to the center of the target, the target is continuously eroded and thinned using the features of a 3*3 pixel window centered on the pixel to be detected until it is eroded to a single layer of pixel width to obtain the center line of the connected region; Step 4.2, applying Hopse line detection, mapping the curves or lines with the same shape in the image space to a point in the Hopse space, so as to transform the problem of detecting arbitrary shapes into the problem of counting the coincidence of points in the Hopse space; Step 5 includes: Step 5.1, traverse the edge points of the target contour in turn, find the connecting line between the current position and all the edge contour pixel points that are not at this position, and record the length; Step 5.2, determine whether it passes through the center line of the connected area, if it passes, it will be retained, if it does not pass, it will be discarded; find the minimum length of all retained ones as the target width of the current pixel position.

2. The method for detecting industrial flexible objects based on machine vision according to claim 1, characterized in that: In step 1, the initial data set is captured in real time by a digital imaging device.

3. The method for detecting industrial flexible objects based on machine vision according to claim 1, characterized in that: In step 2, the data enhancement processing includes median filtering and graying. The median filtering is used to eliminate salt and pepper noise; the graying uses a weighted average graying method.

4. The method for detecting industrial flexible objects based on machine vision according to claim 3, characterized in that: The calculation formula of the weighted average grayscale method is: Gray(i,j)=wR*R(i,j)+wG*G(i,j)+wB*B(i,j), where wR, wG, and wB represent the weighted averages of green, red, and blue, respectively.

5. The method for detecting industrial flexible objects based on machine vision according to claim 4, characterized in that: The wR=0.229, wG=0.578, and wB=0.

114.

6. The method for detecting industrial flexible objects based on machine vision according to claim 1, characterized in that: In step 3, the connected region of the image is a region consisting of pixels having the same pixel value and being adjacent to each other.

7. The method for detecting industrial flexible objects based on machine vision according to claim 1, characterized in that: In step 6, the pixel distance to actual distance mapping method adopts Zhang Zhengyou calibration method; a monocular camera is used for shooting, and with the help of a pinhole imaging model, the position relationship between a certain point in space and the imaging point in the camera is constructed. The unit distance in the actual distance is millimeter, and the unit distance in the pixel coordinate system is pixel.