Rod and wire head alignment detection method based on image recognition

Abstract

The invention provides a rod and wire rod head alignment detection method based on image recognition, and relates to the technical field of image processing. In order to solve the problem that the light source angle of a rod and wire cutting site is fixed, so that illumination of a collected rod and wire image is not uniform, the influence of the light source angle on rod and wire recognition is considered, and a shadow removal method for a full-shadow area and a penumbra area is provided; according to the efficient unmanned cutting requirement, after shadow is eliminated, whether the head of a rod wire reaches a baffle and is aligned with the baffle or not is detected, an alignment line of the head and the baffle is fitted, and the alignment line is moved downwards to the middle and the tail of the rod wire at a fixed step length; whether the numbers of the rod wires linearly penetrating through the head part, the middle part and the tail part of the rod wires are equal or not is identified, so that whether all the rod wires are aligned with the baffle is judged; the detection task of full-automatic cutting of steel rods and wires is achieved, and compared with an existing automatic cutting technology, the use scene is wider.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to a method for detecting the alignment of the heads of bars and wires based on image recognition. Background Art

[0002] In the production process of bars and wires, in the bar and wire cutting process, if the heads of the bars and wires are all aligned at the baffle, the guillotine will cut at the back end. The way of manually supervising the cutting of steel bars and wires is to manually observe whether the bars and wires are all aligned with the baffle. If they are aligned, the operator clicks the cutting button to cut. Otherwise, the operator needs to start the conveyor belt to make the bars and wires hit the baffle to ensure that the heads are all aligned with the baffle.

[0003] Due to the problem of the light source angle at the bar and wire cutting site, the illumination at the bar and wire cutting site is uneven, and some bars and wires have insufficient illumination and shadow occlusion. This phenomenon greatly affects the recognition of the alignment of the heads of bars and wires. Therefore, the shadow problem needs to be considered in the technology of detecting the alignment of the heads of bars and wires. At the same time, in the enterprise scenarios pursuing production quality and efficiency, the recognition accuracy and efficiency need to be considered.

[0004] The way of manually monitoring the cutting of bars and wires, due to the high labor cost and low efficiency, not only reduces the production efficiency of the entire production line, but also has various human uncertainty factors, leading to production hazards such as safety and quality. Li Fan et al. proposed a method for detecting the alignment of the heads of bars with a fixed-length baffle in the "Research on the Method and Device for Detecting the Alignment of the Heads of Bars with a Fixed-length Baffle". A fixed-length baffle alignment detection system based on deep learning is used, an industrial camera is used to collect images of the heads of bars, and image processing and analysis are carried out through deep learning algorithms to automatically detect the positions of the heads of bars, so as to achieve automatic alignment and control. However, this method is not applicable to the recognition of bar and wire pictures with shadows. This method requires specific camera installation and light source design, and cannot recognize the pictures of bar and wire with shadows that may appear in the actual recognition process. Moreover, the specific camera installation and light source design undoubtedly increase the labor cost. Summary of the Invention

[0005] The technical problem to be solved by the present invention is to aim at the above-mentioned deficiencies of the prior art, and provide a method for detecting the alignment of the heads of bars and wires based on image recognition. With the goal of high-efficiency, high-precision and unmanned cutting of bars and wires, considering the influence of the light source angle on the recognition of bars and wires, shadow removal methods for the full-shadow area and the penumbra area are proposed respectively. After shadow removal, image recognition technology is used to detect whether the bars and wires are all aligned and have reached the baffle, and then whether to cut the bars and wires is determined according to the detection result. The present invention avoids the influence of uneven illumination on the head alignment detection effect, and at the same time applies the fitting detection of the head alignment dividing line to the actual production of bars and wires, which can effectively reduce the defective rate of bars and wires in enterprises.

[0006] To solve the above technical problems, the technical solution adopted by the present invention is as follows:

[0007] The present invention provides a method for detecting the alignment of the head of bar and wire rod based on image recognition, including the following steps:

[0008] Step 1: Obtain the RGB image of the bar and wire rod to be detected, identify and eliminate the shadows in the shadow area of the RGB image of the bar and wire rod to be detected, and obtain the RGB image of the bar and wire rod to be detected with the shadows removed;

[0009] Step 1.1: Obtain the RGB image of the bar and wire rod to be detected, identify the shadow area according to the difference between the blue component and the red component and the green component of each pixel in the RGB image of the bar and wire rod to be detected, and perform dilation processing on the identified shadow area to obtain the expanded shadow area;

[0010] Identify the shadow area according to the difference between the blue component and the red component and the green component of each pixel in the RGB image of the bar and wire rod to be detected. To avoid the blue part in the image being identified as the shadow area, the average image brightness is introduced as a collaborative evaluation criterion, as shown in the following formula:

[0011]

[0012] Among them, x is the abscissa of the pixel (x, y), y is the ordinate of the pixel (x, y), fgray(x, y) is the gray value of the pixel (x, y), ave(fgray) is the average brightness value of each pixel after converting the RGB image of the bar and wire rod to be detected into the gray image fgray, f(x, y, 1) is the red component pixel value of the pixel (x, y) in the RGB image of the bar and wire rod to be detected, f(x, y, 2) is the green component pixel value of the pixel (x, y) in the RGB image of the bar and wire rod to be detected, f(x, y, 3) is the blue component pixel value of the pixel (x, y) in the RGB image of the bar and wire rod to be detected, & is the AND operation, fo(x, y) is the recognition result of the pixel (x, y). If fo(x, y) = 0, the pixel (x, y) belongs to the shadow area, otherwise it does not belong to the shadow area;

[0013] Perform dilation processing on the identified shadow area to avoid the appearance of black edges after shadow removal, and obtain the expanded shadow area;

[0014] Step 1.2: Perform edge detection on the expanded shadow area to obtain the shadow edge, and divide the expanded shadow area into a full-shadow area and a penumbra area. The penumbra area is the transition area from the full-shadow area to the non-shadow area. Eliminate the shadows in the full-shadow area and the penumbra area respectively to obtain the RGB image of the bar and wire rod to be detected with the shadows removed;

[0015] Step 1.2.1: Perform edge detection on the expanded shadow area to obtain the shadow edge, and divide the expanded shadow area into a full-shadow area and a penumbra area. The penumbra area is the transition area from the full-shadow area to the non-shadow area. Use an S-shaped function to model the penumbra area and use an inverse S-shaped function model to eliminate the shadow in the penumbra area image;

[0016] Calculate the normal vector n of the pixel (a, b) on the shadow edge BW ab =(n a , n b ), where n a is the magnitude of the normal vector in the horizontal direction, and n b is the magnitude of the normal vector in the vertical direction. Taking the pixel point with the pixel value BW(a, b)=1 on the shadow edge as the center and the directly above the pixel point as the positive direction, determine the positive direction angle θ of the normal vector n ab of this pixel point, and determine the transition direction d from the penumbra area to the non-shadow area according to the positive direction angle θ, as shown in the following formula:

[0017]

[0018] where the X direction is the directly right direction of the pixel point, the Mid1 direction is the direction of rotating 45° counterclockwise from the directly right of the pixel point, the Y direction is the directly above direction of the pixel point, and the Mid2 direction is the direction of rotating 45° counterclockwise from the directly above of the pixel point;

[0019] Model the light intensity change in the penumbra area. Assume that the penumbra area contains ω pixel points in total. Take (ω - 1) / 2 pixel points along the positive and negative directions of the transition direction d from the penumbra area to the non-shadow area, and determine the maximum value f max and the minimum value f min of the pixel values within the transition range. Establish an S-shaped function model for the light intensity change in the penumbra area to obtain the pixel value Fmod (g,h) (i) of the pixel point (g, h) in the penumbra area, as shown in the following formula:

[0020]

[0021] where g is the abscissa of the pixel point in the penumbra area, h is the ordinate of the pixel point in the penumbra area, i is the pixel point label, is the translation term of the S-shaped function, is the scaling term of the S-shaped function;

[0022] Use the inverse S-shaped function model to eliminate the shadow in the penumbra area image, eliminate the influence of the shadow on the recognition of the head alignment of the bar and wire, and obtain the pixel values of each pixel in the penumbra area after shadow elimination, as shown in the following formula:

[0023]

[0024] Among them, Fres (g,h) (i) is the pixel value after removing the shadow of the pixel (g, h) in the penumbra region, and f (g,h) (i) is the pixel value of the pixel (g, h) in the penumbra region before removing the shadow;

[0025] Step 1.2.2: Use the multi-threshold segmentation algorithm to divide the umbra region into several umbra sub-regions, and perform shadow removal on each umbra sub-region;

[0026] Obtain the maximum brightness value I of the grayscale image of the umbra region max and the minimum brightness value I min . For the umbra region with a step size of L, divide the umbra region into (I max -I min ) / L segmentation sub-regions, obtain the brightness mean value of each segmentation sub-region, and obtain the brightness mean value IN of the non-shadow region adjacent to it ave . Calculate the brightness mean ratio Br between the non-shadow region and the segmentation sub-region, perform shadow removal on each segmentation sub-region image, and the pixel value F j (e, f) of the j-th segmentation sub-region after removing the shadow is shown in the following formula:

[0027] F j (e, f) = f j (e, f) * Br + IN ave (5)

[0028] Among them, f j (e, f) is the pixel value of the j-th segmentation sub-region before removing the shadow;

[0029] Step 2: Convert the RGB image of the bar and wire to be detected after removing the shadow into a grayscale image, perform texture segmentation and binarization processing on the grayscale image of the bar and wire to be detected after removing the shadow, use the dilation algorithm to connect the entire bar and wire region, and extract the edge of the bar and wire to obtain the edge image of the bar and wire to be detected;

[0030] Step 2.1: Convert the RGB image of the bar and wire to be detected after removing the shadow into a grayscale image, and perform two texture segmentations on the grayscale image of the bar and wire to be detected after removing the shadow based on the grayscale value to obtain the image of the bar and wire to be detected after texture segmentation;

[0031] For the RGB image of the bar and wire to be detected after removing the shadow, establish a three-dimensional matrix G, as shown in the following formula:

[0032]

[0033] Among them, m and n are the pixel points in the RGB image ([[]] m,n) The abscissa and ordinate, where M and N are the total number of rows and columns of pixel points included in the RGB image respectively, t = {1, 2, 3} is the dimension of the pixel points, t = 1 is the red component of the RGB image, t = 2 is the green component of the RGB image, t = 3 is the blue component of the RGB image, and the value range of each element in the three-dimensional matrix G is [0, 255];

[0034] Convert the shadow-removed RGB image of the bar and wire to be detected into a shadow-removed grayscale image of the bar and wire to be detected. The effective grayscale value f of the pixel (m, n) after conversion 1 (m, n) is as shown in the following formula:

[0035] f 1 (m, n) = 0.3 * G(m, n, 1) + 0.59 * G(m, n, 2) + 0.11 * G(m, n, 3) (7)

[0036] Where,

[0037] Perform texture segmentation on the shadow-removed grayscale image of the bar and wire to be detected twice based on the grayscale value to identify the bar and wire features, and obtain the image of the bar and wire to be detected after two texture segmentations;

[0038] The specific method for performing texture segmentation on the shadow-removed grayscale image of the bar and wire to be detected based on the grayscale value is:

[0039] Taking the pixel ( m,n ) as the center, define a sliding window with a size of k×k, and calculate the local standard deviation f of the pixel and the pixels within the k×k neighborhood 2 (m, n), as shown in the following formula:

[0040]

[0041] Where, U is the average value of the effective grayscale values of k 2 pixels within the k×k neighborhood, and P r is the effective grayscale value of the r-th pixel point of the pixel (m, n) within the k×k neighborhood;

[0042] Scale the local standard deviation f of the pixel point (m, n) within the k×k neighborhood 2 (m, n) to the interval [0, 1] to obtain the normalized local standard deviation f of the pixel point (m, n) within the k×k neighborhood 3 (m, n), which is used as the pixel value of the pixel point (m, n) after texture segmentation, as shown in the following formula:

[0043] f 3 (m, n) = f 2 (m, n) / 255 (9)

[0044] Step 2.2: Perform binarization on the wire rod image to be detected after two texture segmentations to obtain a binarized wire rod image to be detected;

[0045] The specific method for performing binarization on the wire rod image to be detected after two texture segmentations is as follows:

[0046] For the wire rod image f to be detected after two texture segmentations 4 , where the pixel value of the pixel point ( m,n ) is f4 ( m,n ), set a threshold γ, screen the pixel points in the wire rod image to be detected after the second texture segmentation whose pixel values are greater than the threshold γ, set their pixel values to 1, and set the pixel values of the remaining pixel points to 0 to obtain a binarized wire rod image f 5 , as shown in the following formula:

[0047]

[0048] where f 5 (m, n) is the binarized pixel value of the pixel point (m, n);

[0049] Step 2.3: For the binarized wire rod image to be detected, use the dilation algorithm to connect the entire wire rod area, use the sobel operator to perform edge detection on the wire rod area, and extract the edge of the wire rod to obtain an edge image of the wire rod to be detected;

[0050] Step 2.3.1: Use the dilation algorithm to connect the entire wire rod area, ensuring that the pixel values of the pixel points in the area where the wire rod is located are 1 and the pixel values of the pixel points in the area without the wire rod are 0, including the following steps:

[0051] (1) Perform dilation processing on the binarized wire rod image to be detected, connect the connected areas in the binarized wire rod image to be detected whose distances are less than the set threshold, eliminate fragmented connected areas, and the pixel value of the pixel (m, n) after dilation processing is f 6 (m, n);

[0052] (2) Delete the connected areas in the image whose areas are less than the set threshold, and the pixel value of the pixel ( m,n ) after deleting the connected areas is f 7 (m, n);

[0053] (3) Perform a closing operation on the binary wire rod image with connected regions removed. Use the dilation and erosion convolution operation template nhood to scan each pixel point in the binary wire rod image with connected regions removed. The dilation and erosion convolution operation template nhood performs a dilation operation with the binary wire rod image with connected regions removed to obtain the minimum pixel value in the area covered by the dilation and erosion convolution operation template nhood. Use this minimum value to replace the pixel value of the pixel point currently scanned by the dilation and erosion convolution operation template nhood, and perform an erosion operation to connect the wire rod regions. In the binary wire rod image after the closing operation, the pixel value of pixel (m, n) is f 8 (m, n), as shown in the following formula:

[0054]

[0055] where nhood is a 255×255 matrix with all element values being 1, is the dilation operation, is the erosion operation;

[0056] (4) Use a linear structuring element to dilate and enlarge the still unconnected wire rod regions in the binary wire rod image after the closing operation, and delete the connected objects with an area smaller than the set threshold to obtain a binary wire rod image of the connected wire rod regions, where the pixel value of pixel (m, n) is f 9 (m, n);

[0057] Step 2.3.2: Use the sobel operator to detect the edges of the wire rod in the connected wire rod region to obtain the wire rod edge image to be detected;

[0058] Perform planar convolutions on the binary wire rod image to be detected in the connected wire rod region using the sobel convolution factor Gx in the horizontal direction and the sobel convolution factor Gy in the vertical direction respectively, as shown in the following formula:

[0059] Hx = f 9 (m, n) * Gx, Hy = f 9 (m, n) * Gy (13)

[0060]

[0061] where Hx is the convolution result using the sobel convolution factor in the horizontal direction, and Hy is the convolution result using the sobel convolution factor in the vertical direction, is the binary wire rod image to be detected in the connected wire rod region f 9 's first derivative;

[0062] According to the wire rod image to be detected in the connected wire rod region f9 The first derivative Determine the position where the edge of the bar and wire exists, and obtain the edge image of the bar and wire to be detected f10 ( m,n ), as shown in the following formula:

[0063]

[0064] Among them, When it is 0, it means that this area does not belong to the edge area of the bar and wire, and the pixel value of the pixel points in this area is set to 0, When it is not 0, it means that this area belongs to the edge area of the bar and wire, and the pixel value of the pixel points in this area is set to 1;

[0065] Step 3: Obtain the edge pixel points of the head of the bar and wire, establish a solution process model for the alignment line of the head of the bar and wire, and use the genetic algorithm to solve the slope and intercept of the alignment line of the head of the bar and wire to obtain the fitted alignment line of the head of the bar and wire;

[0066] Step 3.1: Obtain the edge pixel points (p, q) of the head of the bar and wire, and establish a solution process model for the alignment line of the head of the bar and wire, as shown in the following formula:

[0067]

[0068] Among them, a is the slope of the alignment line of the head of the bar and wire, b is the intercept of the alignment line of the head of the bar and wire, ε is the number of edge pixel points of the head of the bar and wire, p is the abscissa of the edge pixel point (p, q) of the head of the bar and wire, and q is the ordinate of the edge pixel point (p, q) of the head of the bar and wire;

[0069] Step 3.2: Use the genetic algorithm to solve the slope and intercept of the alignment line of the head of the bar and wire to obtain the fitted alignment line of the head of the bar and wire;

[0070] Step 3.2.1: Initialize the population size, set the crossover probability and mutation probability, and initialize a group of random sequences, including the number of individuals equal to the population size. Each individual is a random sequence, and each random sequence is a binary encoding of z bits. The first z' bits of the binary encoding are used to represent the slope a of the alignment line of the head of the bar and wire, and the last z - z' bits of the binary encoding are used to represent the intercept b of the alignment line of the head of the bar and wire. Take this group of random sequences as the input of the genetic algorithm to determine the initial optimal solution;

[0071] Step 3.2.2: Solve the population fitness fit, as shown in the following formula:

[0072]

[0073] Among them, τ is the number of iterations;

[0074] Step 3.2.3: Use the roulette wheel algorithm to select the individual with the maximum fitness fit in the population, and perform crossover and mutation on the individual with the maximum fitness to generate the optimal solution a of the population in the τ-th iteration τ , b τ ;

[0075] Step 3.2.4: Update the optimal solution of the population, and determine whether the number of iterations is reached. If not, go to Step 3.2.2; otherwise, output the optimal solution, including the slope a′ of the fitted head alignment line of the bar and wire materials and the intercept b′ of the fitted head alignment line of the bar and wire materials

[0076] Step 3.2.5: Based on the slope a′ of the fitted head alignment line of the bar and wire materials and the intercept b′ of the fitted head alignment line of the bar and wire materials, obtain the linear expression of the fitted head alignment line q′ of the bar and wire materials, as shown in the following formula

[0077] q′ = a′p′ + b′ (18)

[0078] where p′ is the abscissa of the pixel point (p′, q′) on the head alignment line of the bar and wire materials, and q′ is the abscissa of the pixel point (p′, q′) on the head alignment line of the bar and wire materials

[0079] Step 4: Move down the fitted head alignment line of the bar and wire materials to obtain several bar and wire alignment lines at different positions, and compare the number of bar and wire materials passed by the bar and wire alignment lines at different positions to detect whether all the bar and wire materials in this batch have reached the fixed-length baffle and are aligned

[0080] Step 4.1: Perform dilation and erosion on the binary image f 5 (m, n) of the bar and wire materials to be detected, and perform noise deletion operation. After dilation, erosion, and noise deletion, in the binary image f 11 of the bar and wire materials to be detected, the pixel value of the pixel point (m, n) is f 11 (m, n);

[0081] Step 4.2: Based on the fitted head alignment line of the bar and wire materials, change the value of the intercept b′ of the head alignment line of the bar and wire materials to move down the fitted head alignment line of the bar and wire materials, move the fitted head alignment line of the bar and wire materials to the tail position of the bar and wire materials to obtain the tail alignment line of the bar and wire materials, and count the number of bar and wire materials on the tail alignment line of the bar and wire materials

[0082] It is assumed that in the binary image f 11 of the bar and wire materials to be detected after dilation, erosion, and noise deletion processing, the pixel value of each pixel in the area where there are bar and wire materials is 1, the pixel value of each pixel in the area where there are no bar and wire materials is 0, and the pixel value of each pixel in the bar and wire gap area is 0, as shown in the following formula

[0083]

[0084] where c is the number of the pixel point in the image;

[0085] Judge the relationship between the pixel values of two adjacent pixel points on the tail alignment line of the bar and wire in sequence and count the number of bars and wires. If the pixel value of the previous pixel point is 1 and the pixel value of the next pixel point is 0, it is determined that there is one bar and wire in this area, and the number of bars and wires s on the tail alignment line of the bar and wire is obtained, as shown in the following formula:

[0086]

[0087] Step 4.3: Change the intercept b′ of the fitted head alignment line of the bar and wire at a specified step length, move down the fitted head alignment line of the bar and wire to obtain multiple middle alignment lines of the bar and wire at different positions, use the method in Step 4.2 to count the number of bars and wires on each middle alignment line of the bar and wire at different positions, and compare it with the number of bars and wires on the tail alignment line of the bar and wire to judge whether all the bars and wires have reached the fixed-length baffle and are aligned;

[0088] The rule for judging whether the bars and wires are aligned is:

[0089] Record the number of bars and wires mycount on the middle alignment line of the bar and wire at different positions w , w is the serial number of the middle alignment line of the bar and wire. After the distance between the fitted head alignment line of the bar and wire and the fixed-length baffle exceeds the specified alignment accuracy, if the number of bars and wires mycount w on the middle alignment line of the bar and wire and the number of bars and wires s on the tail alignment line of the bar and wire are still not equal, stop moving down the head alignment line of the bar and wire, and it is determined that not all the bars and wires have reached the baffle and are aligned. The number of bars and wires that have not reached the baffle is s - mycount w pieces; if after the head alignment line of the bar and wire is moved down to coincide with the tail alignment line of the bar and wire, there is no situation where the number of bars and wires mycount w on the middle alignment line of the bar and wire is not equal to the number of bars and wires s on the tail alignment line of the bar and wire, it is determined that all the bars and wires have reached the fixed-length baffle and are aligned, meeting the cutting requirements.

[0090] The beneficial effects of adopting the above technical solutions are as follows: A method for detecting the alignment of the heads of steel bars and wires based on image recognition provided by the present invention aims at the problem that the light source angle at the cutting site of steel bars and wires is fixed, resulting in uneven illumination in the collected images of steel bars and wires. First, shadow elimination is performed on the recognized images of steel bars and wires. The S-shaped function model is used to model the penumbra region of the image, and the inverse S-shaped function model is used to eliminate the shadow of the penumbra region image. For the umbra region, the multi-threshold segmentation algorithm is used to divide the umbra region into several sub-umbra regions, and shadow elimination is performed on each sub-umbra region. For the high-efficiency unmanned cutting requirement, after shadow elimination, the detection is carried out with the goal of whether the head of the steel bar and wire reaches the baffle and is aligned with the baffle. The alignment line of the head and the baffle is fitted, and the alignment line is moved down to the middle and tail of the steel bar and wire at a fixed step length, and whether the number of steel bars and wires passing through the head, middle, and tail of the steel bar and wire is equal is respectively identified to determine whether all the steel bars and wires are aligned with the baffle. In the whole process, shadow removal and the fitting detection of the head alignment dividing line are particularly important, which is the basis for judging whether the steel bar and wire can be cut. The present invention realizes the detection task of the full-automatic cutting of steel bars and wires. Compared with manual observation of cutting, it is more intelligent, efficient, and has high precision; compared with the existing automatic cutting technology, the application scenario is wider, and it can correctly identify steel bars and wires with uneven illumination, which can reduce the defective rate of steel bars and wires in enterprises, reduce the losses of steel enterprises, and meet the needs of steel enterprises. BRIEF DESCRIPTION OF THE DRAWINGS

[0091] Figure 1 It is a flowchart of a method for detecting the alignment of the heads of steel bars and wires based on image recognition provided by an embodiment of the present invention;

[0092] Figure 2 It is a schematic diagram of the transition direction from the penumbra region to the non-shadow region provided by an embodiment of the present invention;

[0093] Figure 3 It is an image of a steel bar and wire to be detected with shadow eliminated provided by an embodiment of the present invention. Among them, (a) is the image of the steel bar and wire to be detected before shadow elimination, and (b) is the image of the steel bar and wire to be detected after shadow elimination;

[0094] Figure 4 It is an edge effect diagram of extracting a steel bar and wire to be detected provided by an embodiment of the present invention. Among them, (a) is the gray-scale image of the steel bar and wire to be detected with shadow removed, (b) is the image of the steel bar and wire to be detected after two texture segmentations, (c) is the binary image of the steel bar and wire to be detected, (d) is the binary image of the steel bar and wire to be detected after dilation processing, (e) is the binary image of the steel bar and wire to be detected with the connected steel bar region, and (f) is the edge image of the steel bar and wire to be detected;

[0095] Figure 5 It is a flowchart of solving by genetic algorithm provided by an embodiment of the present invention;

[0096] Figure 6 The effect diagram of the head alignment line of the bar and wire provided by the embodiment of the present invention;

[0097] Figure 7 The effect diagram of the judgment of the number of aligned heads of the bar and wire provided by the embodiment of the present invention. Specific embodiments

[0098] The following combines the accompanying drawings and embodiments to further describe in detail the specific embodiments of the present invention. The following embodiments are used to illustrate the present invention, but are not used to limit the scope of the present invention.

[0099] A method for detecting the head alignment of bar and wire based on image recognition in this embodiment, as Figure 1 shown, includes the following steps:

[0100] Step 1: Obtain the RGB image of the bar and wire to be detected, identify and eliminate the shadows in the shadow area of the RGB image of the bar and wire to be detected, and obtain the RGB image of the bar and wire to be detected with shadows removed;

[0101] Step 1.1: Obtain the RGB image of the bar and wire to be detected, identify the shadow area according to the difference between the blue component and the red component and the green component of each pixel in the RGB image of the bar and wire to be detected, and perform dilation processing on the identified shadow area to obtain the expanded shadow area;

[0102] There are both direct light and scattered light in the non-shadow area of the bar and wire image, while only the scattered light of the light source exists in the shadow area. This feature is widely used in image shadow detection. In the RGB color system, an image is composed of three independent monochromatic component images, namely the red component (Red), the green component (Green), and the blue component (Blue). According to Rayleigh scattering law, the scattered light mainly has more short-wave components and more blue components, and the direct light has more long-wave components, and the red component R and the green component G are more;

[0103] Identify the shadow area according to the difference between the blue component and the red component and the green component of each pixel in the RGB image of the bar and wire to be detected. In order to avoid the blue part in the image being recognized as the shadow area, the average image brightness is introduced as a collaborative judgment criterion, as shown in the following formula:

[0104]

[0105] Wherein, x is the abscissa of the pixel (x, y), y is the ordinate of the pixel (x, y), fgray(x, y) is the gray value of the pixel (x, y), ave(fgray) is the average brightness value of each pixel after converting the RGB image of the bar and wire to be detected into the gray image fgray, f(x, y, 1) is the red component pixel value of the pixel (x, y) in the RGB image of the bar and wire to be detected, f(x, y, 2) is the green component pixel value of the pixel (x, y) in the RGB image of the bar and wire to be detected, f(x, y, 3) is the blue component pixel value of the pixel (x, y) in the RGB image of the bar and wire to be detected, & is the AND operation, fo(x, y) is the recognition result of the pixel (x, y). If fo(x, y) = 0, the pixel (x, y) belongs to the shadow area, otherwise it does not belong to the shadow area;

[0106] Perform dilation processing on the recognized shadow area to avoid the appearance of black edges after shadow removal, and obtain the expanded shadow area;

[0107] Step 1.2: Perform edge detection on the expanded shadow area to obtain the shadow edge, and divide the expanded shadow area into a full shadow area and a penumbra area. The penumbra area is the transition area from the full shadow area to the non-shadow area. Eliminate the shadows in the full shadow area and the penumbra area respectively to obtain the RGB image of the bar and wire to be detected with the shadow removed;

[0108] Perform edge detection on the expanded shadow area to determine the shadow edge BW, and divide the expanded shadow area into a full shadow area and a penumbra area. The penumbra area is the transition area from the full shadow area to the non-shadow area. The light intensity change in the penumbra area is approximately an S-shaped function. In this embodiment, considering the actual production efficiency requirements, for the penumbra area, an S-shaped sine function is used for modeling to eliminate the shadow in the penumbra area; for the full shadow area, a multi-threshold segmentation algorithm is used to divide the full shadow area into several full shadow sub-areas, and the shadow in each full shadow sub-area is eliminated to avoid the influence of uneven illumination on the shadow elimination effect;

[0109] Step 1.2.1: Perform edge detection on the expanded shadow area to obtain the shadow edge, and divide the expanded shadow area into a full shadow area and a penumbra area. The penumbra area is the transition area from the full shadow area to the non-shadow area. Use an S-shaped function to model the penumbra area, and use an inverse S-shaped function model to eliminate the shadow in the penumbra area image;

[0110] Calculate the normal vector n of the pixel (a, b) on the shadow edge BW ab =(n a ,n b ), n a is the magnitude of the normal vector in the horizontal direction, n bis the magnitude of the normal vector in the vertical direction. Taking the pixel point where the shadow edge pixel value BW(a, b) = 1 as the center, with the positive direction being directly above the pixel point, determine the normal vector n of this pixel point ab of the positive pointing angle θ, and determine the transition direction d from the penumbra region to the non-shadow region according to the positive pointing angle θ, as shown in the following formula:

[0111]

[0112] where the X direction is the direction directly to the right of the pixel point, the Mid1 direction is the direction obtained by rotating 45° counterclockwise from the direction directly to the right of the pixel point, the Y direction is the direction directly above the pixel point, and the Mid2 direction is the direction obtained by rotating 45° counterclockwise from the direction directly above the pixel point, as Figure 2 shown;

[0113] Due to other scattering effects at the object edge, the approximately S-shaped curve of the light intensity change in the penumbra region will be distorted to a certain extent. Therefore, in this embodiment, model the light intensity change in the penumbra region. Assume that the penumbra region contains ω pixel points in total. Take (ω - 1) / 2 pixel points along the positive and negative directions of the transition direction d from the penumbra region to the non-shadow region, and determine the maximum value f max and the minimum value f min of the pixel values within the transition range, and establish an S-shaped function model for the light intensity change in the penumbra region to obtain the pixel value Fmod (g,h) (i) of the pixel point (g, h) within the penumbra region, as shown in the following formula:

[0114]

[0115] where g is the abscissa of the pixel point within the penumbra region, h is the ordinate of the pixel point within the penumbra region, i is the pixel point label, is the translation term of the S-shaped function, is the scaling term of the S-shaped function;

[0116] Use the inverse S-shaped function model to eliminate the shadow in the penumbra region image, eliminate the influence of the shadow on the head alignment recognition of the bar and wire, and obtain the pixel values of each pixel within the penumbra region after shadow elimination, as shown in the following formula:

[0117]

[0118] where Fres (g,h) (i) is the pixel value after shadow elimination of the pixel (g, h) within the penumbra region, and f (g,h) (i) is the pixel value before shadow elimination of the pixel (g, h) within the penumbra region;

[0119] Step 1.2.2: Use the multi-threshold segmentation algorithm to divide the full-shadow region into several full-shadow sub-regions, and perform shadow elimination on each full-shadow sub-region;

[0120] Obtain the maximum brightness value I of the grayscale image of the full-shadow region max and the minimum brightness value I min , for the full-shadow region with a step size of L, divide the full-shadow region into (I max -I min ) / L segmentation sub-regions, obtain the brightness mean value of each segmentation sub-region, and obtain the brightness mean value IN of the non-shadow region adjacent to it ave , calculate the brightness mean ratio Br between the non-shadow region and the segmentation sub-region, perform shadow elimination on each segmentation sub-region image, and the pixel value F j (e,f) after shadow elimination of the j-th segmentation sub-region is shown in the following formula:

[0121] F j (e,f) = f j (e,f) * Br + IN ave (5)

[0122] where f j (e,f) is the pixel value before shadow elimination of the j-th segmentation sub-region;

[0123] After performing shadow elimination on the shadows in the full-shadow region and the penumbra region respectively, the image of the bar and wire to be detected after shadow elimination is as Figure 3 shown, where (a) is the image of the bar and wire to be detected before shadow elimination, and (b) is the image of the bar and wire to be detected after shadow elimination;

[0124] Step 2: Convert the RGB image of the bar and wire to be detected after shadow elimination into a grayscale image, perform texture segmentation and binarization processing on the grayscale image of the bar and wire to be detected after shadow elimination, use the dilation algorithm to connect the entire bar and wire region, and extract the edge of the bar and wire to obtain the edge image of the bar and wire to be detected;

[0125] Step 2.1: Convert the RGB image of the bar and wire to be detected after shadow elimination into a grayscale image, and perform texture segmentation on the grayscale image of the bar and wire to be detected after removing the shadow twice based on the grayscale value to obtain the image of the bar and wire to be detected after texture segmentation;

[0126] For the RGB image of the bar and wire to be detected after shadow elimination, establish a three-dimensional matrix G, as shown in the following formula:

[0127]

[0128] where m and n are the pixel points in the RGB image ( m,nThe abscissa and ordinate of (), M and N are respectively the total number of rows and columns of the pixel points included in the RGB image, t = {1, 2, 3} is the dimension of the pixel points, t = 1 is the red component of the RGB image, t = 2 is the green component of the RGB image, t = 3 is the blue component of the RGB image, and the value range of each element in the three-dimensional matrix G is [0, 255];

[0129] Convert the shadow-removed RGB image of the bar and wire to be detected into a shadow-removed grayscale image of the bar and wire to be detected. The effective grayscale value f of the pixel (m, n) after conversion 1 (m, n) is shown in the following formula:

[0130] f 1 (m, n) = 0.3 * G(m, n, 1) + 0.59 * G(m, n, 2) + 0.11 * G(m, n, 3) (7)

[0131] Among them,

[0132] Perform two texture segmentations on the shadow-removed grayscale image of the bar and wire to be detected based on the grayscale value to distinguish the bar and wire features, and obtain the image of the bar and wire to be detected after two texture segmentations;

[0133] The specific method for performing texture segmentation on the shadow-removed grayscale image of the bar and wire to be detected based on the grayscale value is:

[0134] Taking the pixel ( m,n ) as the center, define a sliding window with a size of k×k, and calculate the local standard deviation f of the pixel and the pixels within the k×k neighborhood 2 (m, n), as shown in the following formula:

[0135]

[0136] Among them, U is the average value of the effective grayscale values of k 2 pixels within the k×k neighborhood, and P r is the effective grayscale value of the r-th pixel point of the pixel (m, n) within the k×k neighborhood;

[0137] For the convenience of calculation and to ensure the accuracy of texture segmentation, scale the local standard deviation f of the pixel point (m, n) within the k×k neighborhood 2 (m, n) to the interval [0, 1] to obtain the normalized local standard deviation f of the pixel point (m, n) within the k×k neighborhood 3 (m, n), which is used as the pixel value of the pixel point (m, n) after texture segmentation, as shown in the following formula:

[0138] f 3 (m, n) = f 2 (m, n) / 255 (9)

[0139] In this embodiment, with the pixel (m′, n′) as the center, k = 3 is set, and the pixel values of the pixel (m′, n′) within the 3×3 neighborhood are as follows:

[0140] P1 P2 P3 P4 P5 P6 P7 P8 P9

[0141] Among them, the effective gray value P5 of the pixel (m′, n′) = f 1 (m′, n′); the local standard deviation f 2 (m′, n′) ∈ [0, 255] is as shown in the following formula:

[0142]

[0143] Step 2.2: Perform binarization on the wire rod image to be detected after two texture segmentations to obtain a binarized wire rod image to be detected;

[0144] The specific method for performing binarization on the wire rod image to be detected after two texture segmentations is as follows:

[0145] For the wire rod image f 4 to be detected after two texture segmentations, where the pixel value of the pixel point ( m,n ) is f4 ( m,n ), set a threshold γ, screen the pixel points in the wire rod image to be detected after the second texture segmentation whose pixel values are greater than the threshold γ, set their pixel values to 1, and set the pixel values of the remaining pixel points to 0 to obtain a binarized wire rod image f 5 , as shown in the following formula:

[0146]

[0147] Among them, f 5 (m, n) is the binarized pixel value of the pixel point (m, n);

[0148] In this embodiment, since the gray value near the texture is 0.14, the threshold is set to 0.14;

[0149] Step 2.3: For the binarized wire rod image to be detected, use the dilation algorithm to connect the entire wire rod area, use the sobel operator to perform edge detection on the wire rod area, and extract the edge of the wire rod to obtain an edge image of the wire rod to be detected;

[0150] Step 2.3.1: Use an improved dilation algorithm to connect the entire wire rod area, ensure that the pixel values of the pixel points in the area where the wire rod is located are 1, and the pixel values of the pixel points in the area without the wire rod are 0, including the following steps:

[0151] (1) Perform dilation processing on the binarized wire rod image to be detected, connect the connected regions in the binarized wire rod image to be detected with a distance less than the set threshold, eliminate fragmented connected regions, and the pixel value of pixel (m, n) after dilation processing is f 6 (m, n);

[0152] (2) Delete the connected regions in the image with an area less than the set threshold. After deleting the connected regions, the pixel value of pixel ( m,n ) is f 7 (m, n);

[0153] (3) Perform closing operation on the binarized wire rod image to be detected after deleting the connected regions. Use the dilation and erosion convolution operation template nhood to scan each pixel point in the binarized wire rod image to be detected after deleting the connected regions. The dilation and erosion convolution operation template nhood performs dilation operation with the binarized wire rod image after deleting the connected regions to obtain the minimum pixel value of the pixel points in the area covered by the dilation and erosion convolution operation template nhood. Use this minimum value to replace the pixel value of the pixel point currently scanned by the dilation and erosion convolution operation template nhood, and perform erosion operation to connect the wire rod regions. In the binarized wire rod image after the closing operation, the pixel value of pixel (m, n) is f 8 (m, n), as shown in the following formula:

[0154]

[0155]

[0156] Among them, nhood is a 255×255 matrix with all element values being 1, is the dilation operation, is the erosion operation;

[0157] (4) Use a straight-line structuring element to dilate and amplify the still unconnected wire rod regions in the binarized wire rod image after the closing operation, delete the connected objects with an area less than the set threshold, and obtain the binarized wire rod image of the connected wire rod regions, where the pixel value of pixel (m, n) is f 9 (m, n);

[0158] In this embodiment, to reduce the computational amount in the repeated regions generated when the structuring element slides, speed up the operation speed, and ensure the accuracy of the identified wire rods, a straight-line structuring element with a length of 30 pixels and an angle of 80° is used to perform dilation and amplification on the image after deleting the connected regions, and then the connected objects with an area less than 8640 pixels in the image are deleted, so that the wire rod regions are connected, ensuring that there is only the wire rod region in the image, which is convenient for detecting the edge of the wire rod;

[0159] Step 2.3.2: Use the Sobel operator to detect the edges of the connected bar and wire rod regions to obtain the edge image of the bar and wire rod to be detected;

[0160] To ensure that the edge detection result is not affected by noise, the Sobel convolution factor Gx in the horizontal direction and the Sobel convolution factor Gy in the vertical direction are respectively used to perform planar convolution with the binary image of the connected bar and wire rod region to be detected, as shown in the following formula:

[0161] Hx = f 9 (m,n)*Gx, Hy = f 9 (m,n)*Gy (14)

[0162]

[0163] Among them, Hx is the convolution result using the Sobel convolution factor in the horizontal direction, Hy is the convolution result using the Sobel convolution factor in the vertical direction, and ▽f 9 is the first derivative of the binary image f 9 of the connected bar and wire rod region to be detected, and the Sobel convolution factor Gx in the horizontal direction and the Sobel convolution factor Gy in the vertical direction are:

[0164]

[0165] According to the first derivative 9 of the image f of the connected bar and wire rod region to be detected, judge the position where the bar and wire rod edges exist to obtain the edge image of the bar and wire rod to be detected f10 ( m,n ), as shown in the following formula:

[0166]

[0167] Among them, being 0 indicates that the region does not belong to the bar and wire rod edge region, and the pixel value of the pixel points in this region is set to 0, not being 0 indicates that the region belongs to the bar and wire rod edge region, and the pixel value of the pixel points in this region is set to 1;

[0168] Extract the edge effect of the bar and wire rod to be detected as Figure 4 shown, where (a) is the grayscale image of the bar and wire rod to be detected after removing the shadow, (b) is the image of the bar and wire rod to be detected after two texture segmentations, (c) is the binary image of the bar and wire rod to be detected, (d) is the binary image of the bar and wire rod to be detected after dilation processing, (e) is the binary image of the connected bar and wire rod region to be detected, and (f) is the edge image of the bar and wire rod to be detected;

[0169] Step 3: Obtain the pixel points at the edge of the head of the bar and wire material, establish a model for solving the alignment line of the head of the bar and wire material, and use the genetic algorithm to solve the slope and intercept of the alignment line of the head of the bar and wire material to obtain the fitted alignment line of the head of the bar and wire material;

[0170] Step 3.1: Obtain the pixel points (p, q) at the edge of the head of the bar and wire material, and establish a model for solving the alignment line of the head of the bar and wire material as shown in the following formula:

[0171]

[0172] where a is the slope of the alignment line of the head of the bar and wire material, b is the intercept of the alignment line of the head of the bar and wire material, ε is the number of pixel points at the edge of the head of the bar and wire material, p is the abscissa of the pixel point (p, q) at the edge of the head of the bar and wire material, and q is the ordinate of the pixel point (p, q) at the edge of the head of the bar and wire material;

[0173] Step 3.2: Use the genetic algorithm to solve the slope and intercept of the alignment line of the head of the bar and wire material to obtain the fitted alignment line of the head of the bar and wire material;

[0174] Step 3.2.1: Initialize the population size, set the crossover probability and mutation probability, and initialize a group of random sequences containing the number of individuals equal to the population size. Each individual is a random sequence, and each random sequence is a binary encoding of z bits. The first z' bits of the binary encoding are used to represent the slope a of the alignment line of the head of the bar and wire material, and the last z - z' bits of the binary encoding are used to represent the intercept b of the alignment line of the head of the bar and wire material. Use this group of random sequences as the input of the genetic algorithm to determine the initial optimal solution;

[0175] In this embodiment, the solution process of the genetic algorithm is as Figure 5 shown. Initialize the population size to 20, the crossover probability to 0.99, the mutation probability to 0.09, and initialize each random sequence as a binary encoding of 30 bits. The first 14 bits of the binary encoding are used to represent the slope a of the alignment line of the head of the bar and wire material, and the last 16 bits of the binary encoding are used to represent the intercept b of the alignment line of the head of the bar and wire material.

[0176] Step 3.2.2: Solve the fitness fit of the population as shown in the following formula:

[0177]

[0178] where τ is the number of iterations;

[0179] Step 3.2.3: Use the roulette wheel algorithm to select the individual with the largest fitness fit in the population, perform crossover and mutation on the individual with the largest fitness, and generate the optimal solution a τ of the population in the τ-th iteration, τ b;

[0180] Step 3.2.4: Update the optimal solution of the population, and determine whether the iteration times have been reached. If not, go to Step 3.2.2; otherwise, output the optimal solution, including the slope a′ of the fitted head alignment line of the bar and wire materials and the intercept b′ of the fitted head alignment line of the bar and wire materials.

[0181] Step 3.2.5: Based on the slope a′ of the fitted head alignment line of the bar and wire materials and the intercept b′ of the fitted head alignment line of the bar and wire materials, obtain the linear expression of the fitted head alignment line q′ of the bar and wire materials, as shown in the following formula:

[0182] q′ = a′p′ + b′ (19)

[0183] where p′ is the abscissa of the pixel point (p′, q′) on the head alignment line of the bar and wire materials, and q′ is the abscissa of the pixel point (p′, q′) on the head alignment line of the bar and wire materials;

[0184] In this embodiment, taking Figure 4 the gray-scale image of the bar and wire materials to be detected as an example, obtain the fitted head alignment line of the bar and wire materials, as shown in Figure 6 , and its linear expression is:

[0185] q′ = -0.0354p′ + 268.3192 (20)

[0186] Step 4: Move down the fitted head alignment line of the bar and wire materials to obtain multiple bar and wire alignment lines at different positions, and compare the number of bar and wire materials passed by the bar and wire alignment lines at different positions to detect whether all the bar and wire materials in this batch have reached the fixed-length baffle and are aligned;

[0187] Step 4.1: Perform dilation and erosion on the binary image f 5 (m, n) of the bar and wire materials to be detected, and perform noise deletion operation. After dilation and erosion and noise deletion, in the binary image f 11 of the bar and wire materials to be detected, the pixel value of the pixel point (m, n) is f 11 (m, n);

[0188] Step 4.2: Based on the fitted head alignment line of the bar and wire materials, change the value of the intercept b′ of the head alignment line of the bar and wire materials to move down the fitted head alignment line of the bar and wire materials, move the fitted head alignment line of the bar and wire materials to the position of the tail of the bar and wire materials to obtain the tail alignment line of the bar and wire materials, and count the number of bar and wire materials on the tail alignment line of the bar and wire materials;

[0189] It is set that in the binary image f 11 of the bar and wire materials to be detected after dilation and erosion and noise deletion, the pixel value of each pixel in the area where there are bar and wire materials is 1, the pixel value of each pixel in the area where there are no bar and wire materials is 0, and the pixel value of each pixel in the bar and wire gap area is 0, as shown in the following formula:

[0190]

[0191] Among them, c is the number of the pixel point in the image;

[0192] Judge the relationship between the pixel values of two adjacent pixel points on the tail alignment line of the bar and wire in sequence and count the number of bars and wires. If the pixel value of the previous pixel point is 1 and the pixel value of the next pixel point is 0, it is determined that there is one bar and wire in this area, and the number of bars and wires s on the tail alignment line of the bar and wire is obtained, as shown in the following formula:

[0193]

[0194] Step 4.3: Change the intercept b′ of the fitted head alignment line of the bar and wire at a specified step length, move down the fitted head alignment line of the bar and wire to obtain multiple middle alignment lines of the bar and wire at different positions, use the method of Step 4.2 to count the number of bars and wires on each middle alignment line of the bar and wire at different positions, and compare it with the number of bars and wires on the tail alignment line of the bar and wire to determine whether all the bars and wires have reached the fixed-length baffle and are aligned;

[0195] The rule for judging whether the bars and wires are aligned is:

[0196] Record the number of bars and wires mycount on the middle alignment line of the bar and wire at different positions w , w is the serial number of the middle alignment line of the bar and wire. After the distance between the fitted head alignment line of the bar and wire and the fixed-length baffle exceeds the specified alignment accuracy, if the number of bars and wires mycount w on the middle alignment line of the bar and wire and the number of bars and wires s on the tail alignment line of the bar and wire are still not equal, stop moving down the head alignment line of the bar and wire, and determine that not all the bars and wires have reached the baffle and are aligned. The number of bars and wires that have not reached the baffle is s - mycount w pieces; if after the head alignment line of the bar and wire is moved down to coincide with the tail alignment line of the bar and wire, there is no situation where the number of bars and wires mycount w on the middle alignment line of the bar and wire and the number of bars and wires s on the tail alignment line of the bar and wire are not equal, it is determined that all the bars and wires have reached the fixed-length baffle and are aligned, meeting the cutting requirements, as Figure 7 shown.

[0197] Compared with the existing technology, the technical solution proposed in this embodiment is more robust and universal. Applied to the unmanned cutting of bars and wires, it realizes the detection of whether the heads of bars and wires are aligned under uneven illumination. After experiments, the detection accuracy of the technical solution proposed by the present invention reaches 99.99%, and the time consumption is about 300 ms.

[0198] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions recorded in the foregoing embodiments, or perform equivalent replacements on some or all of the technical features; and these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope defined by the claims of the present invention.

Claims

1. A rod and wire head alignment detection method based on image recognition, characterized in that: The following steps are involved: Step 1: Acquire an RGB image of the rod and wire to be detected, identify and eliminate shadows in the shadow area of ​​the RGB image of the rod and wire to be detected, and obtain an RGB image of the rod and wire to be detected with the shadows removed; Step 2: Convert the RGB image of the rod and wire to be detected with the shadows removed into a grayscale image, perform texture segmentation and binarization on the grayscale image of the rod and wire to be detected with the shadows removed, use the expansion algorithm to connect the entire rod and wire area, extract the edge of the rod and wire, and obtain the edge image of the rod and wire to be detected; Step 3: Obtain the edge pixel points of the rod and wire head, establish a rod and wire head alignment line solution process model, use a genetic algorithm to solve the rod and wire head alignment line slope and the rod and wire head alignment line intercept, and obtain the fitted rod and wire head alignment line; Step 4: Move down the fitted rod and wire head alignment line to obtain several rod and wire alignment lines at different positions, compare the number of rods and wires passed by the rod and wire alignment lines at different positions, and detect whether all the rods and wires in this batch have reached the fixed-length baffle and are aligned.

2. The method for detecting the alignment of rod and wire heads based on image recognition according to claim 1, characterized in that: Step 1 includes: Step 1.1: Obtain an RGB image of the rod and wire to be detected, identify the shadow area according to the difference between the blue component and the red component and the green component of each pixel in the RGB image of the rod and wire to be detected, and perform expansion processing on the identified shadow area to obtain the expanded shadow area; The shadow area is identified based on the difference between the blue component and the red component and the green component of each pixel in the RGB image of the rod and wire to be detected. In order to avoid the blue part in the image being identified as the shadow area, the image brightness mean is introduced as a collaborative evaluation criterion, as shown in the following formula: Wherein, x is the horizontal coordinate of the pixel (x, y), y is the vertical coordinate of the pixel (x, y), fgray(x, y) is the gray value of the pixel (x, y), ave(fgray) is the average brightness value of each pixel after converting the RGB image of the rod and wire to be detected into the gray image fgray, f(x, y, 1) is the red component pixel value of the pixel (x, y) in the RGB image of the rod and wire to be detected, f(x, y, 2) is the green component pixel value of the pixel (x, y) in the RGB image of the rod and wire to be detected, f(x, y, 3) is the blue component pixel value of the pixel (x, y) in the RGB image of the rod and wire to be detected, & is an AND operation, fo(x, y) is the recognition result of the pixel (x, y), if fo(x, y) = 0, the pixel (x, y) belongs to the shadow area, otherwise it does not belong to the shadow area; The identified shadow area is expanded to avoid the appearance of black edges after the shadow is removed, and the expanded shadow area is obtained; Step 1.2: Perform edge detection on the extended shadow area to obtain the shadow edge, and divide the extended shadow area into a full shadow area and a penumbra area, wherein the penumbra area is a transition area from the full shadow area to the non-shadow area, and perform shadow elimination on the shadows of the full shadow area and the penumbra area, respectively, to obtain an RGB image of the rod and wire to be detected with the shadow eliminated.

3. The method for detecting the alignment of rod and wire heads based on image recognition according to claim 2, characterized in that: The step 1.2 comprises: Step 1.2.1: Perform edge detection on the extended shadow area to obtain the shadow edge, and divide the extended shadow area into a full shadow area and a penumbra area, where the penumbra area is the transition area from the full shadow area to the non-shadow area, use the S-type function to model the penumbra area, and use the inverse S-type function model to eliminate the shadow of the penumbra area image; Calculate the normal vector n of the pixel (a, b) on the shadow edge BW ab =(n a ,n b ), n a is the size of the normal vector in the horizontal direction, n b is the size of the normal vector in the vertical direction. The normal vector n of the pixel point is determined by taking the pixel point with the shadow edge pixel value BW(a,b)=1 as the center and the point directly above the pixel point as the positive direction. ab The positive pointing angle θ is used to determine the transition direction d from the penumbra area to the non-shadow area, as shown in the following formula: The X direction is the direction to the right of the pixel, the Mid1 direction is the direction to the right of the pixel and rotated 45° counterclockwise, the Y direction is the direction directly above the pixel, and the Mid2 direction is the direction directly above the pixel and rotated 45° counterclockwise. The light intensity change of the penumbra area is modeled. The penumbra area is assumed to contain ω pixels. (ω-1) / 2 pixels are taken in the positive and reverse directions of the transition direction d from the penumbra area to the non-shadow area, and the maximum pixel value f in the transition range is determined. max and the minimum value f min , an S-type function model is established for the light intensity change in the penumbra area to obtain the pixel value Fmod of the pixel point (g, h) in the penumbra area (g,h) (i), as shown in the following formula: Among them, g is the horizontal coordinate of the pixel point in the penumbra area, h is the vertical coordinate of the pixel point in the penumbra area, and i is the pixel point number. is the translation term of the S-type function, is the scaling term of the S-type function; The inverse S-shaped function model is used to remove the shadow of the penumbra area image to eliminate the influence of the shadow on the alignment recognition of the rod and wire head, and the pixel value of each pixel in the penumbra area after the shadow is removed is obtained, as shown in the following formula: Among them, Fres (g,h) (i) is the pixel value of the pixel (g, h) in the penumbra area after removing the shadow, f (g,h) (i) is the pixel value of the pixel (g, h) in the penumbra area before removing the shadow; Step 1.2.2: Use a multi-threshold segmentation algorithm to segment the full shadow area into several full shadow areas, and remove the shadow of each full shadow area; Get the maximum brightness I of the grayscale image in the entire shadow area max and minimum brightness I min , for the full shadow area, take L as the step length and divide the full shadow area into (I max -I min ) / L segmented sub-regions, obtain the brightness mean of each segmented sub-region, and obtain the brightness mean IN of the adjacent non-shadow region ave , calculate the brightness mean ratio Br of the non-shadow area and the segmented sub-area, remove the shadow of each segmented sub-area image, and the pixel value F of the jth segmented sub-area after removing the shadow j (e,f) is shown in the following formula: F j (e,f)=f j (e,f)*Br+IN ave (5) Among them, f j (e, f) are the pixel values ​​of the jth segmented sub-region before removing the shadow.

4. The method for detecting the alignment of rod and wire heads based on image recognition according to claim 3, characterized in that: The step 2 comprises: Step 2.1: Convert the shadow-removed RGB image of the rod and wire to be detected into a grayscale image, and perform two texture segmentations on the shadow-removed grayscale image of the rod and wire to be detected based on the grayscale value to obtain the texture segmented image of the rod and wire to be detected; Step 2.2: Binarize the image of the rod and wire to be detected after the two texture segmentations to obtain a binary image of the rod and wire to be detected; Step 2.3: For the binary image of the rod and wire to be detected, the dilation algorithm is used to connect the entire rod and wire area, and the Sobel operator is used to perform edge detection on the rod and wire area, and the edge of the rod and wire is extracted to obtain the edge image of the rod and wire to be detected.

5. The method for detecting the alignment of rod and wire heads based on image recognition according to claim 4, characterized in that: The specific method of step 2.1 is: For the RGB image of the rod and wire to be detected with shadows eliminated, a three-dimensional matrix G is established, as shown in the following formula: Among them, m and n are the pixels in the RGB image ( m,n ), M and N are the total number of rows and columns of pixels in the RGB image, respectively. t = {1, 2, 3} is the dimension of the pixel. t = 1 is the red component of the RGB image, t = 2 is the green component of the RGB image, and t = 3 is the blue component of the RGB image. The value range of each element in the three-dimensional matrix G is [0, 255]. The shadow-eliminating RGB image of the rod and wire to be detected is converted into a shadow-eliminating grayscale image of the rod and wire to be detected. The effective grayscale value f1(m,n) of the pixel (m,n) after conversion is shown in the following formula: f1(m,n)=0.3*G(m,n,1)+0.59*G(m,n,2)+0.11*G(m,n,3) (7) in, Based on the gray value, the gray image of the rod and wire to be detected with the shadow removed is subjected to two texture segmentations to identify the features of the rod and wire, and the rod and wire image to be detected after two texture segmentations is obtained; The specific method for texture segmentation of the grayscale image of the rod and wire to be detected with shadow removed based on the grayscale value is: In pixels ( m,n ) as the center, define a sliding window of size k×k, and calculate the local standard deviation f2(m,n) between the pixel and the pixels in the k×k neighborhood, as shown in the following formula: Where U is the k in the k×k neighborhood 2 The average value of the effective gray value of pixels, P r is a pixel ( m,n ) is the effective gray value of the rth pixel in the k×k neighborhood; The local standard deviation f2(m,n) of the image of the pixel point (m,n) in the k×k neighborhood is scaled to the interval [0,1] to obtain the normalized local standard deviation f3(m,n) of the image of the pixel point (m,n) in the k×k neighborhood as the pixel value of the pixel point (m,n) after texture segmentation, as shown in the following formula: f3(m,n)=f2(m,n) / 255 (9).

6. The method for detecting the alignment of rod and wire heads based on image recognition according to claim 5, characterized in that: Step 2.2 The specific method for binarizing the image of the rod and wire to be detected after two texture segmentations is as follows: For the rod and wire image f4 to be detected after two texture segmentations, the pixel value of the pixel point (m, n) is f4(m, n), and the threshold γ is set. The pixel points whose pixel values ​​in the rod and wire image to be detected after the second texture segmentation are greater than the threshold γ are screened, and their pixel values ​​are set to 1, and the pixel values ​​of the remaining pixels are set to 0, and the binary rod and wire image to be detected f5 is obtained, as shown in the following formula: Among them, f5(m,n) is the binary pixel value of the pixel point (m,n).

7. The method for detecting the alignment of rod and wire heads based on image recognition according to claim 6, characterized in that: Step 2.3 includes: Step 2.3.1: Use the expansion algorithm to connect the entire rod and wire area to ensure that the pixel value of the pixel point in the area where the rod and wire are located is 1, and the pixel value of the pixel point in the area without rod and wire is 0, including the following steps: (1) Perform dilation processing on the binary image of the rod and wire to be detected, connect the connected areas whose distance in the binary image of the rod and wire to be detected is less than the set threshold, eliminate the fragmented connected areas, and the pixel value of the pixel (m, n) after the dilation processing is f6(m, n); (2) Delete the connected regions in the image whose area is smaller than the set threshold. After deleting the connected regions, the pixels ( m,n ) is f7(m,n); (3) Perform a closing operation on the binary image of the rod and wire to be detected after deleting the connected area, and use the expansion erosion convolution operation template nhood to scan each pixel point in the binary image of the rod and wire to be detected after deleting the connected area. The expansion erosion convolution operation template nhood and the binary image of the rod and wire to be detected after deleting the connected area are expanded to obtain the minimum value of the pixel points in the area covered by the expansion erosion convolution operation template nhood. The minimum value is used to replace the pixel value of the pixel point currently scanned by the expansion erosion convolution operation template nhood, and the corrosion operation is performed to connect the rod and wire area. After the closing operation, the pixel value of the pixel (m,n) in the binary image of the rod and wire to be detected is f8(m,n), as shown in the following formula: A=f7(m,n)⊕nhood (11) Where nhood is a 255×255 matrix whose element values ​​are all 1, ⊕ is the dilation operation, is the corrosion operation; (4) using the straight line structural element to dilate and enlarge the unconnected rod and wire area in the binary rod and wire image to be detected after the closing operation, and deleting the connected objects whose area is smaller than the set threshold, so as to obtain the binary rod and wire image to be detected with connected rod and wire area, wherein the pixel value of pixel (m,n) is f9(m,n); Step 2.3.2: Use the Sobel operator to perform edge detection on the connected rod and wire regions to obtain an edge image of the rod and wire to be detected; The horizontal sobel convolution factor Gx and the vertical sobel convolution factor Gy are used to perform planar convolution with the binary image of the rod and wire to be detected in the connected rod and wire area, respectively, as shown in the following formula: Hx=f9(m,n)*Gx,Hy=f9(m,n)*Gy (13) Among them, Hx is the convolution result using the sobel convolution factor in the horizontal direction, and Hy is the convolution result using the sobel convolution factor in the vertical direction. is the first-order derivative of the binary rod and wire image f9 to be detected in the connected rod and wire area; According to the first-order derivative of the image f9 of the rod and wire to be detected in the connected rod and wire area Determine the location of the edge of the rod and wire and obtain the edge image f of the rod and wire to be detected 10 (m,n), as shown in the following formula: in, 0 means that the area does not belong to the edge area of ​​the rod or wire, and the pixel value of the pixel point in the area is set to 0. If it is not 0, it means that the area belongs to the edge area of ​​the rod or wire, and the pixel value of the pixel points in this area is set to 1.

8. The method for detecting the alignment of rod and wire heads based on image recognition according to claim 7, characterized in that: Step 3 includes: Step 3.1: Obtain the edge pixel points (p, q) of the rod and wire head, and establish a rod and wire head alignment line solution process model, as shown in the following formula: Among them, a is the slope of the rod and wire head alignment line, b is the intercept of the rod and wire head alignment line, ε is the number of pixel points on the edge of the rod and wire head, p is the abscissa of the pixel point (p, q) on the edge of the rod and wire head, and q is the ordinate of the pixel point (p, q) on the edge of the rod and wire head; Step 3.2: Use a genetic algorithm to solve the slope of the rod and wire head alignment line and the intercept of the rod and wire head alignment line to obtain a fitted rod and wire head alignment line.

9. The method for detecting the alignment of rod and wire heads based on image recognition according to claim 8, characterized in that: Step 3.2 includes: Step 3.2.1: Initialize the population size, set the crossover probability and mutation probability, initialize a set of random sequences, including the population size number of individuals, each individual is a random sequence, each random sequence is a z-bit binary code, the first z′ bit binary code is used to represent the slope a of the rod and wire head alignment line, and the last zz′ bit binary code is used to represent the intercept b of the rod and wire head alignment line. Use this set of random sequences as the input of the genetic algorithm to determine the initial optimal solution; Step 3.2.2: Solve the population fitness fit, as shown in the following formula: Among them, τ is the number of iterations; Step 3.2.3: Use the roulette algorithm to select the individual with the largest fitness fit in the population, perform crossover and mutation on the individual with the largest fitness, and generate the optimal solution a of the population for the τth iteration τ , b τ ; Step 3.2.4: Update the optimal solution of the population and determine whether the number of iterations has been reached. If not, proceed to step 3.2.

2. Otherwise, output the optimal solution, including the slope a′ of the fitted rod and wire head alignment line and the intercept b′ of the fitted rod and wire head alignment line; Step 3.2.5: Based on the slope a′ of the fitted rod and wire head alignment line and the intercept b′ of the fitted rod and wire head alignment line, a linear expression of the fitted rod and wire head alignment line q′ is obtained, as shown in the following formula: q′=a′p′+b′ (18) Among them, p′ is the horizontal coordinate of the pixel point (p′, q′) on the rod and wire head alignment line, and q′ is the horizontal coordinate of the pixel point (p′, q′) on the rod and wire head alignment line.

10. The method for detecting the alignment of rod and wire heads based on image recognition according to claim 9, characterized in that: Step 4 include: Step 4.1: Perform expansion and corrosion on the binary image of the rod and wire to be detected f5(m,n) and remove the noise. After expansion and corrosion and noise removal, the binary image of the rod and wire to be detected f 11 In the example, the pixel value of the pixel point (m,n) is f11 ( m,n ); Step 4.2: Based on the fitted rod and wire head alignment line, change the value of the rod and wire head alignment line intercept b′ to move the fitted rod and wire head alignment line downward, move the fitted rod and wire head alignment line downward to the rod and wire tail position, obtain the rod and wire tail alignment line, and count the number of rods and wires on the rod and wire tail alignment line; Set the image of the rod and wire to be detected to be binarized after expansion, corrosion and noise removal. 11 In the figure, the pixel value of each pixel in the area where the rod and wire exist is 1, the pixel value of each pixel in the area where the rod and wire do not exist is 0, and the pixel value of each pixel in the gap area between the rod and wire is 0, as shown in the following formula: Where c is the number of the pixel in the image; Determine the pixel value relationship between two adjacent pixels on the rod and wire tail alignment line in order and count the number of rods and wires. If the pixel value of the previous pixel is 1 and the pixel value of the next pixel is 0, it is determined that there is a rod and wire in the area, and the number of rods and wires on the rod and wire tail alignment line s is obtained, as shown in the following formula: Step 4.3: Change the intercept b′ of the fitted rod and wire head alignment line with a specified step length, move the fitted rod and wire head alignment line downward, and obtain multiple rod and wire middle alignment lines at different positions. Use the method in step 4.2 to count the number of rods and wires on the rod and wire middle alignment lines at different positions, and compare them with the number of rods and wires on the rod and wire tail alignment line to determine whether all the rods and wires have reached the fixed-length baffle and are aligned; The rules for judging whether the rods and wires are aligned are: Record the number of rods and wires on the middle alignment line of rods and wires at different positions mycount w , w is the serial number of the middle alignment line of the rod and wire. After the distance between the fitted rod and wire head alignment line and the fixed-length baffle exceeds the specified alignment accuracy, if the number of rods and wires on the middle alignment line of the rod and wire is mycount w If the number of rods and wires on the alignment line at the tail of the rod and wire is still not equal, the rod and wire head alignment line is stopped, and it is determined that the rods and wires have not all reached the baffle and aligned. The number of rods and wires that have not reached the baffle is s-mycount. w If the head alignment line of the rod and wire moves down to coincide with the tail alignment line of the rod and wire, the number of rods and wires that do not appear on the middle alignment line of the rod and wire is mycount w If the number s of rods and wires on the alignment line at the tail of the rod and wire is not equal, it is determined that all the rods and wires have reached the fixed-length baffle and are aligned, which meets the cutting requirements.

Images