Steel bar size detection method based on machine vision

Abstract

The invention provides a reinforcing steel bar size detection method based on machine vision, and relates to the technical field of machine vision size detection, and the method comprises the following steps: S1, image collection and transmission: obtaining image data of the surface of a reinforcing steel bar through an industrial camera, converting the image data into an image signal, and transmitting the image signal to an image processing system; s2, image preprocessing and segmentation: an image processing system adopts a denoising and double-peak threshold segmentation method to separate a background region from an obtained image signal, and reinforcing steel bar form information is obtained; compared with the existing steel bar size detection technology, the steel bar size detection device has the following beneficial effects: firstly, the straight-line distance and the included angle of the steel bar can be accurately measured, the accuracy of steel bar size measurement is obviously improved, and errors caused by misoperation in manual measurement are overcome; 2, the dimension detection of the reinforcing steel bar can be completed in a short time, and the system performs real-time image processing and automatic measurement; and thirdly, the reinforcing steel bar and the background can be effectively separated.

Description

Technical Field

[0001] The present invention relates to the technical field of machine vision dimension detection, and particularly to a method for detecting the dimensions of steel bars based on machine vision. Background Technique

[0002] Steel bars are the core part of the structure in construction projects, ensuring the strength and stability of buildings. With the complexity of construction, the specifications, dimensions, and quality of steel bars play a crucial role in structural safety. Traditional methods for detecting the dimensions of steel bars mainly rely on manual measurement or mechanical tools. This method is not only inefficient but also easily affected by human factors, resulting in measurement errors. In addition, for steel bars with complex shapes, especially bent or staggered steel bars, manual measurement methods are often difficult to accurately measure and are prone to large errors.

[0003] With the continuous development of automation technology, machine vision technology, as a non-contact measurement technology, has gradually been applied to the detection of steel bar dimensions. The machine vision system can achieve precise detection of steel bar dimensions through image acquisition and processing, with the advantages of high efficiency, precision, and being free from human factor interference.

[0004] However, the existing machine vision-based steel bar dimension detection technologies still have certain deficiencies in aspects such as image processing accuracy, detection speed, and measurement accuracy. Existing technologies usually cannot achieve efficient and accurate detection of steel bars in complex environments. In particular, the accuracy of steel bar edge detection and the precision of centerline extraction often affect the reliability of measurement results. Summary of the Invention

[0005] Aiming at the deficiencies of the above-mentioned existing technologies, the technical problem to be solved by the present invention is to provide a method for detecting the dimensions of steel bars based on machine vision, which can overcome the defects of low detection accuracy, poor efficiency, and inability to work effectively in complex environments in the existing technologies, thereby effectively improving the detection accuracy.

[0006] To solve the above technical problem, the technical solution adopted by the present invention is: The present invention provides a method for detecting the dimensions of steel bars based on machine vision, including the following steps: S1. Image acquisition and transmission: Use an industrial camera to obtain image data of the steel bar surface and convert it into an image signal for transmission to the image processing system; S2. Image preprocessing and segmentation: The image processing system uses denoising and bimodal threshold segmentation methods to separate the steel bar region and the background region from the acquired image signal, obtaining the morphological information of the steel bar; S3. Straight line detection and clustering fitting: Use a specific detection algorithm to perform straight line detection on the steel bar region, extract each cluster of straight lines, and perform clustering and rough straight line fitting according to the feature distribution; S4. Centerline fitting: An accurate centerline is obtained by using a sampling and iterative optimization fitting method. S5. Perspective transformation and physical measurement: The mapping relationship between pixel points and object points is established by using the calibration of the fiducial block to obtain the object coordinates of the key points to be measured. The actual size of the steel bar is obtained by combining mathematical methods to measure the linear distance and angle.

[0007] In the preferred solution, the following steps are further included: S6. System application and operation: The image acquisition module is controlled to adjust the camera angle and position to scan the surface of the steel bar to obtain image data. After the image data is transmitted to the image processing module, the system automatically executes the process from image acquisition to size measurement, and the detection results are fed back in real time for analysis and recording. Among them, the image processing system includes an image acquisition module, an image transmission module, and an image processing module.

[0008] In the preferred solution, in step S1, image data with a resolution of at least 1920×1080 pixels is obtained. The acquired image data is one of a grayscale image or a color image. The acquired image data is converted into an image signal and transmitted to the image processing system through the data transmission module.

[0009] In the preferred solution, in step S2, the image processing system performs denoising processing on the image signal. The bimodal threshold segmentation method is used to construct the image gray histogram and count the pixel frequencies of each gray level. The total number of gray levels is determined. The bimodal threshold is calculated according to the original formula of the bimodal threshold. Its formula is expressed as follows: ; Optimized by combining the maximum inter-class entropy method, calculate the probabilities of the steel bar area and the background at different thresholds as expressed by the formula: ; ; Among them, is the total number of image pixels, and respectively represent the probabilities of the steel bar area and the background when the threshold is , is the threshold variable traversed during the calculation process, and represent a proportional relationship, ranging from 0 to 1; The average gray levels of the steel bar area and the background are respectively expressed by the formula: ; ; Among them, and respectively represent the average gray levels of the steel bar area and the background when the threshold is , is the gray level variable; The between-class entropy , select the threshold that maximizes , compare , and determine the final segmentation threshold to separate the steel bar area from the background and obtain the morphological information of the steel bars.

[0010] In the preferred solution, in step S3, for the segmented steel bar image, the improved Hough line detection algorithm based on the random Hough forest is used to extract the line clusters; Among them, in the improved Hough line detection algorithm of the random Hough forest, each decision tree randomly samples the edge points of the image, classifies them according to the position and gradient direction features of the points, and the leaf nodes of each decision tree record the voting information of the edge points for different line parameters. The line with the voting number exceeding the set threshold is recognized as the detected line Cluster using the line cluster feature distribution. The vertical distance from point to the line defined by points and is defined as: ; Among them, represents the vertical distance from point to the specified line, is the coordinate of any point on the line to be judged, and are two points on the reference line; If the distance from any point on the line to the reference line is less than the specified threshold, it is clustered into the same cluster; Improve the clustering effect based on density peak clustering, and calculate the local density of the line: ; Among them, is the distance between line and , calculated using the above vertical distance formula, is the truncation distance, determined by calculating the th percentile of the distances between all lines, is the total number of lines, Calculate the distance between each line and the line with the closest distance among the lines with a higher density than it , and draw a decision diagram to select the clustering center; For the clustered line clusters, perform rough line fitting using the least squares method; Set the straight-line equation , and minimize the error function: ; where, is a point in the straight-line cluster, is the number of points, and solve the following system of equations: ; Obtain and values, and determine the straight-line equation of the rough fitting based on and values.

[0011] In a preferred solution, in step S4, calculate the normal vector of the given straight-line segment, and perform the initial straight-line fitting using the midpoint of the normal line segment; Adopt the iterative optimization fitting based on the least median of squares method; Set as the fitting straight-line equation, and calculate the error between the fitting straight line and each point on the straight-line segment: , take the median error to measure the fitting effect, and adjust the parameters using the gradient descent method: , , until is less than the set convergence threshold to obtain the accurate center line; where, is the learning rate.

[0012] In a preferred solution, in step S5, set a flag block in the detection scene, and obtain its image pixel coordinates and the actual physical coordinates , and use the formula: to calculate the 3x3 perspective transformation matrix ; Adopt the direct linear transformation method to construct the system of equations: ( , ); Convert it into a system of linear equations: ; where, is the coefficient matrix, is the vector obtained by expanding the matrix by columns, and use the singular value decomposition method to solve and obtain the 3x3 perspective transformation matrix ; Convert the pixel coordinates of the key points of the steel bar to be measured into the object space coordinates through the formula: ; where, ​is the inverse matrix of; According to the object coordinates, for two points and , the straight-line distance is calculated using the formula: ; Two straight lines are respectively determined by two points , and , , and the direction vector is calculated, ; And the formula for calculating the included angle is as follows: ; The actual size of the steel bar is obtained.

[0013] In the preferred solution, in step S2, when optimizing by the maximum inter-class entropy method, if , then the final segmentation threshold is ; if , calculate the variance of the image gradient. When the variance is greater than the set threshold, take as the final segmentation threshold, otherwise take as the final segmentation threshold.

[0014] In the preferred solution, in step S4, when iteratively optimizing by the least median of squares method, the learning rate is initially set to 0.01 - 0.1 and is dynamically adjusted according to the change trend of in each iteration. If the decrease amplitude of is less than 0.01 for three consecutive iterations, then is halved. If increases, then is multiplied by 0.9 and re-iterated.

[0015] In the preferred solution, in step S5, when setting the marker blocks, there are at least 4 marker blocks distributed in the image that are not on the same straight line, and the measurement accuracy error range of the actual physical coordinates of the marker blocks is 0 - 0.1 mm.

[0016] The present invention provides a method for detecting the size of steel bars based on machine vision. Compared with the existing steel bar size detection technologies, it has the following beneficial effects: First, the automated detection method based on machine vision adopted by the present invention can accurately extract the morphological information of steel bars through image processing algorithms and accurately obtain the center line of steel bars through iterative optimization fitting methods. Combining the principle of perspective transformation, it can accurately measure the straight-line distance and included angle of steel bars, significantly improving the accuracy of steel bar size measurement and overcoming the errors caused by improper operation in manual measurement; Second, the automated steel bar size detection system of the present invention can complete the size detection of steel bars in a short time through the cooperation of the image acquisition module and the image processing module. Through real-time image processing and automatic measurement, the system greatly improves the work efficiency of steel bar detection, can achieve efficient automated detection, and saves a large amount of labor costs and time. Third, by adopting advanced image processing technology, the present invention can effectively separate the steel bars from the background. Even in the case of certain degree of light change, noise interference and complex background in the construction environment, it can still stably obtain clear images of steel bars and ensure the accuracy of the detection results. In addition, the method of using calibration blocks for perspective transformation calibration further enhances the adaptability of the system to complex environments. Brief Description of the Drawings

[0017] The present invention will be further described below in conjunction with the drawings and embodiments: Figure 1 is the overall structure diagram of the vision system of the present invention; Figure 2 is the schematic diagram of the steel bar size measurement area of the present invention; Figure 3 is the steel bar image taken by the vision system of the present invention; Figure 4 is the present invention Figure 3 in the steel bar image segmentation effect diagram; Figure 5 is the present invention Figure 4 in the steel bar center line extraction effect diagram; Figure 6 is the steel bar size measurement result effect diagram of the present invention. Detailed Embodiments

[0018] To better understand the purpose, structure and function of the present invention, without conflict, the embodiments in this application and the features in the embodiments can be combined with each other. The present invention will be described in detail below with reference to the drawings and embodiments.

[0019] A steel bar size detection method based on machine vision in this embodiment uses an industrial camera to collect steel bar images, transmits them to an image processing system, and uses various algorithms to process and analyze the images, thereby achieving precise measurement of the steel bar size. It mainly includes several key links such as image acquisition and transmission, image preprocessing and segmentation, line detection and clustering fitting, center line fitting, and perspective transformation and physical measurement. Each link is closely connected. The previous link provides a data basis for the next link, and the next link is further processed and analyzed based on the results of the previous link, and finally the size information of the steel bar is obtained.

[0020] Embodiment 1 As Figures 1 to 6As shown, the steel bar size detection method based on machine vision is as follows Figure 1 As shown, it includes an image acquisition module, an image transmission module, and an image processing module. The specific process steps are as follows: S1. Image acquisition and transmission: Use an industrial camera installed on a mobile bracket to perform an all-round scan of the steel bar surface to obtain the Figure 3 image data of the steel bar as shown.

[0021] Among them, the acquired image data is one of grayscale images or color images. To ensure image quality and subsequent detection accuracy, it is required that the image resolution reaches at least 1920×1080 pixels.

[0022] According to the actual placement position, shape and size of the steel bar, and the on-site environment, flexibly adjust the shooting angle and distance of the camera to ensure that the acquired image is clear, complete and free of distortion.

[0023] After the acquisition is completed, convert the acquired image data into an image signal, and use the data transmission module to stably transmit it to the image processing system for subsequent analysis and processing.

[0024] S2. Image preprocessing and segmentation: After the image processing system receives the image data, the image processing system preprocesses the image. First, perform denoising processing on the image to remove the noise generated by factors such as uneven illumination and environmental interference during the acquisition process, so as to improve the clarity and quality of the image.

[0025] Then, use the bimodal threshold segmentation method to process the image. According to the image histogram distribution, find the threshold that makes the difference in the number of pixels on the left and right sides of the histogram the smallest. This point of the smallest difference usually corresponds to the bottom of the histogram valley, and use this as the threshold for image segmentation to separate the steel bar area and the background area, clearly presenting the steel bar contour, so as to obtain the preliminary morphological information of the steel bar. The specific calculation steps are as follows: Construct a grayscale histogram for the denoised image and count the pixel frequencies of each gray level , and determine the total number of gray levels of the image , among which, the common 8-bit image .

[0026] Calculate the bimodal threshold according to the original formula of the bimodal threshold , and its formula is expressed as follows: ; Among them, represents the finally calculated bimodal threshold, which is a key parameter for image segmentation, is the variable that traverses the gray levels during the calculation process. By calculating and comparing different values, find the one that makes the absolute value in the formula the smallest , at this time the value is , which represents the pixel frequency of the th gray level in the image histogram, reflecting the frequency of occurrence of this gray level in the image, and is the basic data for calculation. is the total number of gray levels of the image, which limits the calculation range. For example, the gray level range of an 8-bit image is 0-255, so .

[0027] Specifically, the double-peak threshold formula is based on the image histogram distribution. By finding the gray value that minimizes the difference in the number of pixels on the left and right sides of the histogram, it is determined as the threshold for image segmentation. This gray value usually corresponds to the bottom of the valley between the two peaks in the histogram. Using this threshold, different gray-level regions belonging to steel bars and the background in the image can be distinguished.

[0028] In the preferred solution, as Figure 4 shown, the steel bar area is displayed in white and the background area is displayed in black, so as to obtain a clear outline of the steel bars. Using the calculated threshold , the denoised image is binarized. The part of the image with a gray value lower than the threshold is determined as the steel bar area, and the part with a gray value higher than the threshold is determined as the background area, thus realizing the separation of the steel bar area and the background area and obtaining a rough outline image of the steel bars.

[0029] Furthermore, on the basis of the double-peak threshold segmentation, the maximum inter-class entropy method is introduced for optimization.

[0030] Calculate the probabilities of the steel bar area and the background at different thresholds , and the formula is as follows: ; ; where is the total number of pixels in the image, and respectively represent the probabilities of the steel bar area and the background when the threshold is , is the threshold variable traversed during the calculation process, and have the same meaning as the above double-peak threshold formula, is the total number of pixels in the image, used for normalization calculation, and represent a proportional relationship, ranging from 0 to 1.

[0031] Specifically, these two formulas are used to calculate the proportions of the foreground and background in the image at different thresholds, providing basic data for subsequent calculation of the between-class entropy. The average grayscales of the steel bar region and the background are expressed by the formulas as follows: ; ; where and respectively represent the average grayscales of the steel bar region and the background when the threshold is , is the grayscale level variable, which is used to calculate the contribution of pixels at different grayscale levels to the average grayscale during the summation process.

[0032] Specifically, the average grayscales of the foreground and background at different thresholds are calculated through these two formulas, reflecting the respective grayscale characteristics of the foreground and background, and are used to measure the grayscale differences between the foreground and background at different thresholds.

[0033] The between-class entropy is expressed by the formula as follows: ; where represents the between-class entropy when the threshold is , is the logarithm operation with base 2, which is used to calculate the information entropy.

[0034] Specifically, this formula obtains the between-class entropy by calculating the relationship between the probabilities and average grayscales of the steel bar region and the background. The larger the between-class entropy, the greater the difference between the steel bar region and the background. By finding the threshold that maximizes , the optimal threshold for distinguishing the steel bar from the background can be found more accurately.

[0035] Calculate the between-class entropy at different thresholds, select the threshold that makes reach the maximum value, and compare with the bimodal threshold ; If the difference between the two is within a certain reasonable range, that is, , then take the average of the two as the final segmentation threshold; If the difference is large, that is, , then calculate the variance of the image gradient to judge the complexity of the image, that is, the larger the variance, the more complex the image. If the image is complex, then select as the final threshold, otherwise select as the final threshold.

[0036] In this way, the steel bar and the background can be segmented more accurately in different image scenarios.

[0037] S3. Straight line detection and clustering fitting: Perform straight line detection on the segmented steel bar image, extract the straight line clusters of each segment. According to the distribution characteristics of the straight line clusters, use a specific clustering algorithm to cluster similar straight lines, and perform rough straight line fitting on the clustered straight lines to prepare for accurately determining the center line of the steel bar in the subsequent steps. The specific calculation steps are as follows: Improve the Hough straight line detection based on the random Hough forest, and use the basic principle of the Hough straight line detection to map the edge points in the image to the Hough space to detect straight lines. Considering the problems that the traditional Hough straight line detection is sensitive to noise and has a large amount of calculation, introduce the random Hough forest algorithm for improvement.

[0038] The random Hough forest consists of multiple decision trees. Each decision tree randomly samples the edge points in the image and classifies them according to the characteristics of the sampled points such as position and gradient direction. In the Hough space, the existence and parameters of the straight line are determined through the voting results of multiple decision trees.

[0039] The specific operation process is as follows: For each edge point in the image, randomly select a decision tree for processing. The decision tree assigns the point to different leaf nodes according to the characteristics of the point, and the leaf nodes record the voting information of the point for different straight line parameters. After all edge points are processed, count the number of votes for each straight line parameter, and identify the straight lines with the number of votes exceeding the pre-set threshold as the detected straight lines, and then obtain multiple clusters of straight lines.

[0040] Adopt a clustering method based on the feature distribution of the straight line clusters. The core condition for clustering is that if the distance from any point on a straight line to the reference straight line is less than the specified threshold, it is determined that this straight line is close enough to the reference straight line and can be clustered into the same cluster. The distance from the point to the straight line defined by the points and is calculated as the perpendicular distance and is expressed as follows by the formula: ; where represents the perpendicular distance from the point to the specified straight line, which is the key basis for judging whether the straight lines belong to the same cluster. is the coordinate of any point on the straight line to be judged, and are two points on the reference straight line used to determine the position and direction of the reference straight line. The numerator part of the formula obtains the signed distance from the point to the straight line through coordinate operations, and the denominator part is used to normalize the distance to ensure the accuracy and consistency of the distance calculation.

[0041] Specifically, the distance from a point on a line to a reference line is calculated through this formula. Based on the comparison result between the distance and a specified threshold, it is determined whether the line is close enough to the reference line, thereby realizing the clustering operation of the lines.

[0042] To improve the clustering effect based on density peak clustering, the density peak clustering algorithm is introduced to further optimize the clustering effect. First, calculate the local density of each line, and its formula is as follows: ; where is the distance between line and line , which is calculated using the above-mentioned perpendicular distance formula from a point to a line, reflecting the spatial relationship between the two lines. is the truncation distance, which is used to control the neighborhood range of density calculation and avoid the influence of lines with too large a distance on density calculation. It is determined by calculating the percentile of the distances between all lines, that is , , is the total number of lines, which is used to determine the summation range. represents the local density of the th line, which is used to measure the density of the lines around the line. is the exponential function, which is used to convert the distance information into density weights.

[0043] Specifically, this formula obtains the local density of each line by weighted summation of the distances to other lines around the line, providing an important basis for subsequent determination of the clustering center.

[0044] At the same time, calculate the distance between each line and the line with the closest distance among the lines with a density greater than it .

[0045] represents the distance between the th line and the line with the closest distance among the lines with a density greater than it, which is used to assist in determining the clustering center in the decision graph. It reflects the relative position relationship of the line in the density distribution.

[0046] Specifically, by combining and to draw a decision graph, it is possible to intuitively find the points with both large density and large distance as the clustering centers, thereby clustering the line clusters more reasonably.

[0047] Taking as the ordinate, Draw a decision diagram with the abscissa, and select points with larger density and distance in the decision diagram as the clustering centers. For other lines, according to their distances from the clustering centers, allocate them to the corresponding clusters. This method can cluster the line clusters more reasonably and effectively avoid the misjudgment problems that may occur in traditional clustering methods when dealing with line clusters with complex distributions.

[0048] For each line cluster after clustering, use the least squares method to perform rough line fitting. The specific steps are as follows: Let the line equation be: , for the points in the line cluster, by minimizing the error function to determine the parameters and of the line; where, is the general equation form of the line, is the slope of the line, which determines the inclination degree of the line; is the intercept of the line on the axis, which determines the position of the line. are the coordinates of each point in the line cluster, used to fit the line. is the error function, used to measure the deviation degree between the fitted line and the actual points. is the number of points in the line cluster.

[0049] Specifically, by minimizing the error function , the most suitable and values can be found, so that the fitted line is as close as possible to each point in the line cluster, thereby obtaining the roughly fitted line equation.

[0050] For and , find the partial derivatives respectively and set the partial derivatives to 0, and the following system of equations is obtained: ; Solving this system of equations can obtain the and values, thereby determining the roughly fitted line equation.

[0051] S4. Centerline fitting: Adopt the method of sampling and iterative optimization fitting to further process the previously obtained rough line fitting result, gradually optimize it, and finally obtain the accurate centerline of the steel bar; The specific calculation steps are as follows: For a given line segment, first calculate its normal vector; Then, use the midpoint of the normal line segment for line fitting to obtain an initial centerline estimate.

[0052] In the preferred solution, the traditional sampling and iterative optimization fitting method is vulnerable to outliers, resulting in inaccurate fitting results.

[0053] As Figure 5 shown, to ensure that the obtained center line of the steel bar is as close as possible to the actual shape of the steel bar, the Least Median of Squares (LMedS) is used for improvement.

[0054] Calculate the error between the fitting line and each point on the line segment, and select the median of the errors as the standard for measuring the fitting effect.

[0055] Let the equation of the fitting line be: , for the point on the line segment; Then the formula for the error between the fitting line and each point on the line segment is expressed as follows: ; Among them, represents the median of the errors. Compared with the average value, the median is more robust to outliers and can more accurately reflect the deviation between the fitting line and the actual points. and are the slope and intercept of the current fitting line. is the coordinate of the point on the line segment. is the error value of each point to the fitting line.

[0056] Specifically, by calculating the error of each point to the fitting line and then taking the median of the errors, this is used as an index to measure the fitting effect, avoiding the excessive influence of outliers on the fitting result.

[0057] By continuously adjusting the parameters and of the fitting line, recalculate the median of the errors; Furthermore, the gradient descent method is used to adjust the parameters: ; ; Among them, is the learning rate, which is used to control the step size of each parameter adjustment. It determines the convergence speed of the iterative process, and are the partial derivatives of the median of the errors with respect to the current slope and intercept , respectively, which are used to indicate the direction of parameter adjustment.

[0058] During the iterative optimization of the Least Median of Squares, the learning rate The initial value is set to 0.01 - 0.1 and is dynamically adjusted according to the changing trend in each iteration. If the decrease amplitude is less than 0.01 for three consecutive iterations, then it is halved. If an increase occurs, then it is multiplied by 0.9 and re - iterated.

[0059] Specifically, according to the partial derivative of the parameter with respect to the median error and the learning rate, the slope and intercept of the fitted straight line are continuously adjusted so that the fitted straight line gradually approaches the true center line until it is less than the pre - set convergence threshold, at which point the iteration stops. The straight line obtained at this time is the accurate center line of the steel bar.

[0060] S5. Perspective Transformation and Physical Measurement: Based on the principle of perspective transformation, by setting fiducial blocks and calibrating, an accurate mapping relationship between pixel points and object - space points is established. Using this mapping relationship, the object - space coordinates of the key points to be measured are obtained, and then combined with corresponding mathematical methods, the accurate measurement of the straight - line distance and angle of the steel bar is realized, thereby obtaining the actual size of the steel bar.

[0061] When setting the fiducial blocks, there are at least 4 fiducial blocks whose distributions in the image are not on the same straight line, and the measurement accuracy error range of the actual physical coordinates of the fiducial blocks is 0 - 0.1 mm.

[0062] The specific steps are as follows: Set multiple fiducial blocks in the steel bar detection scenario, and accurately obtain the pixel coordinates and actual physical coordinates of the fiducial blocks in the image. Through the perspective transformation unit, the pixel coordinates of the key points to be measured are converted into physical coordinates. The perspective transformation matrix from one set of source points to another set of target points can be used to transform an image or a point set from one perspective to another. Calculate the 3x3 perspective transformation matrix whose principle formula is as follows: ; Among them, the homogeneous coordinate is usually set to 1, represents the pixel coordinates of the fiducial block in the image, which indicates the position information of the fiducial block on the two - dimensional image plane and is the coordinate value directly obtained from the image data. is the actual physical coordinate of the fiducial block, which reflects the position of the fiducial block in the real physical space and is obtained through actual measurement or known calibration information. is a 3x3 perspective transformation matrix, which is the key to establishing the mapping relationship between pixel points and object points. By calculating and applying this matrix, the pixel coordinates in the image can be converted into corresponding actual physical coordinates. "~" represents a transformation relationship, that is, the pixel coordinate vector on the left has a corresponding relationship with the physical coordinate vector on the right after being transformed by the matrix .

[0063] Specifically, this formula constructs the connection between pixel coordinates and physical coordinates, providing a basic framework for subsequent calculation of the perspective transformation matrix. By substituting the pixel coordinates and physical coordinate information of multiple fiducial blocks into this formula, a system of equations can be constructed, and then the perspective transformation matrix can be solved .

[0064] To calculate more precisely, the direct linear transformation (DLT) method is used for solution.

[0065] Construct a system of equations: ; where , is the number of fiducial blocks, and , convert it into a system of linear equations , where is the coefficient matrix, is the vector formed by expanding the matrix by columns, and then use the singular value decomposition (SVD) method to solve this system of equations to obtain the accurate perspective transformation matrix .

[0066] Furthermore, are the elements of the perspective transformation matrix , which determine the specific parameters of the perspective transformation. represents the number of fiducial blocks. A sufficient number of fiducial blocks ( ) can provide sufficient constraint conditions to accurately calculate each element of the matrix. is the coefficient matrix constructed according to the fiducial block coordinates, which integrates the information of the pixel coordinates and physical coordinates of the fiducial blocks and is used to solve the system of linear equations. is the vector formed by expanding the matrix by columns, converting the matrix form into a vector form for convenient solution using linear algebra methods.

[0067] Specifically, by constructing these systems of equations, the perspective transformation matrix The problem is transformed into solving a system of linear equations. Using the SVD method to solve this system of equations can improve the accuracy and stability of the calculation, overcome the potential numerical instability problem in direct solution, and thus obtain a more accurate perspective transformation matrix. .

[0068] Apply the calculated transformation matrix , and convert the pixel coordinates of the key points of the steel bar to be measured into object space coordinates . The specific conversion formula is as follows: ; where, is the inverse matrix of is the pixel coordinates of the key points of the steel bar to be measured in the image, which are the coordinate information obtained from the processed image. is the object space coordinates obtained after coordinate conversion, that is, the coordinates of the key points of the steel bar in the actual physical space, which are used for subsequent dimension calculation. is the inverse matrix of the perspective transformation matrix . By left-multiplying the pixel coordinate vector by , the conversion from pixel coordinates to object space coordinates is realized.

[0069] Specifically, this formula uses the previously calculated perspective transformation matrix and its inverse matrix to realize the conversion from image pixel coordinates to actual physical coordinates, providing a coordinate basis for accurately measuring the dimensions of the steel bar.

[0070] According to the obtained object space coordinates, combine mathematical methods to calculate the straight-line distance and angle of the steel bar.

[0071] Furthermore, for two points and , the straight-line distance is: ; where, and are the object space coordinates of two points to be measured on the steel bar. represents the straight-line distance between these two points, which is an important parameter for measuring the dimensions of the steel bar. The square sum and square root operations in the formula are used to calculate the Euclidean distance based on the coordinates of two points.

[0072] Specifically, through this formula, the actual straight-line distance between any two points on the steel bar can be calculated, providing a calculation method for determining the dimensions such as the length and spacing of the steel bar.

[0073] For two straight lines, respectively composed of two points , and , It is determined that their direction vectors are respectively: ; ; The included angle is: ; Wherein, , , , are respectively the object coordinates of the points on the two straight lines. and are the direction vectors of the two straight lines, which are used to describe the direction of the straight lines. is the dot product of the two vectors, which reflects the similarity degree of the two vectors in direction. and are respectively the moduli of the vectors and , that is, the lengths of the vectors. represents the included angle between the two straight lines, which is calculated through the inverse cosine function.

[0074] Specifically: Using this formula, the included angle between the two straight lines on the steel bar can be calculated, which is very crucial for measuring the bending angle, crossing angle and other dimensional characteristics of the steel bar, thus realizing the accurate measurement of the actual dimensions of the steel bar completely, obtaining the actual dimensions of the steel bar, and the dimensional measurement results are as Figure 6 shown S6. System Application and Operation Process. At the actual construction site, the operator adjusts the camera angle and position of the image acquisition module to scan the surface of the steel bar to be measured and obtain image data. After the image data is transmitted to the image processing module, the system automatically executes the above processing process, from image acquisition, edge detection, straight line detection, center line fitting to perspective transformation and physical measurement, to realize automatic steel bar dimension detection. The detection results are fed back to the operator in real time for further analysis and recording.

[0075] Through the specific description of the above embodiments, it can be seen that the present invention realizes the accurate detection of the steel bar dimensions through machine vision technology combined with advanced image processing algorithms. This technology can not only operate efficiently in complex construction environments, but also ensure the accuracy and reliability of the detection results, and is applicable to the automatic detection task of steel bar dimensions Embodiment 2 Further described in combination with Embodiment 1, the embodiment of the present application provides an electronic device, which may include: A memory, a processor, and a computer program stored on the memory and executable on the processor.

[0076] When the processor executes the program, it implements the method for detecting the size of steel bars based on machine vision provided in the above embodiments.

[0077] Furthermore, the electronic device further includes: A communication interface for communication between the memory and the processor. A memory for storing a computer program that can run on the processor.

[0078] The memory may include a high-speed RAM memory and may also include non-volatile memory, such as at least one disk memory.

[0079] If the memory, the processor, and the communication interface are implemented independently, the communication interface, the memory, and the processor can be interconnected through a bus and communicate with each other. The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 4 only a thick line is shown in the figure, but it does not mean that there is only one bus or one type of bus.

[0080] The processor may include one or more processing units. For example, the processor may include an application processor (AP), an Application Specific Integrated Circuit (ASIC), a modem processor, a Central Processing Unit (CPU), an image signal processor (ISP), a controller, a memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Among them, different processing units may be independent devices or integrated in one or more processors. Among them, the controller may be the nerve center and command center. The controller can generate operation control signals according to the instruction operation code and timing signal to complete the control of fetching and executing instructions. A memory may also be provided in the processor for storing instructions and data. In some embodiments, the memory in the processor is a cache memory. This memory can save the instructions or data that the processor has just used or recycled. If the processor needs to use the instruction or data again, it can be directly called from the memory. This avoids repeated accesses, reduces the waiting time of the processor, and thus improves the efficiency of the system.

[0081] A visualization module for displaying images, videos, etc. The visualization module may include a display panel, which may be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), a Miniled, a MicroLed, a Micro-oLed, a quantum dot light-emitting diode (QLED), etc.

[0082] Optionally, in specific implementations, if the memory, the processor, and the communication interface are integrated on a single chip, the memory, the processor, and the communication interface can communicate with each other through an internal interface.

[0083] The embodiments of the present application further provide a computer-readable storage medium, on which a computer program is stored, and when the program is executed by a processor, the above-mentioned steel bar size detection method based on machine vision is implemented.

[0084] The embodiments of the present application further provide a computer program product, the computer program can run computer instructions, and when the computer instructions are executed by a processor, the above-mentioned steel bar size detection method based on machine vision is implemented.

[0085] The logic and / or steps represented in the flowchart or described in other ways herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be specifically implemented in any computer-readable medium for use by an instruction execution system, apparatus or device (such as a computer-based system, a system including a processor, or other systems that can fetch instructions from the instruction execution system, apparatus or device and execute the instructions), or in combination with these instruction execution systems, apparatus or devices. For the purposes of this specification, a "computer-readable medium" can be any device that can contain, store, communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus or device. More specific examples (non-exhaustive list) of computer-readable media include the following: an electrical connection part (electronic device) having one or N wirings, a portable computer disk cartridge (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disc read-only memory (CDROM). Additionally, the computer-readable medium can even be paper or other suitable media on which the program can be printed, because the program can be obtained electronically by optically scanning the paper or other media, followed by editing, interpretation, or otherwise processing as appropriate, and then storing it in a computer memory.

[0086] The above-mentioned storage medium can be a read-only memory, a disk, or an optical disc, etc. Although the embodiments of the present application have been shown and described above, it can be understood that the above embodiments are exemplary and should not be construed as limiting the present application. Those of ordinary skill in the art can make changes, modifications, substitutions, and variations to the above embodiments within the scope of the present application.

Claims

1. A method for detecting steel bar size based on machine vision, characterized in that: The following steps are involved: S1. Image acquisition and transmission: Use industrial cameras to obtain image data on the steel bar surface, convert it into image signals and transmit it to the image processing system; S2. Image preprocessing and segmentation: The image processing system uses denoising and bimodal threshold segmentation method to separate the image signal and the background area to obtain the steel bar morphology information; S3, straight line detection and cluster fitting: use a specific detection algorithm to perform straight line detection on the steel bar area, extract each straight line cluster and perform clustering and rough straight line fitting according to feature distribution; S4, centerline fitting: using sampling and iterative optimization fitting method to obtain accurate centerline; S5. Perspective transformation and physical measurement: Use marker block calibration to establish the mapping relationship between pixel points and object points, obtain the object coordinates of the key points to be measured, and use mathematical methods to measure the straight-line distance and angle to obtain the actual size of the steel bar.

2. The steel bar size detection method based on machine vision according to claim 1 is characterized in that: The following steps are also included: S6, system application and operation: controlling the image acquisition module to adjust the camera angle and position, scanning the steel bar surface to obtain image data, after the image data is transmitted to the image processing module, the system automatically executes the process from image acquisition to dimension measurement, and feeds back the detection results in real time to perform analysis and recording; The image processing system includes an image acquisition module, an image transmission module and an image processing module.

3. The steel bar size detection method based on machine vision according to claim 2 is characterized in that: In step S1, image data with a resolution of at least 1920×1080 pixels is acquired, and the acquired image data is a grayscale image or a color image. The acquired image data is converted into an image signal and transmitted to an image processing system through a data transmission module.

4. The method for detecting steel bar size based on machine vision according to any one of claims 1 to 3, characterized in that: In step S2, the image processing system performs denoising processing on the image signal; The bimodal threshold segmentation method is used to construct the image grayscale histogram and count the frequency of pixels at each grayscale level. , determine the total number of gray levels , calculate the bimodal threshold according to the original formula of the bimodal threshold , the formula is as follows: ； Combined with the maximum inter-class entropy method optimization, calculate different thresholds The probability of the lower reinforcement area and the background is expressed by the formula as follows: ； ； in, is the total number of pixels in the image, and Respectively represent the threshold The probability of the steel bar area and the background, is the threshold variable traversed during the calculation process, and It represents a proportional relationship, ranging from 0 to 1; The average grayscale of the steel bar area and the background are expressed by the formula: ； ； in, and Respectively represent the threshold The average grayscale of the steel bar area and the background, is a gray-level variable; Inter-class entropy , select Maximum threshold ,Compare , and determine the final segmentation threshold to separate the steel bar area from the background and obtain the steel bar morphology information.

5. The method for detecting steel bar size based on machine vision according to claim 4 is characterized in that: In step S3, for the segmented steel bar image, a straight line cluster is extracted based on the random Hough forest improved Hough line detection algorithm; In the random Hough forest improved Hough line detection algorithm, each decision tree randomly samples the edge points of the image, classifies them according to the location and gradient direction characteristics of the points, and each decision tree leaf node records the voting information of the edge points for different line parameters. The line with a voting number exceeding the set threshold is identified as the detected line. Clustering is performed using linear cluster feature distribution, point To point and Define the vertical distance of the line: ； in, Indicate point The vertical distance to the specified line, is the coordinate of any point on the line to be judged, and are two points on the reference line; If the distance from any point on the straight line to the reference straight line is less than the specified threshold, they are clustered into the same cluster; Improve the clustering effect based on density peak clustering and calculate the local density of the line: ； in, For straight line and The distance is calculated by using the vertical distance Formula calculation, is the cutoff distance, which is calculated by Percentile determination, is the total number of straight lines, Calculate the distance between each line and the line closest to it that has a higher density than it , and draw a decision diagram to select the cluster center; For the clustered straight line clusters, the least square method is used to perform rough straight line fitting; Let the equation of the line , by minimizing the error function: ； in, is a point in the line cluster, is the number of points, solve the following system of equations: ; get and The value of and The value of determines the equation of the roughly fitted straight line.

6. The method for detecting steel bar size based on machine vision according to claim 5, characterized in that: In step S4, a normal vector is calculated for a given straight line segment, and an initial straight line fitting is performed using the midpoint of the normal line segment; An iterative optimization fitting based on the least median square method was used; set up To fit the straight line equation, and calculate the error between the fitted straight line and each point on the straight line segment: , take the median error Measure the fitting effect and use the gradient descent method to adjust the parameters: , , until If it is smaller than the set convergence threshold, the precise center line is obtained; in, is the learning rate.

7. The method for detecting steel bar size based on machine vision according to claim 4, characterized in that: In step S5, a marker block is set in the detection scene and its image pixel coordinates are obtained. and the actual physical coordinates , using the formula: Calculate the 3x3 perspective transformation matrix ; The system of equations is constructed using the direct linear transformation method: （ ， ）； Transformed into a system of linear equations: ; in, is the coefficient matrix, for The matrix expands the vector by column and uses the singular value decomposition method to solve the 3x3 perspective transformation matrix ; The pixel coordinates of the key points of the steel bar to be tested By formula: Convert to object space coordinates ; in, for The inverse matrix of According to the object coordinates, for two points and , use the formula to calculate the straight-line distance: ; Two straight lines are drawn from two points , and , Determine and calculate the direction vector , ; The formula for calculating the angle is as follows: ； Get the actual size of the steel bar.

8. The method for detecting steel bar size based on machine vision according to claim 4, characterized in that: In step S2, when the maximum inter-class entropy method is optimized, if , then the final segmentation threshold is ;like , calculate the image gradient variance, and take is the final segmentation threshold, otherwise is the final segmentation threshold.

9. The method for detecting steel bar size based on machine vision according to claim 6, characterized in that: In step S4, during the minimum median squares iterative optimization, the learning rate The initial value is set to 0.01-0.1, and the Dynamic adjustment of the changing trend, if three consecutive iterations If the decrease is less than 0.01, Halved, if If an increase occurs, Multiply by 0.9 and iterate again.

10. The method for detecting steel bar size based on machine vision according to claim 7, characterized in that: In step S5, when setting the marker blocks, the distribution of the marker blocks in the image includes at least 4 marker blocks that are not on the same straight line, and the actual physical coordinate measurement accuracy error range of the marker blocks is 0-0.1 mm.

Images