Capillary length determination method, device, electronic device and storage medium

Abstract

The present invention relates to the technical field of online detection of steel pipe dimensions, and in particular to a method, device, electronic device, and storage medium for determining capillary pipe length. The method of the present invention first obtains a first capillary pipe image and a second capillary pipe image; then, based on the shooting positions of corresponding cameras, the end edge lines of the first capillary pipe image and the end edge lines of the second capillary pipe image are identified to obtain a first end edge line and a second end edge line; then, image coordinate transformation is performed on the coordinates of the first end edge line in the first capillary pipe image and the coordinates of the second end edge line in the second capillary pipe image, and the obtained coordinates of the first end edge line in the camera coordinate system and the obtained coordinates of the second end edge line in the camera coordinate system are used as first camera coordinates and second camera coordinates, respectively; finally, the capillary pipe length is determined based on the relative coordinates of the two cameras, the first camera coordinates, and the second camera coordinates. The present invention ensures the accuracy and stability of capillary pipe length detection.

Description

Technical Field

[0001] The present invention relates to the technical field of online steel pipe size detection, and in particular to a capillary length determination method, device, electronic equipment and storage medium. Background Art

[0002] Steel pipe roughing refers to a hollow, long, semi-finished product with a defined size and surface quality, obtained by processing steel ingots or billets through processes such as perforation during the steel pipe production process. It serves as the foundation for subsequent processing into steel pipes of various specifications and applications. Steel pipe roughing is primarily used as an intermediate raw material for further processing. Through subsequent processes such as hot rolling, cold rolling, and cold drawing, it can be processed into finished steel pipes of various specifications and applications. These are widely used in a variety of fields, including construction, machinery, petrochemicals, shipbuilding, and aerospace.

[0003] Online detection of capillary tube production refers to the real-time detection of multiple parameters including capillary tube length during the capillary tube production process, so as to adjust the production equipment in time to meet the quality requirements of the capillary tube.

[0004] At present, the mainstream technologies for online detection of capillary outer diameter include: machine vision detection method, photoelectric online diameter measuring instrument detection method, and ultrasonic detection method. Among them, machine vision detection method is highly praised due to its high environmental adaptability.

[0005] However, the main problem with the machine vision detection method is that the positioning of the measured end point of the capillary is inaccurate, resulting in large errors in the detection results.

[0006] Based on this, it is necessary to develop and design a method for determining the capillary length. Summary of the Invention

[0007] The embodiments of the present invention provide a method, device, electronic device and storage medium for determining the length of a capillary tube, which are used to solve the problem of inaccurate online capillary tube length detection results based on machine vision in the prior art.

[0008] In a first aspect, an embodiment of the present invention provides a method for determining capillary length, comprising:

[0009] Acquire a first capillary tube image and a second capillary tube image, wherein the first capillary tube image and the second capillary tube image are respectively acquired from two fixed cameras;

[0010] Identifying the end edge line of the first capillary tube image and the end edge line of the second capillary tube image according to the shooting position of the corresponding camera to obtain a first end edge line and a second end edge line;

[0011] performing image coordinate transformation on the coordinates of the first end edge line in the first capillary image and the coordinates of the second end edge line in the second capillary image, and using the obtained coordinates of the first end edge line in the camera coordinate system and the obtained coordinates of the second end edge line in the camera coordinate system as first camera coordinates and second camera coordinates, respectively;

[0012] The capillary length is determined according to the relative coordinates of the two cameras, the first camera coordinates, and the second camera coordinates.

[0013] In one possible implementation, identifying the end edge line of the first capillary tube image and the end edge line of the second capillary tube image according to the shooting position of the corresponding camera to obtain the first end edge line and the second end edge line includes:

[0014] For any one of the first capillary tube image and the second capillary tube image, perform the following steps respectively:

[0015] Perform Gaussian filtering on the capillary image to obtain a smooth capillary image;

[0016] Performing a gradient operation on the smooth capillary image using an operator to obtain a gradient capillary image;

[0017] performing non-maximum suppression on the gradient capillary image to obtain a first edge capillary image;

[0018] performing dual-threshold detection on the first edge capillary tube image to obtain a second edge capillary tube image;

[0019] The second edge capillary image is fed into a deep learning model to identify the image block where the end edge line is located.

[0020] In one possible implementation, the deep learning model is trained based on a plurality of sample images, wherein the sample label construction process includes:

[0021] For each sample image, perform the following steps:

[0022] Acquire a target edge point, wherein the target edge point is located on an edge line at an end of the sample image;

[0023] Taking the target edge point as the center, obtaining multiple first edge points within a preset neighborhood radius;

[0024] According to the first formula and the plurality of first edge points, an edge line equation expressing the end edge line is constructed using the least squares method, wherein the first formula is:

[0025]

[0026] Where, 、 、 、 、 as well as are the first coefficient, the second coefficient, the third coefficient, the fourth coefficient, the fifth coefficient and the sixth coefficient respectively, is the x-axis coordinate of the first edge point in the sample image, is the y-axis coordinate of the first edge point in the sample image;

[0027] Substituting the plurality of first edge points into the edge line equation to obtain a plurality of deviation values;

[0028] taking the maximum value among the plurality of deviation values ​​as a reference value;

[0029] Taking the target edge point as the center, and according to the edge line equation, obtaining a plurality of second edge points, wherein a value obtained by substituting the second edge point into the edge line equation is not greater than the reference value, and the plurality of second edge points are continuous;

[0030] Using a minimum rectangle surrounding the plurality of second edge points as a marking rectangle;

[0031] The center coordinates of the marking rectangle, the width of the marking rectangle, and the length of the marking rectangle are used as sample labels.

[0032] In one possible implementation, performing image coordinate transformation on the coordinates of the first end edge line in the first capillary image and the coordinates of the second end edge line in the second capillary image, and using the obtained coordinates of the first end edge line in the camera coordinate system and the obtained coordinates of the second end edge line in the camera coordinate system as the first camera coordinates and the second camera coordinates, respectively, includes:

[0033] For each end edge line, perform the following steps:

[0034] The end edge line is fitted using the least square method to obtain the first ellipse general equation describing the end edge line;

[0035] Formulating the first ellipse general equation, and determining a first focus and a second focus according to a result of the formulation;

[0036] The midpoint between the first focus and the second focus is used as the center point of the capillary;

[0037] The coordinates of the center point of the capillary tube are converted into a camera coordinate system to obtain camera coordinates.

[0038] In one possible implementation, formulating the first ellipse general equation and determining the first focus and the second focus according to the formulation result includes:

[0039] If the first elliptic general equation has a cross term, a second elliptic general equation is obtained by performing a rotation transformation on the first elliptic general equation according to a second formula, wherein the second formula is:

[0040]

[0041] Where, is the horizontal axis coordinate in the coordinate system after rotation transformation, is the vertical axis coordinate in the coordinate system after rotation transformation, is the horizontal axis coordinate in the original coordinate system, is the vertical axis coordinate in the original coordinate system, is the rotation transformation angle, 、 as well as are the first, second and third coefficients of the first elliptic general equation, is the inverse tangent function;

[0042] otherwise, taking the first ellipse general equation as the second ellipse general equation;

[0043] Formulating the second ellipse general equation into the first ellipse standard equation;

[0044] Determine the third focus and the fourth focus according to the first standard ellipse equation;

[0045] If the first ellipse general equation has a cross term, performing a coordinate inverse rotation transformation on the third focus and the fourth focus according to the second formula to obtain the first focus and the second focus;

[0046] Otherwise, the third focus and the fourth focus are used as the first focus and the second focus.

[0047] In one possible implementation, performing camera coordinate system conversion on the coordinates of the center point of the capillary tube to obtain camera coordinates includes:

[0048] Get the camera intrinsic parameter matrix and depth information;

[0049] The coordinates of the center point of the capillary tube are transformed according to a third formula, the camera intrinsic parameter matrix, and the depth information to obtain the camera coordinates, wherein the third formula is:

[0050]

[0051] Where, 、 as well as are the x-axis, y-axis, and z-axis coordinates of the capillary center point in the camera coordinate system, is the depth information, is the internal parameter matrix, is the x-axis focal length, is the y-axis focal length, as well as are the x-axis and y-axis coordinates of the intersection of the optical axis and the image plane on the image plane, as well as are the x-axis and y-axis coordinates of the capillary center point in the image coordinate system respectively.

[0052] In one possible implementation, determining the capillary length according to the relative coordinates of the two cameras, the first camera coordinates, and the second camera coordinates includes:

[0053] The capillary length is determined according to a fourth formula, the relative coordinates of the two cameras, the first camera coordinates, and the second camera coordinates, wherein the fourth formula is:

[0054]

[0055] Where, 、 as well as are the x-axis, y-axis, and z-axis components of the first camera coordinates, respectively. 、 as well as are the x-axis, y-axis, and z-axis components of the second camera coordinates, 、 as well as They are the x-axis, y-axis, and z-axis components of the camera's relative coordinates.

[0056] In a second aspect, an embodiment of the present invention provides a capillary length determination device for implementing the capillary length determination method described in the first aspect or any possible implementation of the first aspect, the capillary length determination device comprising:

[0057] a capillary tube image acquisition module, configured to acquire a first capillary tube image and a second capillary tube image, wherein the first capillary tube image and the second capillary tube image are respectively acquired from two fixed cameras;

[0058] a capillary tube edge recognition module, configured to recognize an end edge line of the first capillary tube image and an end edge line of the second capillary tube image according to a shooting position of a corresponding camera, and obtain a first end edge line and a second end edge line;

[0059] a coordinate transformation module, configured to perform image coordinate transformation on the coordinates of the first end edge line in the first capillary image and the coordinates of the second end edge line in the second capillary image, and use the obtained coordinates of the first end edge line in the camera coordinate system and the obtained coordinates of the second end edge line in the camera coordinate system as the first camera coordinates and the second camera coordinates, respectively;

[0060] as well as,

[0061] The capillary length determination module is used to determine the capillary length according to the relative coordinates of the two cameras, the first camera coordinates and the second camera coordinates.

[0062] In a third aspect, an embodiment of the present invention provides an electronic device comprising a memory and a processor, wherein the memory stores a computer program that can be run on the processor, and when the processor executes the computer program, it implements the steps of the method described in the first aspect or any possible implementation of the first aspect.

[0063] In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, it implements the steps of the method described in the first aspect or any possible implementation of the first aspect.

[0064] Compared with the prior art, the embodiments of the present invention have the following beneficial effects:

[0065] Embodiments of the present invention disclose a method for determining capillary length. The method first acquires a first capillary image and a second capillary image, wherein the first and second capillary images are obtained from two fixed cameras, respectively. The method then identifies the end edge lines of the first and second capillary images based on the shooting positions of the corresponding cameras, obtaining first and second end edge lines. The method then performs image coordinate transformation on the coordinates of the first and second end edge lines in the first and second capillary images, using the obtained coordinates of the first and second end edge lines in the camera coordinate system as the first and second camera coordinates, respectively. Finally, the capillary length is determined based on the relative coordinates of the two cameras, the first and second camera coordinates. This method, based on end edge line identification, converts the edge lines from image coordinates to camera coordinates, and then determines the capillary length based on the relative coordinates of the two cameras. Because the method determines the capillary length based on the end edge lines, the end positioning is accurate, ensuring the accuracy and stability of capillary length detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0066] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the embodiments or descriptions of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0067] Figure 1 is a flow chart of a method for determining capillary length provided by an embodiment of the present invention;

[0068] Figure 2 This is a diagram illustrating an application scenario of the method for determining capillary length provided by an embodiment of the present invention;

[0069] Figure 3 This is a functional block diagram of a capillary length determination device provided by an embodiment of the present invention;

[0070] Figure 4 This is a functional block diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0071] In the following description, specific details such as particular system structures and techniques are provided for purposes of illustration, not limitation, to facilitate a thorough understanding of the embodiments of the present invention. However, it will be apparent to those skilled in the art that the present invention may be practiced in alternative embodiments without these specific details. In other instances, detailed descriptions of well-known systems, devices, and methods are omitted so as not to obscure the description of the present invention with unnecessary detail.

[0072] In order to make the objectives, technical solutions and advantages of the present invention more clear, the following will be described through specific implementation methods in conjunction with the accompanying drawings.

[0073] The following is a detailed description of an embodiment of the present invention. This example is implemented based on the technical solution of the present invention, and provides a detailed implementation method and specific operation process, but the protection scope of the present invention is not limited to the following embodiment.

[0074] Figure 1 Flowchart of a method for determining capillary length provided in an embodiment of the present invention.

[0075] like Figure 1 As shown, it shows a flow chart of the implementation method of the capillary length determination method provided by an embodiment of the present invention, which is described in detail as follows:

[0076] In step 101 , a first capillary tube image and a second capillary tube image are acquired, wherein the first capillary tube image and the second capillary tube image are acquired from two fixed cameras, respectively.

[0077] For example, Figure 2 The figure shows an application scenario of the capillary tube length determination method of the present invention. In the figure, two cameras 203 are fixed to one side of a capillary tube 201. Generally speaking, the line connecting the two cameras 203 is parallel to the axis of the capillary tube 201. A hot metal detector 202 is installed on one side of the capillary tube 201. The two cameras 203 are instructed by the hot metal detector 202 to synchronously capture images. The two captured capillary tube images are visually analyzed and combined with the positions of the two cameras 203 to determine the length of the capillary tube 201.

[0078] The method of the present invention uses the edge line of the capillary end to identify the edge line of the capillary end through the capillary image, then determines the center of the capillary end based on the edge line, and finally determines the capillary length based on the centers of the two ends. Compared with the existing visual recognition method, this method has a small error and a more stable result because it measures the length based on the center point.

[0079] The implementation of the present invention discusses the implementation process of the above principles in detail from the following aspects.

[0080] In step 102 , the end edge line of the first capillary tube image and the end edge line of the second capillary tube image are identified according to the shooting position of the corresponding camera to obtain a first end edge line and a second end edge line.

[0081] In some embodiments, identifying the end edge line of the first capillary tube image and the end edge line of the second capillary tube image according to the shooting position of the corresponding camera to obtain the first end edge line and the second end edge line includes:

[0082] For any one of the first capillary tube image and the second capillary tube image, perform the following steps respectively:

[0083] Perform Gaussian filtering on the capillary image to obtain a smooth capillary image;

[0084] Performing a gradient operation on the smooth capillary image using an operator to obtain a gradient capillary image;

[0085] performing non-maximum suppression on the gradient capillary image to obtain a first edge capillary image;

[0086] performing dual-threshold detection on the first edge capillary tube image to obtain a second edge capillary tube image;

[0087] The second edge capillary image is fed into a deep learning model to identify the image block where the end edge line is located.

[0088] In some embodiments, the deep learning model is trained based on a plurality of sample images, wherein the process of constructing sample labels includes:

[0089] For each sample image, perform the following steps:

[0090] Acquire a target edge point, wherein the target edge point is located on an edge line at an end of the sample image;

[0091] Taking the target edge point as the center, obtaining multiple first edge points within a preset neighborhood radius;

[0092] According to the first formula and the plurality of first edge points, an edge line equation expressing the end edge line is constructed using the least squares method, wherein the first formula is:

[0093]

[0094] Where, 、 、 、 、 as well as are the first coefficient, the second coefficient, the third coefficient, the fourth coefficient, the fifth coefficient and the sixth coefficient respectively, is the x-axis coordinate of the first edge point in the sample image, is the y-axis coordinate of the first edge point in the sample image;

[0095] Substituting the plurality of first edge points into the edge line equation to obtain a plurality of deviation values;

[0096] taking the maximum value among the plurality of deviation values ​​as a reference value;

[0097] Taking the target edge point as the center, and according to the edge line equation, obtaining a plurality of second edge points, wherein a value obtained by substituting the second edge point into the edge line equation is not greater than the reference value, and the plurality of second edge points are continuous;

[0098] Using a minimum rectangle surrounding the plurality of second edge points as a marking rectangle;

[0099] The center coordinates of the marking rectangle, the width of the marking rectangle, and the length of the marking rectangle are used as sample labels.

[0100] Exemplarily, the present invention performs edge processing on the capillary image to obtain an edge image, and then inputs the edge image into a deep learning model to perform end edge line recognition.

[0101] In terms of edge processing, the present invention first applies a Gaussian filter (Gaussian blur) to the capillary image to remove noise. Then, an operator is used to perform a gradient operation on the image. For example, the Sobel operator is used to calculate the horizontal and vertical gradients of the image to detect edges.

[0102] Specifically, two Sobel operators (horizontal and vertical) are used to perform convolution operations with the image respectively, one for detecting horizontal edges and the other for detecting vertical edges, and then the gradient magnitude and gradient direction are determined according to the results of the two convolutions.

[0103] In edge detection, possible edge points are initially identified by calculating the image's gradient magnitude and direction. However, these initially detected edge points may have a certain width, and at a true edge, the gradient magnitude should be locally maximum. The purpose of non-maximum suppression is to retain only those possible edge points with the local maximum gradient magnitude and suppress other non-maximum points, thereby refining the edge to a single-pixel-wide line and improving edge location accuracy.

[0104] Therefore, after obtaining the gradient magnitude and gradient direction, non-maximum analysis is required. First, the gradient direction is discretized into several specific angles, usually 4 or 8 directions, such as 0°, 45°, 90°, 135°, etc. This facilitates subsequent comparison and judgment.

[0105] Then traverse the pixels: starting from the upper left corner of the image, traverse each pixel row by row and column by column.

[0106] Next, compare the local gradient magnitudes: For the current pixel, compare its gradient magnitude with the gradient magnitudes of its adjacent pixels in the same direction, based on its gradient direction. For example, if the gradient direction of the current pixel is 0°, compare its gradient magnitude with the horizontally adjacent pixels to the left and right; if the gradient direction is 45°, compare its gradient magnitude with the diagonally adjacent pixels in the same direction.

[0107] Then suppress non-maximum points: if the gradient amplitude of the current pixel is not the local maximum (that is, smaller than the gradient amplitude of the adjacent pixel in its gradient direction), set the grayscale value of the pixel to 0, that is, suppress the point; if the gradient amplitude of the current pixel is the local maximum, keep the grayscale value of the point unchanged.

[0108] Finally, the refined edge image is obtained: after processing all pixels, only the points with the local maximum gradient amplitude are retained in the obtained image, and these points constitute the refined edge image.

[0109] After the above steps, the image edge has taken shape. However, there are still breakpoints and noise points in the image. Therefore, dual-threshold detection is required. The edge points in the image can be divided into strong edge points and weak edge points based on the size of their gradient amplitude. Dual-threshold detection is to set a high threshold and a low threshold. Points with gradient amplitudes greater than the high threshold are determined as strong edge points, points with gradient amplitudes less than the low threshold are directly excluded, and points between the low and high thresholds are marked as weak edge points. Then, through further processing of weak edge points, the final determination of whether they are true edge points is made based on their connectivity with strong edge points.

[0110] A dual threshold detection process includes:

[0111] First, set the thresholds: Based on the image characteristics and application requirements, select appropriate upper and lower thresholds (Th and Tl). Typically, the upper threshold is 2-3 times the lower threshold. Next, classify edge points: Iterate through each pixel in the image and classify them based on the relationship between their gradient magnitude and the threshold. Pixels with gradient magnitudes greater than Th are marked as strong edge points and colored white (or another specific color); pixels with gradient magnitudes less than Tl are marked as non-edge points and colored black; and pixels with gradient magnitudes between Tl and Th are marked as weak edge points and colored gray. Finally, connect weak edge points: Iterate through each pixel marked as a weak edge point and check whether there is a strong edge point in their 8-neighborhood (i.e., the eight surrounding pixels). If a weak edge point has a strong edge point within its 8-neighborhood, it is retained as an edge point. If a weak edge point has no strong edge points within its 8-neighborhood, it is deleted and not considered an edge point. In this way, weak edge points connected to strong edge points are retained to form complete edge lines, while isolated weak edge points are removed.

[0112] After the above steps, the edge of the image is extracted to form an edge capillary image.

[0113] In the identification of the end edges of capillaries, the present invention applies deep learning methods to extract the end edges. Deep learning recognition methods include Faster R-CNN and YOLO. These deep learning methods require sample images for training to extract the end edges of capillaries. These training sample images are provided with labels to identify the end edges of capillaries. For example, one label is in the form of the center point of the end edge, the length and width of the image block where it is located.

[0114] To more accurately identify the end edge of a capillary tube, an embodiment of the present invention constructs a label for a sample image by specifying an end edge point. A plurality of edge pixels are searched with a preset pixel radius, with this end edge point as the center. For example, 18 end edge pixels are searched with a radius of 10 pixels. The searched end edge pixels are then fitted with a first formula using the least squares method to determine the values ​​of the six coefficients in the first formula. The first formula is:

[0115]

[0116] Where, 、 、 、 、 as well as are the first coefficient, the second coefficient, the third coefficient, the fourth coefficient, the fifth coefficient and the sixth coefficient respectively, is the x-axis coordinate of the first edge point in the sample image, is the y-axis coordinate of the first edge point in the sample image.

[0117] After the construction is completed, these searched edge pixels are substituted into the left side of the first line of the above formula respectively. The value on the right is the deviation value. The maximum value of the deviation value is used as the reference value. Then, with the specified end edge point as the center, the edge pixel is found and substituted into the left side of the first line of the above formula. If the obtained value is less than the reference value, the pixel is retained and the search continues. Otherwise, the search process is terminated. It should be noted that during the search process, it is necessary to ensure that the found pixel is connected to the end edge pixel that has been found, so as to form a continuous edge line.

[0118] After the search is completed, the minimum rectangle surrounding the found pixels is used as the identification rectangle, the center of the identification rectangle is used as the label center coordinates, and the length and width of the identification rectangle are used as the length and width of the label.

[0119] In this way, the deep learning model constructed based on the sample image can find the edge line that conforms to the first formula, thus building an image foundation for length detection.

[0120] In step 103, image coordinate transformation is performed on the coordinates of the first end edge line in the first capillary image and the coordinates of the second end edge line in the second capillary image, and the obtained coordinates of the first end edge line in the camera coordinate system and the obtained coordinates of the second end edge line in the camera coordinate system are used as the first camera coordinates and the second camera coordinates, respectively.

[0121] In some embodiments, performing image coordinate transformation on the coordinates of the first end edge line in the first capillary image and the coordinates of the second end edge line in the second capillary image, and using the obtained coordinates of the first end edge line in the camera coordinate system and the obtained coordinates of the second end edge line in the camera coordinate system as the first camera coordinates and the second camera coordinates, respectively, includes:

[0122] For each end edge line, perform the following steps:

[0123] The end edge line is fitted using the least square method to obtain the first ellipse general equation describing the end edge line;

[0124] Formulating the first ellipse general equation, and determining a first focus and a second focus according to a result of the formulation;

[0125] The midpoint between the first focus and the second focus is used as the center point of the capillary;

[0126] The coordinates of the center point of the capillary tube are converted into a camera coordinate system to obtain camera coordinates.

[0127] In some embodiments, formulating the first ellipse general equation and determining the first focus and the second focus according to the formulation result includes:

[0128] If the first elliptic general equation has a cross term, a second elliptic general equation is obtained by performing a rotation transformation on the first elliptic general equation according to a second formula, wherein the second formula is:

[0129]

[0130] Where, is the horizontal axis coordinate in the coordinate system after rotation transformation, is the vertical axis coordinate in the coordinate system after rotation transformation, is the horizontal axis coordinate in the original coordinate system, is the vertical axis coordinate in the original coordinate system, is the rotation transformation angle, 、 as well as are the first, second and third coefficients of the first elliptic general equation, is the inverse tangent function;

[0131] otherwise, taking the first ellipse general equation as the second ellipse general equation;

[0132] Formulating the second ellipse general equation into the first ellipse standard equation;

[0133] Determine the third focus and the fourth focus according to the first standard ellipse equation;

[0134] If the first ellipse general equation has a cross term, performing a coordinate inverse rotation transformation on the third focus and the fourth focus according to the second formula to obtain the first focus and the second focus;

[0135] Otherwise, the third focus and the fourth focus are used as the first focus and the second focus.

[0136] In some embodiments, performing a camera coordinate system conversion on the coordinates of the center point of the capillary tube to obtain the camera coordinates includes:

[0137] Get the camera intrinsic parameter matrix and depth information;

[0138] The coordinates of the center point of the capillary tube are transformed according to a third formula, the camera intrinsic parameter matrix, and the depth information to obtain the camera coordinates, wherein the third formula is:

[0139]

[0140] Where, 、 as well as are the x-axis, y-axis, and z-axis coordinates of the capillary center point in the camera coordinate system, is the depth information, is the internal parameter matrix, is the x-axis focal length, is the y-axis focal length, as well as are the x-axis and y-axis coordinates of the intersection of the optical axis and the image plane on the image plane, as well as are the x-axis and y-axis coordinates of the capillary center point in the image coordinate system respectively.

[0141] For example, in detecting the length of the capillary tube, the present invention first finds the center point of the end according to the end edge line of the image, then converts the center points of the two ends into coordinates in the corresponding camera coordinate system, and finally determines the capillary tube length based on the coordinates of the two center points of the end in the camera coordinate system and the relative coordinates of the two cameras.

[0142] The end edge line of the capillary tube is an ellipse in the image. The coordinates of the pixel points of the end edge line obtained in the previous step can be fitted to the general ellipse equation using the least squares method. In order to distinguish it from the ellipse equation in the first equation, the general ellipse equation used in this step is:

[0143]

[0144] Where, 、 、 、 、 as well as They are the seventh coefficient, the eighth coefficient, the ninth coefficient, the tenth coefficient, the eleventh coefficient and the twelfth coefficient, is the horizontal coordinate of the first edge point in the sample image, is the vertical coordinate of the first edge point in the sample image.

[0145] After fitting, the ellipse equations corresponding to the two ends are obtained. According to these two ellipse equations, the image coordinates of the centers of the edges of the two ends of the capillary can be found. Specifically, if there is a cross term ( ) requires a coordinate system rotation transformation, and the transformation method of the present invention is performed according to the following equation:

[0146]

[0147] Where, is the horizontal axis coordinate in the coordinate system after rotation transformation, is the vertical axis coordinate in the coordinate system after rotation transformation, is the horizontal axis coordinate in the original coordinate system, is the vertical axis coordinate in the original coordinate system, is the rotation transformation angle, 、 as well as are the first, second and third coefficients of the first elliptic general equation, is the inverse tangent function.

[0148] The transformed equation is used as the equation to be analyzed. If there is no cross term, the original equation is used as the equation to be analyzed.

[0149] Then, the equation to be analyzed is formulated. Taking the equation to be analyzed after the transformation as an example, the formula is:

[0150]

[0151] According to this equation, the major and minor axes of the ellipse are obtained ( , otherwise, in the following formula 、 exchange):

[0152]

[0153]

[0154] In the above formula, 、 are the major axis and minor axis (or minor axis and major axis), when The two focal points are , ;when The two focal points are , .

[0155] The two foci obtained in the above process are then inversely transformed through the transformation equation to obtain the coordinates of the two foci in the image coordinate system.

[0156] The median of the two foci is the coordinate of the center of the capillary end.

[0157] After the above steps, the coordinates of the center points of the two ends in the image coordinate system are obtained. These two coordinates are then transformed from the image coordinate system to the camera coordinate system using the camera intrinsic parameter matrix and depth information. The embodiment of the present invention uses the third formula:

[0158]

[0159] Where, 、 as well as are the x-axis, y-axis, and z-axis coordinates of the capillary center point in the camera coordinate system, is the depth information, is the internal parameter matrix, is the x-axis focal length, is the y-axis focal length, as well as are the x-axis and y-axis coordinates of the intersection of the optical axis and the image plane on the image plane, as well as are the x-axis and y-axis coordinates of the capillary center point in the image coordinate system respectively.

[0160] The above intrinsic parameter matrix is ​​obtained through camera calibration, for example, Zhang Zhengyou calibration method:

[0161] This method uses a planar template, such as a checkerboard, and captures images of the template in different poses. Based on the pixel coordinates of the checkerboard's corners in the image and their coordinates in the world coordinate system, a system of equations is established to solve the camera's intrinsic parameter matrix. Specifically, a corner detection algorithm is used to find the coordinates of the checkerboard's corners in the image. Given the coordinates of the checkerboard's corners in the world coordinate system (usually assuming the plane of the checkerboard is the xy plane of the world coordinate system, (z = 0)), the relationship between these corresponding points is used to construct a system of linear equations to solve the camera's intrinsic parameter matrix.

[0162] The process begins with preparing a checkerboard template and placing it at various positions and angles. A camera then captures multiple images of the checkerboard. Corner detection is then performed on each image, extracting the pixel coordinates of the checkerboard corners. Finally, the camera's intrinsic parameter matrix is ​​calculated using Zhang Zhengyou's calibration algorithm, using these corner coordinates and the known world coordinates of the checkerboard.

[0163] The camera's depth information refers to the distance between the camera and the object in the scene corresponding to the pixel in the image. Because the camera in this embodiment of the present invention is fixed in a fixed position, the location of the capillary tube is relatively fixed. Furthermore, some scenes may be equipped with a hot metal detector that can detect the coordinates of the capillary tube's predetermined location. Therefore, depth information can be obtained relatively easily.

[0164] In step 104 , the capillary length is determined according to the relative coordinates of the two cameras, the first camera coordinates, and the second camera coordinates.

[0165] In some embodiments, determining the capillary length according to the relative coordinates of the two cameras, the first camera coordinates, and the second camera coordinates includes:

[0166] The capillary length is determined according to a fourth formula, the relative coordinates of the two cameras, the first camera coordinates, and the second camera coordinates, wherein the fourth formula is:

[0167]

[0168] Where, 、 as well as are the x-axis, y-axis, and z-axis components of the first camera coordinates, respectively. 、 as well as are the x-axis, y-axis, and z-axis components of the second camera coordinates, 、 as well as They are the x-axis, y-axis, and z-axis components of the camera's relative coordinates.

[0169] For example, after obtaining the coordinates of the center point of the capillary end in camera coordinates, the capillary length can be calculated by applying the fourth formula in combination with the relative coordinates of the two cameras:

[0170]

[0171] Where, 、 as well as are the x-axis, y-axis, and z-axis components of the first camera coordinates, respectively. 、 as well as are the x-axis, y-axis, and z-axis components of the second camera coordinates, 、 as well as They are the x-axis, y-axis, and z-axis components of the camera's relative coordinates.

[0172] In an embodiment of a method for determining capillary length according to the present invention, a first capillary image and a second capillary image are first acquired, wherein the first capillary image and the second capillary image are obtained from two fixed cameras, respectively. The end edge lines of the first capillary image and the second capillary image are then identified based on the shooting positions of the corresponding cameras to obtain a first end edge line and a second end edge line. The coordinates of the first end edge line in the first capillary image and the coordinates of the second end edge line in the second capillary image are then transformed into image coordinates, and the obtained coordinates of the first end edge line in the camera coordinate system and the second end edge line in the camera coordinate system are used as the first camera coordinates and the second camera coordinates, respectively. Finally, the capillary length is determined based on the relative coordinates of the two cameras, the first camera coordinates, and the second camera coordinates. This embodiment of the present invention determines the capillary length based on end edge line identification by converting the edge line from image coordinates to camera coordinates and then combining the relative coordinates of the two cameras. Because the method of the present invention determines the capillary length based on the end edge line, the end position is accurately positioned, ensuring the accuracy and stability of capillary length detection.

[0173] It should be understood that the size of the serial numbers of each step in the above embodiment does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of the present invention.

[0174] The following is an embodiment of the device of the present invention. For details not described in detail, please refer to the corresponding method embodiment described above.

[0175] Figure 3 This is a functional block diagram of a capillary length determination device provided by an embodiment of the present invention, referring to Figure 3 The capillary length determination device includes: a capillary image acquisition module 301, a capillary edge recognition module 302, a coordinate transformation module 303 and a capillary length determination module 304, wherein:

[0176] The capillary tube image acquisition module 301 is configured to acquire a first capillary tube image and a second capillary tube image, wherein the first capillary tube image and the second capillary tube image are acquired from two fixed cameras respectively;

[0177] The capillary tube edge recognition module 302 is configured to recognize the end edge line of the first capillary tube image and the end edge line of the second capillary tube image according to the shooting position of the corresponding camera, and obtain the first end edge line and the second end edge line;

[0178] a coordinate transformation module 303 for performing image coordinate transformation on the coordinates of the first end edge line in the first capillary image and the coordinates of the second end edge line in the second capillary image, and using the obtained coordinates of the first end edge line in the camera coordinate system and the obtained coordinates of the second end edge line in the camera coordinate system as first camera coordinates and second camera coordinates, respectively;

[0179] The capillary length determining module 304 is configured to determine the capillary length according to the relative coordinates of the two cameras, the first camera coordinates, and the second camera coordinates.

[0180] Figure 4 : is a functional block diagram of an electronic device provided by an embodiment of the present invention. Figure 4 As shown, the electronic device 4 of this embodiment includes: a processor 400 and a memory 401, wherein the memory 401 stores a computer program 402 that can be run on the processor 400. When the processor 400 executes the computer program 402, the steps in the above-mentioned capillary length determination method and embodiment are implemented, for example Figure 1 Steps 101 to 104 are shown.

[0181] Illustratively, the computer program 402 may be divided into one or more modules / units, and the one or more modules / units are stored in the memory 401 and executed by the processor 400 to implement the present invention.

[0182] The electronic device 4 may be a computing device such as a desktop computer, a notebook, a PDA, or a cloud server. The electronic device 4 may include, but is not limited to, a processor 400 and a memory 401. Those skilled in the art will understand that Figure 4 It is only an example of the electronic device 4 and does not constitute a limitation of the electronic device 4. It may include more or fewer components than shown in the figure, or a combination of certain components, or different components. For example, the electronic device 4 may also include input and output devices, network access devices, buses, etc.

[0183] The processor 400 may be a central processing unit (CPU), other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor.

[0184] The memory 401 may be an internal storage unit of the electronic device 4, such as a hard drive or memory of the electronic device 4. The memory 401 may also be an external storage device of the electronic device 4, such as a plug-in hard drive, a Smart Media Card (SMC), a Secure Digital (SD) card, a flash memory card, etc. equipped on the electronic device 4. Furthermore, the memory 401 may include both an internal storage unit of the electronic device 4 and an external storage device. The memory 401 is used to store the computer program 402 and other programs and data required by the electronic device 4. The memory 401 may also be used to temporarily store data that has been output or is about to be output.

[0185] Those skilled in the art can clearly understand that, for the convenience and brevity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In actual applications, the above-mentioned functions can be distributed and completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the implementation method can be integrated into one processing unit, or each unit can exist physically alone, or two or more units can be integrated into one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method implementation method, and will not be repeated here.

[0186] In the above embodiments, the description of each embodiment has its own focus. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0187] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present invention.

[0188] In the embodiments provided by the present invention, it should be understood that the disclosed devices / electronic devices and methods can be implemented in other ways. For example, the device / electronic device embodiments described above are merely illustrative. For example, the division of the modules or units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0189] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of this embodiment.

[0190] In addition, the functional units in various embodiments of the present invention may be integrated into a single processing unit, each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0191] If the integrated module / unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present invention can also implement all or part of the processes in the above-mentioned method embodiments by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium. When executed by a processor, the computer program can implement the steps of the above-mentioned method and device embodiments. The computer program includes computer program code, which can be in source code form, object code form, executable file, or some intermediate form. The computer-readable medium can include: any entity or device capable of carrying the computer program code, recording medium, USB flash drive, mobile hard drive, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signal, telecommunication signal, and software distribution medium.

[0192] The above-described embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention, and should all be included in the scope of protection of the present invention.

Claims

1. A method for determining capillary length, characterized in that: include: Acquire a first capillary tube image and a second capillary tube image, wherein the first capillary tube image and the second capillary tube image are respectively acquired from two fixed cameras; Identifying the end edge line of the first capillary tube image and the end edge line of the second capillary tube image according to the shooting position of the corresponding camera to obtain a first end edge line and a second end edge line; performing image coordinate transformation on the coordinates of the first end edge line in the first capillary image and the coordinates of the second end edge line in the second capillary image, and using the obtained coordinates of the first end edge line in the camera coordinate system and the obtained coordinates of the second end edge line in the camera coordinate system as first camera coordinates and second camera coordinates, respectively; determining the capillary length according to the relative coordinates of the two cameras, the first camera coordinates, and the second camera coordinates; The performing image coordinate transformation on the coordinates of the first end edge line in the first capillary image and the coordinates of the second end edge line in the second capillary image, and using the obtained coordinates of the first end edge line in the camera coordinate system and the obtained coordinates of the second end edge line in the camera coordinate system as the first camera coordinates and the second camera coordinates, respectively, includes: For each end edge line, perform the following steps: The end edge line is fitted using the least square method to obtain the first ellipse general equation describing the end edge line; Formulating the first ellipse general equation, and determining a first focus and a second focus according to a result of the formulation; The midpoint between the first focus and the second focus is used as the center point of the capillary; Performing camera coordinate system conversion on the coordinates of the center point of the capillary tube to obtain camera coordinates; The step of formulating the first ellipse general equation and determining the first focus and the second focus according to the result of the formulation includes: If the first elliptic general equation has a cross term, a second elliptic general equation is obtained by performing a rotation transformation on the first elliptic general equation according to a second formula, wherein the second formula is: Where, is the horizontal axis coordinate in the coordinate system after rotation transformation, is the vertical axis coordinate in the coordinate system after rotation transformation, is the horizontal axis coordinate in the original coordinate system, is the vertical axis coordinate in the original coordinate system, is the rotation transformation angle, 、 as well as are the first, second and third coefficients of the first elliptic general equation, is the inverse tangent function; otherwise, taking the first ellipse general equation as the second ellipse general equation; Formulating the second ellipse general equation into the first ellipse standard equation; Determine the third focus and the fourth focus according to the first standard ellipse equation; If the first ellipse general equation has a cross term, performing a coordinate inverse rotation transformation on the third focus and the fourth focus according to the second formula to obtain the first focus and the second focus; Otherwise, the third focus and the fourth focus are used as the first focus and the second focus.

2. The method for determining capillary length according to claim 1, wherein: The step of identifying the end edge line of the first capillary tube image and the end edge line of the second capillary tube image according to the shooting position of the corresponding camera to obtain the first end edge line and the second end edge line includes: For any one of the first capillary tube image and the second capillary tube image, perform the following steps respectively: Perform Gaussian filtering on the capillary image to obtain a smooth capillary image; performing a gradient operation on the smooth capillary image using an operator to obtain a gradient capillary image; performing non-maximum suppression on the gradient capillary image to obtain a first edge capillary image; performing dual-threshold detection on the first edge capillary tube image to obtain a second edge capillary tube image; The second edge capillary image is fed into a deep learning model to identify the image block where the end edge line is located.

3. The method for determining capillary length according to claim 2, wherein: The deep learning model is obtained by training multiple sample images, wherein the sample label construction process includes: For each sample image, perform the following steps: Acquire a target edge point, wherein the target edge point is located on an edge line at an end of the sample image; Taking the target edge point as the center, obtaining multiple first edge points within a preset neighborhood radius; According to the first formula and the plurality of first edge points, an edge line equation expressing the end edge line is constructed using the least squares method, wherein the first formula is: Where, 、 、 、 、 as well as are the first coefficient, the second coefficient, the third coefficient, the fourth coefficient, the fifth coefficient and the sixth coefficient respectively, is the x-axis coordinate of the first edge point in the sample image, is the y-axis coordinate of the first edge point in the sample image; Substituting the plurality of first edge points into the edge line equation to obtain a plurality of deviation values; taking the maximum value among the plurality of deviation values ​​as a reference value; Taking the target edge point as the center, and according to the edge line equation, obtaining a plurality of second edge points, wherein a value obtained by substituting the second edge point into the edge line equation is not greater than the reference value, and the plurality of second edge points are continuous; Using a minimum rectangle surrounding the plurality of second edge points as a marking rectangle; The center coordinates of the marking rectangle, the width of the marking rectangle, and the length of the marking rectangle are used as sample labels.

4. The method for determining capillary length according to claim 1, wherein: The step of converting the coordinates of the center point of the capillary tube into a camera coordinate system to obtain the camera coordinates includes: Get the camera intrinsic parameter matrix and depth information; The coordinates of the center point of the capillary tube are transformed according to a third formula, the camera intrinsic parameter matrix, and the depth information to obtain the camera coordinates, wherein the third formula is: Where, 、 as well as are the x-axis, y-axis, and z-axis coordinates of the capillary center point in the camera coordinate system, is the depth information, is the internal parameter matrix, is the x-axis focal length, is the y-axis focal length, as well as are the x-axis and y-axis coordinates of the intersection of the optical axis and the image plane on the image plane, as well as are the x-axis and y-axis coordinates of the capillary center point in the image coordinate system respectively.

5. The method for determining capillary length according to any one of claims 1 to 4, characterized in that: The determining of the capillary length according to the relative coordinates of the two cameras, the first camera coordinates, and the second camera coordinates includes: The capillary length is determined according to a fourth formula, the relative coordinates of the two cameras, the first camera coordinates, and the second camera coordinates, wherein the fourth formula is: Where, 、 as well as are the x-axis, y-axis, and z-axis components of the first camera coordinates, 、 as well as are the x-axis, y-axis, and z-axis components of the second camera coordinates, 、 as well as They are the x-axis, y-axis, and z-axis components of the camera's relative coordinates.

6. A capillary length determination device, characterized in that: For implementing the capillary length determination method according to any one of claims 1 to 5, the capillary length determination device comprises: a capillary tube image acquisition module, configured to acquire a first capillary tube image and a second capillary tube image, wherein the first capillary tube image and the second capillary tube image are respectively acquired from two fixed cameras; a capillary tube edge recognition module, configured to recognize an end edge line of the first capillary tube image and an end edge line of the second capillary tube image according to a shooting position of a corresponding camera, and obtain a first end edge line and a second end edge line; a coordinate transformation module, configured to perform image coordinate transformation on the coordinates of the first end edge line in the first capillary image and the coordinates of the second end edge line in the second capillary image, and use the obtained coordinates of the first end edge line in the camera coordinate system and the obtained coordinates of the second end edge line in the camera coordinate system as the first camera coordinates and the second camera coordinates, respectively; as well as, The capillary length determination module is used to determine the capillary length according to the relative coordinates of the two cameras, the first camera coordinates and the second camera coordinates.

7. An electronic device comprising a memory and a processor, wherein the memory stores a computer program that can be run on the processor, characterized in that: When the processor executes the computer program, the capillary length determination method according to any one of claims 1 to 5 is implemented.

8. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the capillary length determination method according to any one of claims 1 to 5 is implemented.

Images