Target positioning and recognition method for high-precision photogrammetry of jacket

Abstract

The invention discloses a target positioning and recognition method for high-precision photogrammetry of a jacket. The target positioning and recognition method for high-precision photogrammetry of the jacket comprises the following steps: obtaining a plurality of jacket images through photogrammetry; drawing a histogram of the image; determining imaging conditions; using a real-time target detection algorithm to coarsely detect a possible circular target area; ellipse detection under different imaging conditions; positioning a circle center; and decoding the coding mark. According to the target positioning and recognition method for high-precision photogrammetry of the jacket, high-precision measurement work of the deepwater ultra-large jacket can be effectively completed through the target positioning and recognition method, and the measurement precision of completion measurement of the deepwater ultra-large jacket is effectively improved.

Description

Technical Field

[0001] The invention belongs to the field of target detection in offshore oil engineering, and in particular relates to a target positioning and identification method for high-precision photogrammetry of a jacket. Background Art

[0002] For the completion measurement of a super-large deepwater jacket, the targets are evenly pasted on the jacket surface at a certain interval before the photogrammetry is carried out. After the jacket is completed, it is usually placed on the dock. The jacket with the targets pasted is photographed with a camera to obtain the jacket image. The target positioning and identification in the image can achieve high-precision measurement of the key points of the jacket.

[0003] However, in the process of photogrammetry, different humidity and temperature in the air will produce different atmospheric refractions. In addition, factors such as the light and wind strength at the seaside will have a negative impact on the image quality and measurement taken by the camera.

[0004] Therefore, it is urgent to design a target positioning and identification method for high-precision photogrammetry of the catheter rack to solve the problem of high-precision measurement of the catheter rack mentioned above. Summary of the invention

[0005] In order to solve the technical problems mentioned in the background technology that in the process of photogrammetry, different humidity and temperature in the air will produce different atmospheric refractions, and factors such as light and wind strength at the seaside will have a negative impact on the image quality and measurement taken by the camera, a target positioning and identification method for high-precision photogrammetry of a conductor frame is provided to solve the problem of high-precision measurement of the conductor frame.

[0006] To achieve the above-mentioned purpose, the specific technical scheme of the target positioning and identification method of the catheter frame high-precision photogrammetry of the present invention is as follows: A target positioning and identification method for high-precision photogrammetry of a catheter rack comprises the following steps: S1, obtaining a plurality of images of the catheter rack by photogrammetry; S2, draw a histogram of the image; S3, determining imaging conditions; S4, using a real-time target detection algorithm to roughly detect possible circular target areas; S5, ellipse detection under different imaging conditions; S6, locate the center of the circle; S7. Decode the encoding flag.

[0007] Furthermore, in S1, a plurality of images of the catheter rack are taken from different angles and directions using a drone and cut into slices.

[0008] Furthermore, in S2, based on the images captured by the drone, a histogram of each image is calculated and plotted, where the horizontal axis represents the brightness value of 0-255, and the vertical axis represents the number of pixels of the corresponding brightness in the photo.

[0009] Further, in S3, the imaging conditions of the image are divided into normal imaging conditions and abnormal imaging conditions based on the histogram of the image; normal imaging conditions refer to good lighting conditions and good overall contrast of the image, and abnormal imaging conditions refer to situations where the lighting and wind strength at the seaside will affect the imaging.

[0010] Furthermore, in S4, the catheter rack target data sets in the above two scenarios are respectively labeled and constructed, and the two data sets are expanded using image enhancement technology; the data sets are divided, and the corresponding training sets are sent to the real-time target detection algorithm for training, and the weight of the model's last training and the weight when the training effect reaches the best are saved.

[0011] Furthermore, the test image is sent to the trained model for target mark detection to obtain its confidence score and the location information of the corresponding target box; the area detected as the target is cropped from the image and its location information is recorded.

[0012] Further, in S5, the boundary portion of the ellipse can be fitted based on the least squares principle for the photos taken under normal imaging conditions.

[0013] Furthermore, in S5, for photos taken under abnormal imaging conditions, relying on the flow conservation property, the amount of gradient entering through the arc of the outermost ellipse must be equal to the amount of gradient flowing out through the arc of the innermost ellipse, thereby achieving ellipse detection.

[0014] Furthermore, in S6, based on the principle that the intersection of two straight lines after the projective transformation is still the intersection of the two straight lines after the transformation, the center of the landmark point can be obtained, and the outermost circular arc segment is used to locate the center of the circle.

[0015] Further, in S7, the region of interest where the coding mark point is located is selected, the annular bands of the region of interest are extracted, the numbers represented by each annular band are calculated, the relative position relationship between the annular bands is calculated, the binary code value of the coding mark point is calculated, and converted into a decimal code number.

[0016] The target positioning and identification method of the catheter frame high-precision photogrammetry of the present invention has the following advantages: Through the target positioning and identification method, the high-precision measurement of deep-water super-large jackets can be effectively completed, and the measurement accuracy of the completion measurement of deep-water super-large jackets can be effectively improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1It is an overall flow chart of the target positioning and identification method of the high-precision photogrammetry of the jacket of the present invention; Figure 2 This is a target example diagram of the target positioning and identification method for high-precision photogrammetry of a jacket according to the present invention; Figure 3 A schematic diagram of a circular reference coding mark with a size of 50 mm*50 mm for the target positioning and identification method of the high-precision photogrammetry of the catheter frame of the present invention; Figure 4 A schematic diagram of a target of 50 mm*50 mm set at the center of the catheter in the target positioning and identification method of the catheter rack high-precision photogrammetry of the present invention; Figure 5 A schematic diagram of a circular reference coding mark with a size of 200mm*200mm for the target positioning and identification method for high-precision photogrammetry of a conductor frame of the present invention; Figure 6 It is a schematic diagram of ellipse transformation of the target positioning and identification method of the high-precision photogrammetry of the jacket of the present invention; Figure 7 This is an example diagram of the overlapping area of ​​circular targets in the target positioning and identification method for high-precision photogrammetry of a conductor frame of the present invention.

[0018] Description of the markings in the figure: 1. 50mm*50mm coding mark; 2. 200mm*200mm coding mark; 3. Steel bar; 4. Catheter ring. DETAILED DESCRIPTION

[0019] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0020] Those skilled in the art will appreciate that, although some embodiments herein include certain features included in other embodiments but not other features, the combination of features of different embodiments is meant to be within the scope of the present invention and form different embodiments. For example, in the claims, any one of the claimed embodiments may be used in any combination.

[0021] Please refer to the attached Figure 1 To Attachment Figure 7 The invention describes a target positioning and identification method for high-precision photogrammetry of a catheter frame.

[0022] like Figure 1As shown, the target positioning and identification method of the high-precision photogrammetry of the jacket in the present invention comprises the following steps: S1. Obtain several images of the catheter rack through photogrammetry; S2, draw a histogram of the image; S3, determining imaging conditions; S4, using a real-time target detection algorithm to roughly detect possible circular target areas; S5, ellipse detection under different imaging conditions; S6, locate the center of the circle; S7. Decode the encoding flag.

[0023] Through the target positioning and identification method, the high-precision measurement of deep-water super-large jackets can be effectively completed, and the measurement accuracy of the completion measurement of deep-water super-large jackets can be effectively improved.

[0024] Further, if Figures 1 to 5 As shown, in S1, a number of images of the catheter rack are taken from different angles and directions using a drone and cut into slices; in S2, a histogram of each image is calculated and drawn based on the images taken by the drone, with the horizontal axis representing the brightness value of 0-255 and the vertical axis representing the number of pixels of the corresponding brightness in the photo; in S3, the imaging conditions of the image are divided into normal imaging conditions and abnormal imaging conditions based on the histogram of the image; normal imaging conditions refer to good lighting conditions and good overall contrast of the image, and abnormal imaging conditions refer to the situation where the lighting and wind strength at the seaside will affect the imaging.

[0025] In this embodiment, S1 acquires several images of the catheter rack, including: designing two diameter coding marks on the target, including a coding mark 1 example with a size of 50mm*50mm, which is mainly used for identification at a position with spatial restrictions and is attached to the center of the catheter.

[0026] Preferably, two steel bars 3 with a width greater than 5 cm are provided above the catheter, the two steel bars are the diameter of the closed circle, and both pass through the center of the catheter ring 4; it also includes an example of a coding mark 2 with a size of 200mm*200mm, which can be attached to any position of the object to be measured for photographic measurement.

[0027] This design not only takes into account the space requirements of different scenarios, but also ensures the effectiveness of each position of the object to be tested. From different angles and directions, a drone was used to take several images of the catheter rack and cut them into slices with a size of 768*768 pixels.

[0028] S2 includes calculating and drawing the histogram of each image based on the images taken by the drone. The horizontal axis represents the brightness value of 0-255, and the vertical axis represents the number of pixels with corresponding brightness in the photo.

[0029] S3 judges the imaging conditions of the image according to the histogram of the image, which is divided into two cases: the first case is that the shape of the image histogram is high in the middle and low at both ends, indicating that the lighting conditions are good and the overall contrast of the image is good. The histogram of the photos of the catheter rack taken by the camera in a normal scene should show this distribution; the second case is that the histogram distribution is concentrated at a certain grayscale value, with a very high peak somewhere and a very low peak or even 0 elsewhere. For example, when most of the data in the histogram is concentrated near the right edge, that is, the grayscale of the image is concentrated in the high brightness range, indicating that the image light is too exposed.

[0030] When most of the data in the histogram is concentrated near the left edge, that is, the grayscale of the image is concentrated in the low brightness range, indicating that the image is underexposed. In the process of photogrammetry, different humidity and temperature in the air will produce different atmospheric refractions, causing image distortion; in addition, due to factors such as the light and wind strength at the seaside, we classify these scenes as abnormal scenes. The photos taken in these scenes may be blurred or have weak light, and their histograms should show the described shape, with poor image quality.

[0031] Further, if Figure 1 As shown, in S4, the catheter rack target data sets in the above two scenarios are respectively labeled and constructed, and the two data sets are expanded using image enhancement technology; the data sets are divided, and the corresponding training sets are respectively sent to the real-time target detection algorithm for training, and the weights of the model's last training and the weights when the training effect reaches the best are saved; the test image is sent to the trained model for target mark detection, and its confidence score and the corresponding target box position information are obtained; the area detected as the target is cropped from the image, and its position information is recorded.

[0032] In this embodiment, the YOLOv5 network is used to roughly detect possible circular target areas, including: respectively marking and constructing the catheter rack target data sets in the above two scenarios, and converting them into YOLO format; using image enhancement techniques such as mosaic, random hue change, and random saturation change to expand the two data sets.

[0033] The two catheter target data sets were divided into training set and test set at a ratio of 8:2. The image input size was defined as 768*768, the model training rounds and batchsize were defined. The corresponding training sets were sent to the YOLOv5 network for training, and the weights were updated. The weights of the last training of the model and the weights when the training effect reached the best were saved.

[0034] Finally, the test image is sent to the trained model for target mark detection to obtain its confidence score and the location information of the corresponding target frame; according to the target frame information predicted by the YOLOv5 network, the area detected as the target is cropped from the image and its location information is recorded.

[0035] Further, if Figure 1 , Figure 6 and Figure 7 As shown, in S5, for the photos taken under normal imaging conditions, the boundary part of the ellipse can be fitted according to the least squares principle; in S5, for the photos taken under abnormal imaging conditions, relying on the flow conservation characteristics, the gradient entering through the arc of the outermost ellipse must be equal to the gradient flowing out through the arc of the innermost ellipse, thereby realizing ellipse detection.

[0036] In S6, the outermost circular arc segment is used to locate the center of the circle based on the principle that the intersection of two straight lines after projective transformation is still the intersection of the two straight lines after transformation, and the center of the circle is located; in S7, the area of ​​interest where the coded mark point is located is selected, the annular bands of the area of ​​interest are extracted, the numbers represented by each annular band are calculated, the relative position relationship between the annular bands is calculated, the binary code value of the coded mark point is calculated, and converted into a decimal code number.

[0037] In this embodiment, for photos taken under normal imaging conditions, firstly, the pixels of the cropped possible circular target area are analyzed one by one in 8 neighborhoods, and then the offset of the precise edge point A' in the direction is calculated according to the grayscale difference of the adjacent pixels in 8 directions around the center point. Finally, the boundary part of the ellipse can be fitted according to the least squares principle. Assuming that the center of the ellipse is (x0, y0), the major semi-axis is a, the minor semi-axis diameter is b, and the angle between the major semi-axis and the x-axis is θ, the coordinate system is translated and rotated to form a regular ellipse with (x0, y0) as the center and the major semi-axis and the x-axis angle of 0, then the coordinates of the point (x, y) on the ellipse are transformed to (xr, yr). The functional relationship between (xr, yr) and (x, y) is:

[0038] = = The transformed point (xr, yr) satisfies the equation: + =1 The unknowns are xr, yr, a, and b. The elliptic equation is nonlinear for the unknowns, and the error equation is obtained by linearizing them. The initial values ​​of the unknowns (x0, y0), a, and b are determined by the edge points obtained from each area. According to the least squares principle, it is continuously iterated until the correction number of the unknowns is less than the set threshold. Assume the function:

[0039] + -1 Let v1 be 4 / a2; v2 be 16 / a2. Substitute all edge points of the area of ​​interest into the above function f. If the number of points whose absolute value of f is less than the set threshold v1 exceeds 95% of the total number of points, and the maximum absolute value of f does not exceed the set threshold v2, that is, most edge points are within the threshold v1 range of the fitted ellipse, and the farthest point does not exceed the threshold v2. If the above two conditions are met, it is considered that the edge points constitute an ellipse.

[0040] Photos taken under abnormal imaging conditions use the 45-degree overlapping area of ​​the three-layer circular arc in the target and rely on the flow conservation characteristics. The gradient entering through the arc of the outermost ellipse must be equal to the gradient flowing out through the arc of the innermost ellipse, thus realizing ellipse detection.

[0041] First, pixel selection and region pair grouping are performed to create a path in the image in the form of a sequence of 6 connected edge pixels (3 overlapping circular arc segments), so that the direction of the path segment starting from pixel p is given by the image gradient p. Then, the pixel with the most occurrences is selected as the path endpoint through a voting procedure.

[0042] The selected endpoints are grouped into inner regions, forming an approximate polygon of convex arcs, and the associated start points are grouped into outer regions. The voting process applied to real images is then followed by region pair selection, selecting only those region pairs that satisfy a "conservative constraint", which essentially guarantees that the gradient magnitude inflow through the outer region must equal the gradient magnitude outflow through the inner region.

[0043] Preferably, in S6, based on the principle that the center of the marker point can be obtained when the intersection of two straight lines is still the intersection of the two straight lines after projective transformation, the center of the circle is located using the arc segment of the outermost circular ring. First, based on the extracted possible circular target area, the center of mass of the innermost elliptical ring is extracted, and the center of mass of the innermost elliptical ring and the center of mass of the connected domain of the outermost coding ring are connected by a straight line.

[0044] Calculate the distance from all edge points on the connected domain to the straight line, set a distance threshold, and count the number of points less than the distance threshold; rotate the straight line 180 degrees around the center of the circle in a fixed step size to both sides of the straight line, and count the number of points less than the distance threshold each time the rotation occurs, and select the one with the largest number in each direction. The continuous contour of the points less than the threshold at that moment on the edge is the straight line edge.

[0045] Finally, the same operation is performed on each connected domain on the coding ring to obtain all straight edges; then the Gaussian fitting method with good noise resistance and positioning accuracy is selected for sub-pixel edge positioning. In the gradient direction of the image edge, the first-order derivative of the grayscale distribution is approximately a Gaussian distribution, and the center of the Gaussian distribution is the position with the highest edge point accuracy.

[0046] exp[- ] In the relationship, x is the coordinate point in the direction of the edge gradient, y is the first-order derivative of the grayscale value, μ is the mean of the Gaussian function, σ is the standard deviation of the Gaussian function, and k is the amplitude of the Gaussian function. By obtaining the position parameter μ, the sub-pixel position of the edge can be obtained.

[0047] First, a straight line is fitted for each edge, and the normal direction of the straight line edge is used to represent the gradient direction of the straight line edge. Then, points are taken along the normal direction to calculate the grayscale difference and fit the Gaussian model to obtain the sub-pixel edge position. Finally, based on the Hough transform idea and the method of finding the center coordinates using the random sampling consensus (RANSAC) algorithm, the obtained sub-pixel edge points are mapped to the parameter space. Then, all the intersection points of the mapping curve corresponding to the points on each straight line are found. In order to reduce the influence of noise points and mis-extracted points on the results, the intersection points are fitted in the parameter space using the RANSAC algorithm, and the curve is mapped to the original space as the center coordinates.

[0048] Preferably, S7 selects the area of ​​interest where the coded mark point is located, which contains the coded annular band information; extracts the annular band of the area of ​​interest; calculates the number represented by each annular band, and calculates the ratio of the area of ​​the smallest unit annular band to the area of ​​the center circle according to the ratio of the center circle radius to the inner and outer diameters of the annular band when the coded mark point is designed, to obtain the number represented by each annular band; calculates the relative position relationship between the annular bands, through the angle between the center of mass of the connected domain and the center of the center circle; finally, calculates the binary code value of the coded mark point, and converts it into a decimal code number, i.e., decoding is achieved.

[0049] The target positioning and identification method based on high-precision photogrammetry of the catheter rack obtains several images of the catheter rack through photogrammetry, draws the histogram of the image and judges the imaging conditions, uses the real-time target detection algorithm to roughly detect possible circular target areas, ellipse detection under different imaging conditions, locates the center of the circle, and decodes the coded mark. Among them, the imaging conditions are divided into normal imaging conditions and abnormal imaging conditions. The images obtained under normal imaging conditions are detected by edge detection for ellipse detection, and the images obtained under abnormal imaging conditions are detected by ellipse detection using three-layer overlapping areas according to the flow conservation principle, effectively realizing high-precision measurement of the key points of the catheter rack.

[0050] Obviously, the above embodiments of the present invention are merely examples for clearly illustrating the present invention, and are not intended to limit the embodiments of the present invention. For those skilled in the art, other different forms of changes or modifications can be made based on the above description. It is not necessary and impossible to list all the embodiments here. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the claims of the present invention.

Claims

1. A target positioning and identification method for high-precision photogrammetry of a catheter frame, characterized in that: The following steps are involved: S1. Obtain several images of the catheter rack through photogrammetry; S2, draw a histogram of the image; S3, determining imaging conditions; S4, using a real-time target detection algorithm to roughly detect possible circular target areas; S5, ellipse detection under different imaging conditions; S6, locate the center of the circle; S7. Decode the encoding flag.

2. The target positioning and identification method for high-precision photogrammetry of a catheter frame according to claim 1 is characterized in that: In S1, a number of images of the catheter rack were taken from different angles and directions using a drone and cut into slices.

3. The target positioning and identification method for high-precision photogrammetry of a catheter frame according to claim 1, characterized in that: In S2, based on the images captured by the drone, the histogram of each image is calculated and drawn. The horizontal axis represents the brightness value of 0-255, and the vertical axis represents the number of pixels with corresponding brightness in the photo.

4. The target positioning and identification method for high-precision photogrammetry of a catheter frame according to claim 1, characterized in that: In S3, the imaging conditions of the image are divided into normal imaging conditions and abnormal imaging conditions according to the histogram of the image; normal imaging conditions refer to good lighting conditions and good overall contrast of the image, and abnormal imaging conditions refer to the situation where the lighting and wind strength at the seaside will affect the imaging.

5. The target positioning and identification method for high-precision photogrammetry of a catheter frame according to claim 1, characterized in that: In S4, the catheter rack target data sets in the above two scenarios are labeled and constructed respectively, and the two data sets are expanded using image enhancement technology; the data sets are divided, and the corresponding training sets are sent to the real-time target detection algorithm for training, and the weight of the model's last training and the weight when the training effect reaches the best are saved.

6. The target positioning and identification method for high-precision photogrammetry of a catheter frame according to claim 5, characterized in that: The test image is sent to the trained model to detect the target mark, and its confidence score and the location information of the corresponding target box are obtained; the area detected as the target is cropped from the image and its location information is recorded.

7. The target positioning and identification method for high-precision photogrammetry of a conductor frame according to claim 4, characterized in that: In S5, the border part of the ellipse can be fitted based on the least squares principle for the photos taken under normal imaging conditions.

8. The target positioning and identification method for high-precision photogrammetry of a catheter frame according to claim 7, characterized in that: In S5, for photos taken under abnormal imaging conditions, relying on the flow conservation property, the amount of gradient entering through the arc of the outermost ellipse must be equal to the amount of gradient flowing out through the arc of the innermost ellipse, thus achieving ellipse detection.

9. The target positioning and identification method for high-precision photogrammetry of a catheter frame according to claim 1, characterized in that: In S6, the outermost circular arc segment is used to locate the center of the circle based on the principle that the intersection of two straight lines after projective transformation is still the intersection of the two straight lines after transformation and the center of the landmark point can be obtained.

10. The target positioning and identification method for high-precision photogrammetry of a catheter frame according to claim 1, characterized in that: In S7, the region of interest where the coding mark point is located is selected, the annular bands of the region of interest are extracted, the numbers represented by each annular band are calculated, the relative position relationship between the annular bands is calculated, the binary code value of the coding mark point is calculated, and converted into a decimal code number.

Images