A rectangular target positioning method and device based on line segment fusion

Abstract

The present application relates to the technical field of image recognition, and discloses a rectangular target positioning method and device based on line segment fusion, comprising: converting an input image into a grayscale image, and performing dynamic threshold segmentation on the grayscale image to obtain a first binary image; using an image enhancement algorithm to enhance the grayscale image to obtain an enhanced image with equalized brightness; performing adaptive threshold segmentation to obtain a second binary image, and ANDing the first binary image with the second binary image to obtain a third binary image; extracting contours from the third binary image, and screening and eliminating contours that do not meet the requirements to obtain a preliminary positioning contour; dividing the preliminary positioning contour into four parts according to a rotated rectangle, performing line segment detection on each part to obtain a corresponding line segment set; fusing each line segment set separately to obtain only one line segment corresponding to each side; refitting the rotated rectangle to obtain a final positioning contour; the present application is lightweight and efficient, and effectively improves the positioning accuracy and efficiency of rectangular objects.

Description

Technical Field

[0001] The present application relates to the technical field of image recognition, and in particular to a method and device for locating a rectangular target based on line segment fusion. Background Art

[0002] Industrial vision inspection plays a vital role in modern manufacturing, and its application scope covers a variety of processes. In particular, in the field of rectangular object positioning, such as packaging boxes, packaging boxes, product labels, etc., the automated detection and positioning has become an important demand in industrial production. In traditional manufacturing, the accurate positioning of these rectangular objects plays a key role in subsequent production processes, quality control, and automated processing. Therefore, efficient and accurate rectangular object positioning technology is of great significance to improving production efficiency and reducing costs.

[0003] However, existing rectangular object positioning technology faces a series of challenges in industrial visual inspection. One of the most notable problems is the unevenness of lighting conditions and the complexity of the background. In the manufacturing site, due to changes in lighting angle and intensity, as well as reflections on the surface of objects, rectangular objects often appear blurred, distorted, or interfered with by the background in the image, such as Figure 1 As shown in Figure 2, the existing technology performs poorly in locating rectangular objects in complex industrial environments. Figure 2 shown.

[0004] In some special manufacturing scenarios, such as the product flow process on a conveyor belt, the stray light sources on the conveyor belt and the reflection of the object itself are more serious. This not only reduces the clarity of the rectangular object outline, but also increases the complexity of the positioning algorithm. Traditional image processing methods often show limitations in dealing with these problems, resulting in the inability to meet the high-precision and high-efficiency rectangular object positioning requirements in industrial production. Summary of the invention

[0005] The purpose of this application is to provide a rectangular target positioning method and device based on line segment fusion that is lightweight and efficient and effectively improves the positioning accuracy and positioning efficiency of rectangular objects.

[0006] In order to achieve the above object, the present invention adopts the following technical solutions:

[0007] A rectangular target positioning method based on line segment fusion of the present invention operates an input image through image recognition and image processing technology to obtain a positioning contour of the target, comprising the following steps:

[0008] Convert the input image into a grayscale image, and perform dynamic threshold segmentation on the grayscale image to obtain a first binary image;

[0009] Use image enhancement algorithm to enhance the grayscale image and obtain an enhanced image with equalized brightness;

[0010] Performing adaptive threshold segmentation on the enhanced image to obtain a second binary image, and ANDing the first binary image with the second binary image to obtain a third binary image;

[0011] Extract the contour features of the third binary image, and remove the contours that do not meet the requirements from the extracted contour features, and the rotated rectangles corresponding to the remaining contours are the preliminary positioning contours;

[0012] The preliminary positioning contour is divided into four parts according to the four sides of the rotated rectangle, and line segment detection is performed on each part to obtain the corresponding line segment set;

[0013] By merging line segments and removing redundant line segments, each line segment set is merged separately to obtain a unique line segment corresponding to each edge;

[0014] The rotated rectangle is fitted according to the intersection points of the line segments to obtain the target positioning contour.

[0015] Furthermore, the image enhancement algorithm includes an image enhancement algorithm using an improved sigmoid curve, and the pixel value of the input image is adjusted by a gain function of Formula 1 to obtain an enhanced image;

[0016]

[0017] Among them, hdr_alpha, hdr_beta and hdr c They represent the steepness of the curve, which are parameters adjusted by the user. i represents the input grayscale value, gains i represents the gain value corresponding to the input gray value i, gain_sub represents the difference between the upper limit of the gain and the lower limit of the gain;

[0018] The upper limit of the gain is calculated according to formula 2;

[0019]

[0020] The lower limit of the gain is calculated according to formula 3;

[0021]

[0022] The difference between the upper limit of the gain and the lower limit of the gain is calculated according to formula 4;

[0023] gain_sub=max_gain-min_gain Formula 4.

[0024] Furthermore, the method for removing contours that do not meet the requirements from the extracted contour features includes: extracting corresponding contours based on the boundary between the foreground and the background of the third binary image, fitting a minimum circumscribed rotated rectangle based on each extracted contour, and determining whether the aspect ratio of the fitted rotated rectangle is within the preset rectangle aspect ratio range, and if not, removing the contour; if so, determining whether the area of ​​the rotated rectangle is within the area range of the preset rectangle, and if not, removing the contour; if so, taking the contour corresponding to the rotated rectangle as the preliminary positioning contour.

[0025] Further, the preliminary positioning contour is divided into four parts according to the four sides of the rotated rectangle, and the method of performing line segment detection on each part includes:

[0026] Fill the preliminary positioned contour to obtain a contour binary image;

[0027] The rotated rectangle circumscribing the outline is used as the ROI area for affine sampling to obtain the ROI image;

[0028] Search inward from the top, bottom, left, and right boundaries of the ROI;

[0029] Map the ROI position to the original image, find the non-zero pixel points in the original image, and record the non-zero pixel points as the contour points of the edges corresponding to the corresponding parts, and obtain the contour points corresponding to the four edges;

[0030] For each edge, the Hough line segment detection algorithm is used to obtain the line segment set corresponding to the edge.

[0031] Furthermore, the method for merging similar line segments in each line segment set includes:

[0032] Determine whether the two line segments are merged based on the angles between the two line segments and the horizontal line;

[0033] Determine whether two line segments are merged based on their intersection relationship;

[0034] Calculate the new angle of the fused line according to the intersection relationship of the two line segments;

[0035] Calculate the points that the fused line must pass through based on the intersection relationship of the two line segments;

[0036] Calculate the starting point coordinates and the end point coordinates of the fused line segment according to the intersection relationship of the two line segments;

[0037] A fused line is constructed according to the new angle of the fused line and the points that the fused line must pass through, and the coordinates of the starting point and the end point are mapped to the fused line to obtain a fused line segment.

[0038] Further, the method of determining whether two line segments are merged according to the angles between the two line segments and the horizontal line includes:

[0039] Calculate the angle difference between the two angles between the two line segments and the horizontal line. If the angle difference exceeds the set threshold, the line segments will not be merged.

[0040] Further, the method of judging whether two line segments are merged according to the intersection relationship of the two line segments includes:

[0041] When two line segments are collinear, the interval between the two collinear line segments is calculated. If the interval exceeds the set value, they are not merged.

[0042] When two line segments are parallel but not collinear, the distance between the two parallel line segments is calculated. If the distance exceeds the set value, they are not merged. At the same time, the interval between the two parallel line segments is calculated. If the interval exceeds the set value, they are not merged.

[0043] When two line segments have an intersection, calculate the distance between the two line segments with the intersection. If the distance exceeds the set value, they will not be merged. At the same time, calculate the interval between the intersecting line segments. If the interval exceeds the set value, they will not be merged.

[0044] When two line segments are neither parallel nor intersecting, the distance between the two line segments is calculated. If the distance exceeds the set value, they are not merged. At the same time, the interval between the two line segments is calculated. If the interval exceeds the set value, they are not merged.

[0045] Further, the method of calculating the new angle of the fused line according to the intersection relationship of the two line segments includes:

[0046] When two line segments are parallel or collinear, the new angle of the fused line is consistent with the original angle of the two line segments;

[0047] When two line segments are neither parallel nor intersecting, the length ratio of the two line segments that are neither parallel nor intersecting is calculated, the length ratio is used as the weight of the angle, and the weighted sum of the angles is obtained to obtain the new angle of the fused line;

[0048] When two line segments intersect, calculate the intersection of the two line segments, divide each line segment into two parts according to the intersection point, use the intersection point as the center of each line segment, and use the two parts divided by the intersection point as the radius. Set the angle to one degree to obtain two sectors, calculate the areas of the two sectors, and sum the two areas to obtain the angle weight of the corresponding line segment; normalize the two angle weights and calculate the new angle of the fused line.

[0049] Furthermore, the method of calculating the necessary passing points of the fused line according to the intersection relationship of two line segments includes:

[0050] When two line segments are collinear, take any point on the two line segments as the necessary point for the fusion line;

[0051] When two line segments are parallel or non-parallel and non-intersecting, the midpoints of the two line segments are calculated respectively, and the length of the line segment is used as the weight, and the weighted sum of the two midpoints is obtained to obtain the necessary passing point of the fused line;

[0052] When two line segments intersect, the midpoints of the two line segments are calculated respectively, and the angle weight is used as the midpoint weight. The weighted sum is calculated to obtain the points that the fused line must pass through.

[0053] Further, the method of calculating the starting point coordinates and the end point coordinates of the fused line segment according to the intersection relationship of the two line segments includes:

[0054] Calculate the starting coordinates of the merged line segment:

[0055] Take the minimum value of the x coordinates of the four endpoints as the x coordinate of the starting point of the fused line segment;

[0056] Take the minimum value of the y coordinates of the four endpoints as the y coordinate of the starting point of the fused line segment;

[0057] Calculate the coordinates of the end point of the merged line segment:

[0058] Take the maximum value of the x coordinates of the four endpoints as the x coordinate of the end point of the fused line segment;

[0059] Take the maximum value of the y coordinates of the four endpoints as the y coordinate of the end point of the merged line segment.

[0060] Furthermore, the method of removing redundant line segments includes:

[0061] Filter the line segments according to their rotation angles, and retain the line segments with angles close to the sides of the corresponding rotated rectangle;

[0062] Calculate the lengths of the line segments that have been filtered and retained, and compare them to get the line segment with the largest length.

[0063] Delete line segments whose length is less than 50% of the maximum length;

[0064] The line segments that have been filtered by angle and length are filtered according to their projection positions on the rectangular area.

[0065] Furthermore, the method for filtering the line segments after angle filtering and length filtering includes:

[0066] When the line segment corresponds to the upper side of the rotated rectangle, the mean of the y values ​​of the intersections of the line segment and the left and right sides of the rotated rectangle is calculated, and the line segment corresponding to the maximum mean is retained;

[0067] When the line segment corresponds to the lower side of the rotated rectangle, calculate the mean of the y values ​​of the intersections of the line segment and the left and right sides of the rotated rectangle, and retain the line segment corresponding to the minimum mean value;

[0068] When the line segment corresponds to the left side of the rotated rectangle, calculate the x value of the intersection of the line segment and the upper and lower sides of the rotated rectangle, and keep the line segment corresponding to the maximum x value;

[0069] When the line segment corresponds to the right side of the rotated rectangle, calculate the x value of the intersection of the line segment and the upper and lower sides of the rotated rectangle, and retain the line segment corresponding to the minimum x value.

[0070] A rectangular target positioning device based on line segment fusion is a device used to implement any one of the rectangular target positioning methods based on line segment fusion.

[0071] Compared with the prior art, the beneficial effects of this application are:

[0072] The present invention combines dynamic threshold segmentation with enhancement algorithm and adaptive threshold segmentation, which not only solves the background adhesion problem, but also improves the contour extraction speed, effectively copes with interference such as uneven lighting and reflection, ensures the high accuracy of the positioning result, and removes the raised part of the rectangular contour through line segment fusion technology, solves the problem of the positioning frame becoming larger due to strong background interference, and then finds the accurate rectangular target boundary line to achieve accurate positioning. It does not rely on deep learning, making the algorithm lighter and more efficient, and provides advantages in applications in scenarios with limited resources or high real-time requirements. By more accurate and robust rectangular object positioning, the efficiency of the automated production line is effectively improved, and production interruptions and resource waste caused by positioning errors are reduced; the present invention is lightweight and efficient, and effectively improves the positioning accuracy and efficiency of rectangular objects. BRIEF DESCRIPTION OF THE DRAWINGS

[0073] In order to more clearly illustrate the technical solutions of the implementation methods of the present application, the drawings required for use in the implementation methods will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0074] Figure 1 is the input original image that needs to be recognized;

[0075] Figure 2 It is a binarized image directly binarized by the prior art;

[0076] Figure 3 It is a flow chart of a rectangular target positioning method based on line segment fusion of the present invention;

[0077] Figure 4 It is an image enhancement effect diagram of grayscale image enhancement by a rectangular target positioning method based on line segment fusion of the present invention;

[0078] Figure 5 is a second binary image obtained by a rectangular target positioning method based on line segment fusion according to the present invention;

[0079] Figure 6 It is a contour extraction result diagram obtained by a rectangular target positioning method based on line segment fusion of the present invention;

[0080] Figure 7 It is a preliminary positioning outline obtained by a rectangular target positioning method based on line segment fusion of the present invention;

[0081] Figure 8 It is a contour segmentation effect diagram obtained by a rectangular target positioning method based on line segment fusion of the present invention;

[0082] Fig. 9 It is an effect diagram obtained through line segment detection of a rectangular target positioning method based on line segment fusion of the present invention;

[0083] Fig.10 This is another effect diagram obtained through line segment detection of a rectangular target positioning method based on line segment fusion of the present invention;

[0084] Fig.11 It is an effect diagram obtained by line segment fusion of a rectangular target positioning method based on line segment fusion of the present invention;

[0085] Fig.12 It is the final positioning contour map obtained by the rectangular target positioning method based on line segment fusion of the present invention;

[0086] Fig.13 It is a line segment graph with two unequal slopes but not intersecting in a rectangular target positioning method based on line segment fusion of the present invention;

[0087] Fig.14 It is a graph of two intersecting line segments with unequal slopes in a rectangular target positioning method based on line segment fusion of the present invention;

[0088] Fig.15 It is a graph of two line segments being collinear and spaced in a rectangular target positioning method based on line segment fusion of the present invention;

[0089] Fig.16 It is a diagram of two parallel and spaced line segments in a rectangular target positioning method based on line segment fusion of the present invention;

[0090] Fig.17 It is a diagram in which two line segments do not touch each other and have an interval in a rectangular target positioning method based on line segment fusion of the present invention;

[0091] Fig.18It is another type of two line segments not touching and having interval graph in a rectangular target positioning method based on line segment fusion of the present invention;

[0092] Fig.19 It is a non-contact and non-spaced graph of two line segments in a rectangular target positioning method based on line segment fusion of the present invention;

[0093] Fig. 20 It is an effect diagram of two line segments rotating with the intersection point as the center in a rectangular target positioning method based on line segment fusion of the present invention;

[0094] Fig.21 In the process of merging two line segments in a rectangular target positioning method based on line segment fusion of the present invention, when the two line segments are parallel or non-parallel and non-intersecting, a new line segment and a graph of the passing points are obtained by merging;

[0095] Fig. 22 In the process of fusion of two line segments in a rectangular target positioning method based on line segment fusion of the present invention, when the two line segments are parallel or non-parallel and non-intersecting, a new line segment and a graph of the starting point and end point of the new line segment are obtained by merging. DETAILED DESCRIPTION

[0096] The following describes the embodiments of the present invention by specific examples, and those skilled in the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the following embodiments and features in the embodiments can be combined with each other without conflict.

[0097] It should be noted that the illustrations provided in the following embodiments are only schematic illustrations of the basic concept of the present invention, and thus the drawings only show the layers related to the present invention rather than being drawn according to the number, shape and size of the layers in actual implementation. In actual implementation, the type, quantity and proportion of each layer may be changed arbitrarily, and the layer layout may also be more complicated.

[0098] In the following description, numerous details are discussed to provide a more thorough explanation of embodiments of the present invention; however, it is apparent to one skilled in the art that embodiments of the present invention may be practiced without these specific details.

[0099] See also Figure 3 A rectangular target positioning method based on line segment fusion is proposed. The input image is operated by image recognition and image processing technology to obtain the positioning contour of the target. The method includes the following steps:

[0100] Convert the input image into a grayscale image, and perform dynamic threshold segmentation on the grayscale image to obtain a first binary image;

[0101] See also Figure 4 , use the image enhancement algorithm to enhance the grayscale image and obtain an enhanced image with equalized brightness;

[0102] See also Figure 5 , perform adaptive threshold segmentation on the enhanced image to obtain a second binary image, and AND the first binary image with the second binary image to obtain a third binary image;

[0103] See also Figure 6 , extract the contours of the third binary image, and remove the contours that do not meet the requirements in the contour features. The rotated rectangles corresponding to the remaining contours are the preliminary positioning contours. Please refer to Figure 7 ;

[0104] See also Figure 8 , divide the preliminary positioning contour into four parts according to the rotation rectangle, see Fig. 9 and Fig.10 Perform line segment detection on each part to obtain the corresponding line segment set;

[0105] See also Fig.11 , by merging line segments and removing redundant line segments, each line segment set is merged separately, so that each edge corresponds to only one line segment;

[0106] See also Fig.12 , refit the rotated rectangle according to the intersection of the straight lines to obtain the final positioning contour.

[0107] If there are no intersections, less than four line segments, or other abnormal situations, the original rotated rectangle is used as the result.

[0108] The present invention combines dynamic threshold segmentation with enhancement algorithm and adaptive threshold segmentation, which not only solves the background adhesion problem, but also improves the contour extraction speed, effectively copes with interference such as uneven lighting and reflection, ensures the high accuracy of the positioning result, and removes the raised part of the rectangular contour through line segment fusion technology, solves the problem of the positioning frame becoming larger due to strong background interference, and then finds the accurate rectangular target boundary line to achieve accurate positioning. It does not rely on deep learning, making the algorithm lighter and more efficient, and provides advantages in applications in scenarios with limited resources or high real-time requirements. By more accurate and robust rectangular object positioning, the efficiency of the automated production line is effectively improved, and production interruptions and resource waste caused by positioning errors are reduced; the present invention is lightweight and efficient, and effectively improves the positioning accuracy and efficiency of rectangular objects.

[0109] In this embodiment, dynamic threshold segmentation is to dynamically determine the threshold of each pixel according to the local features of the image; specifically, a mean-based dynamic threshold segmentation method is adopted, and for each pixel, the mean of the neighborhood pixels of the corresponding pixel is calculated, and the mean is used as the threshold of the corresponding pixel.

[0110] The principle of dynamic threshold segmentation is to dynamically determine the threshold of each pixel based on the local features of the image. This is different from the global threshold, which is a fixed threshold determined based on the pixel value of the entire image. Dynamic threshold segmentation takes into account factors such as illumination changes and noise in different areas of the image, and can therefore better adapt to conditions in different areas. The present invention adopts a mean-based dynamic threshold segmentation method. Specifically, for each pixel, the mean of its surrounding neighborhood is calculated, and the mean is used as the threshold of the pixel. The neighborhood range is adjustable. The neighborhood range of the present application is generally around 10, such as 11 or 15.

[0111] In this embodiment, the image enhancement algorithm specifically adopts an image enhancement algorithm of an improved sigmoid curve, and adjusts the pixel value of the input image through the gain function of Formula 1 to obtain a visualized image with better brightness perception and contrast;

[0112]

[0113] Among them, hdr_alpha, hdr_beta and hdr c They represent the steepness of the curve and are user-adjustable parameters. i represents the input grayscale value. gains[i] is the gain value corresponding to the input grayscale value i. gain_sub represents the difference between the upper limit of the gain and the lower limit of the gain.

[0114] The upper limit of the gain is calculated according to formula 2;

[0115]

[0116] The lower limit of the gain is calculated according to formula 3;

[0117]

[0118] The difference between the upper limit of the gain and the lower limit of the gain is calculated according to formula 4;

[0119] gain_sub=max_gain-min_gain Formula 4.

[0120] See also Figure 4The gain function adjusts the pixel values ​​of the input image so that the image has better brightness perception and contrast when visualized. The calculation of the gain value involves exponents, exponential functions, and a series of basic mathematical operations. Each grayscale value corresponds to a gain value, and the original grayscale value multiplied by the gain value is the final mapping curve. The midpoint of the sigmod curve is the maximum gain multiple.

[0121] In this embodiment, the adaptive threshold segmentation specifically adopts the Otsu threshold segmentation method, and finds an optimal threshold by maximizing the inter-class variance so that the inter-class variance reaches the maximum value.

[0122] See also Figure 5 , the enhanced image is segmented by Otsu's method to obtain the second binary image, and the first binary image and the second binary image are ANDed to obtain the third binary image; Otsu's method is to find an optimal threshold by maximizing the inter-class variance so that the inter-class variance reaches the maximum value. Otsu's method is an adaptive threshold segmentation method that does not require manual input of any parameters.

[0123] The results of dynamic threshold segmentation are often too detailed and have a lot of noise, which greatly increases the time consumption of the next step. The foreground of the result after Otsu threshold segmentation is more concentrated and has fewer noise points, but sometimes there will be adhesion of part of the background. Therefore, the two can be combined, that is, Otsu threshold segmentation can be regarded as the denoising step of dynamic threshold segmentation, which makes it possible to analyze the connected domain more quickly and increase the speed of contour extraction. Dynamic threshold segmentation can also be regarded as post-processing of Otsu threshold segmentation to solve the occasional background adhesion problem of Otsu method.

[0124] In this embodiment, the contour extraction is based on the connected domain analysis of the image;

[0125] See also Figure 6 ,Contour feature screening specifically includes contour detection, according to the boundary between the foreground and the background in the binary image, a minimum circumscribed rotated rectangle is fitted according to each extracted contour, and it is determined whether the aspect ratio of the rotated rectangle is within the preset rectangle aspect ratio range. If not, the contour is removed; if so, it is determined whether the area of ​​the rotated rectangle is within the preset rectangle area range. If not, the contour is removed; if so, the rotated rectangle is used as the preliminary positioning contour, that is, the contour that does not meet any of the two judgment conditions is removed.

[0126] A connected domain is a group of pixels in an image that have the same pixel value and are adjacent to each other. In contour detection, the main focus is on the boundary between the foreground (white part) and the background (black part) in the binary image. After extracting the contour, a minimum circumscribed rotated rectangle is fitted for each contour. The aspect ratio range of the preset rectangle refers to the range between the maximum and minimum values ​​of the aspect ratio of the preset rectangle; the area range of the preset rectangle refers to the range between the maximum and minimum values ​​of the area of ​​the preset rectangle; respectively determine whether the aspect ratio and area of ​​the rotated rectangle are within the aspect ratio range of the preset rectangle and the area range of the preset rectangle, remove the rotated rectangles that are not within any range, and use the remaining rotated rectangles as the preliminary positioning contours, please refer to Figure 7 .

[0127] In this embodiment, the preliminary positioning contour is divided into four parts according to the rotation rectangle, see Figure 8 , perform line segment detection on each part, the specific method is:

[0128] First, fill the preliminary positioning contour to obtain the contour binary image;

[0129] Then the rotated rectangle circumscribing the outline is used as the ROI area for affine sampling to obtain the ROI image;

[0130] Search inward from the top, bottom, left, and right boundaries of the ROI;

[0131] Map the ROI position to the original image, find the non-zero pixel point in the original image, and record this non-zero pixel point as the contour point of the edge, and obtain the contour points corresponding to the four edges;

[0132] Then, the Hough line segment detection algorithm is used for each edge to obtain the line segment set corresponding to the edge.

[0133] The rotated rectangle has four sides. For each side, traverse each point. For each point, search the binary image of the contour in the direction perpendicular to the side and toward the inside of the rotation matrix. Once a point that is not 0 is found, the search ends and moves to the next point on this side. Repeat the operation to find the contour point corresponding to this side. Then perform the same operation on the remaining three sides. When there is background interference in the image, the preliminary positioning contour often has bulges. In order to deal with this situation, the present invention removes the protruding area after line segment detection. In order to ensure the accuracy of line segment detection, first divide the corresponding rotated rectangle into four parts according to each preliminary positioning contour, corresponding to the four sides of the rectangular contour.

[0134] Specifically, the preliminary positioning contour is first filled to obtain a contour binary image; then the circumscribed rotated rectangle of the contour is used as the ROI area for affine sampling to obtain the ROI image. For the sake of subsequent segmentation efficiency, the size of the rotated rectangle can be reduced during sampling. Then the boundary of the ROI image begins to search inward along the direction of the four sides of the rotated rectangle, and the ROI position is mapped to the original image to find the non-zero pixel points of the original image. It is recorded as the contour point of the edge, and the contour points corresponding to the four edges are obtained. Then, the Hough line segment detection algorithm is used for each edge to obtain the line segment set corresponding to the edge.

[0135] In this embodiment, the conditions for merging line segments include the angle, intersection relationship, length relationship, and distance relationship of the line segments.

[0136] The definition of the distance between line segments and the interval between line segments in the present invention is as follows: Fig.13 , indicating that the slopes are unequal but do not intersect, see Fig.14 , indicating that the slopes are not equal and intersect, where the red dashed line is the distance between the two line segments. First, sort the endpoints of the line segments and take the two middle points. Then, based on these two points, calculate the distance from each to the other line segment. If the two line segments are collinear, the distance is (0, 0). If the two line segments are parallel, then calculate according to the distance of parallel lines.

[0137] See also Figures 15 to 19 If two collinear line segments do not have overlapping areas, it is considered that there is a gap between them. If the two line segments are parallel, they are mapped on a straight line and the gap is calculated according to the collinearity rule. If the line segments intersect, there is no gap. If the two line segments have different slopes but do not touch, the two endpoints form a horizontal rectangle. If the two rectangles touch, it is considered that there is no gap, otherwise it is considered that there is a gap, and the minimum distance between the endpoints is considered to be the gap.

[0138] In this embodiment, line segments are merged, and the specific method of merging similar line segments is as follows:

[0139] Whether to merge is determined based on the angles between the two line segments and the horizontal line. Specifically, the angle difference between the two angles between the two line segments and the horizontal line is calculated. If the angle difference exceeds the set threshold, the line segments are not merged. The angle range is -90 degrees to 90 degrees, and the threshold is generally very small, such as 10 degrees, 13 degrees, and 7 degrees.

[0140] Determine whether to merge two line segments based on their intersection relationship. Specifically:

[0141] When two line segments are collinear, the interval between the two collinear line segments is calculated. If the interval exceeds the set value, they are not merged.

[0142] When two line segments are parallel but not collinear, the distance between the two parallel line segments is calculated. If the distance exceeds the set value, they are not merged. At the same time, the interval between the two parallel line segments is calculated. If the interval exceeds the set value, they are not merged either. That is, as long as either the distance or the interval exceeds the set value, they are not merged.

[0143] When two line segments have an intersection, the distance between the two line segments with the intersection is calculated. If the distance exceeds the set value, they are not merged. At the same time, the interval between parallel line segments is calculated. If the interval exceeds the set value, it is not merged either. That is, as long as either the distance or the interval exceeds the set value, they are not merged.

[0144] When two line segments are neither parallel nor intersecting, the distance between the two line segments is calculated. If the distance exceeds the set value, they are not merged. At the same time, the interval between the two line segments is calculated according to the collinear line segments. If the interval exceeds the set value, they are not merged; that is, as long as either the distance or the interval exceeds the set value, they are not merged.

[0145] Calculate the new angle, specifically:

[0146] When the two line segments are parallel or collinear, the angle is the same as the original angle. When the two line segments are neither parallel nor intersecting, the length ratio of the two line segments that are neither parallel nor intersecting is calculated, and the length ratio is used as the weight of the angle. The weighted sum of the angles is used as the new angle. Please refer to Fig. 20 , when two line segments intersect, calculate the intersection of the two line segments, and then divide each line segment into two parts according to the intersection point. Each line segment takes the intersection point as the center, and the two parts divided by the intersection point are used as the radius respectively. The angle is set to one degree to obtain two sectors. The areas of the two sectors are calculated, and the sum of the two areas is used as the angle weight of the corresponding line segment; finally, the two angle weights are normalized and the new angle is calculated;

[0147] Calculate the necessary points, specifically:

[0148] When two line segments are collinear, take any point on the two line segments as the necessary point; a point and an angle can determine a straight line, and if the two lines are collinear, then the line is determined. This point only needs to be on the line, and for convenience, you can take an endpoint of any line.

[0149] See also Fig.21 , when two line segments are parallel or non-parallel and non-intersecting, calculate the midpoints of the two line segments respectively, and use the line segment length as the weight, and sum the two midpoints to get the must-pass point;

[0150] See also Fig. 22 , when two line segments intersect, calculate the midpoints of the two line segments respectively, use the angle weight as the midpoint weight, and calculate the necessary passing point by weighted summation;

[0151] Calculate the starting point coordinates and end point coordinates of the merged straight line, specifically:

[0152] Starting point coordinates:

[0153] Take the minimum value of the x coordinates of the four endpoints as the x coordinate of the starting point of the new line segment;

[0154] Take the minimum value of the y coordinates of the four endpoints as the y coordinate of the starting point of the new line segment;

[0155] End point coordinates:

[0156] Take the maximum value of the x coordinates of the four endpoints as the x coordinate of the end point of the new line segment;

[0157] Take the maximum value of the y coordinates of the four endpoints as the y coordinate of the end point of the new line segment;

[0158] Construct a straight line based on the new angle and the required points, and map the start and end points onto the straight line to form a line segment.

[0159] First determine the angle, then determine a point, then a straight line can be determined, and then a line segment can be determined by specifying any two points on the straight line.

[0160] In this embodiment, the specific method of removing redundant line segments is as follows:

[0161] Filter the line segments according to their rotation angles, and only keep the line segments with angles close to the corresponding rotated rectangle sides;

[0162] The rotated rectangle has a rotation angle, which refers to the angle with the horizontal direction. The angle of the line segment is also the angle with the horizontal direction. When filtering, only the line segments close to the rotation angle are retained. The filtering threshold is set to about 20.

[0163] Calculate the length of all the line segments retained after angle filtering and find the longest line segment; delete the line segments whose length is less than 50% of the longest line segment to ensure that only relatively long line segments are retained;

[0164] Sort the remaining line segments according to their projection positions on the rectangular area;

[0165] Specific sorting method:

[0166] If the line segment after merging corresponds to the upper side of the rotated rectangle, then the sorting condition is the average of the y values ​​of the intersection points of the line segment and the left and right sides of the rotated rectangle. The larger the average, the closer it is to the center of the rectangle.

[0167] If the line segment after merging corresponds to the lower side of the rotated rectangle, then the sorting condition is the average of the y values ​​of the intersection points of the line segment and the left and right sides of the rotated rectangle. The smaller the average, the closer it is to the center of the rectangle.

[0168] If the line segment after merging corresponds to the left side of the rotated rectangle, then the sorting condition is the x value of the coordinate of the intersection of the line segment and the upper and lower sides of the rotated rectangle. The larger the x value, the closer it is to the center of the rectangle.

[0169] If the line segment after merging corresponds to the right side of the rotated rectangle, then the sorting condition is the x value of the coordinate of the intersection of the line segment and the upper and lower sides of the rotated rectangle. The smaller the x value, the closer it is to the center of the rectangle.

[0170] The specific sorting method depends on the position of the rectangle edge, and the purpose is to keep the lines closer to the center of the rectangle as much as possible. In this way, the protruding segments can be removed. First, because the contour is divided into four parts, the four line segments after the line segments are fused correspond to the four parts respectively, and the search is started by starting from the edge of the rotated rectangle, so it is known that the four line segments after the line segments are fused should correspond to the edge of the rotated rectangle.

[0171] See also Fig. 9 and Fig.10 , the present invention performs line segment detection on each part. Then, line segment merging is performed to merge multiple similar line segments into a longer line segment. Finally, the merged line segments are further screened, and the longer ones, those with angles close to the rectangle sides, and those closer to the center of the rectangle are retained as much as possible. At this point, only one line segment is retained for each part of the contour. In the figure, white is the contour point, blue is the line segment detection result, green is the line segment merging result, and red is the only line segment of the contour point after the final screening.

[0172] A rectangular target positioning device based on line segment fusion is a device used to implement any of the rectangular target positioning methods based on line segment fusion.

[0173] In the above-mentioned embodiments, although the present invention has been described in conjunction with the specific embodiments of the present invention, many replacements, modifications and variations of these embodiments will be apparent to those of ordinary skill in the art based on the foregoing description. The embodiments of the present invention are intended to cover all such replacements, modifications and variations falling within the broad scope of the appended claims.

[0174] The above embodiments are merely illustrative of the principles and effects of the present invention, and are not intended to limit the present invention. Anyone familiar with the art may modify or alter the above embodiments without departing from the spirit and scope of the present invention. Therefore, all equivalent modifications or alterations made by a person of ordinary skill in the art without departing from the spirit and technical concept disclosed by the present invention shall still be covered by the claims of the present invention.

Claims

1. A rectangular target positioning method based on line segment fusion, characterized by: The input image is operated by image recognition and image processing technology to obtain the positioning contour of the target, including the following steps: Convert the input image into a grayscale image, and perform dynamic threshold segmentation on the grayscale image to obtain a first binary image; Use image enhancement algorithm to enhance the grayscale image and obtain an enhanced image with equalized brightness; Performing adaptive threshold segmentation on the enhanced image to obtain a second binary image, and ANDing the first binary image with the second binary image to obtain a third binary image; Extract the contour features of the third binary image, and remove the contours that do not meet the requirements from the extracted contour features, and the rotated rectangles corresponding to the remaining contours are the preliminary positioning contours; The preliminary positioning contour is divided into four parts according to the four sides of the rotated rectangle, and line segment detection is performed on each part to obtain the corresponding line segment set; By merging line segments and removing redundant line segments, each line segment set is merged separately to obtain a unique line segment corresponding to each edge; Fit the rotating rectangle according to the intersection of the line segments to obtain the target positioning contour; The image enhancement algorithm includes an image enhancement algorithm using an improved sigmoid curve, which adjusts the pixel value of the input image through the gain function of Formula 1 to obtain an enhanced image; Among them, hdr_alpha, hdr_beta and hdr c They represent the steepness of the curve, which are parameters adjusted by the user. i represents the input grayscale value, gains i represents the gain value corresponding to the input gray value i, gain_sub represents the difference between the upper limit of the gain and the lower limit of the gain; The upper limit of the gain is calculated according to formula 2; The lower limit of the gain is calculated according to formula 3; The difference between the upper limit of the gain and the lower limit of the gain is calculated according to formula 4; gain_sub=max_gain-min_gain Formula 4.

2. The rectangular target positioning method based on line segment fusion according to claim 1 is characterized in that: The method for removing contours that do not meet the requirements from the extracted contour features includes: extracting corresponding contours according to the boundary between the foreground and the background of the third binary image, fitting a minimum circumscribed rotated rectangle according to each extracted contour, and judging whether the aspect ratio of the fitted rotated rectangle is within the preset rectangle aspect ratio range, if not, removing the contour; if so, judging whether the area of ​​the rotated rectangle is within the preset rectangle area range, if not, removing the contour, and if so, taking the contour corresponding to the rotated rectangle as the preliminary positioning contour.

3. The rectangular target positioning method based on line segment fusion according to claim 1 is characterized in that: The preliminary positioning contour is divided into four parts according to the four sides of the rotated rectangle, and the method of performing line segment detection on each part includes: Fill the preliminary positioned contour to obtain a contour binary image; The rotated rectangle circumscribing the outline is used as the ROI area for affine sampling to obtain the ROI image; Search inward from the top, bottom, left, and right boundaries of the ROI; Map the ROI position to the original image, find the non-zero pixel points in the original image, and record the non-zero pixel points as the contour points of the edges corresponding to the corresponding parts, and obtain the contour points corresponding to the four edges; For each edge, the Hough line segment detection algorithm is used to obtain the line segment set corresponding to the edge.

4. The rectangular target positioning method based on line segment fusion according to claim 1 is characterized in that: The method for merging similar line segments in each line segment set includes: Determine whether the two line segments are merged based on the angles between the two line segments and the horizontal line; Determine whether two line segments are merged based on their intersection relationship; Calculate the new angle of the fused line according to the intersection relationship of the two line segments; Calculate the points that the fused line must pass through based on the intersection relationship of the two line segments; Calculate the starting point coordinates and the end point coordinates of the fused line segment according to the intersection relationship of the two line segments; A fused line is constructed according to the new angle of the fused line and the points that the fused line must pass through, and the coordinates of the starting point and the end point are mapped to the fused line to obtain a fused line segment.

5. The rectangular target positioning method based on line segment fusion according to claim 4 is characterized in that: Methods for determining whether two line segments are merged according to the angles between the two line segments and the horizontal line include: Calculate the angle difference between the two angles between the two line segments and the horizontal line. If the angle difference exceeds the set threshold, the line segments will not be merged.

6. The rectangular target positioning method based on line segment fusion according to claim 4 is characterized in that: Methods for determining whether two line segments are merged according to the intersection relationship of the two line segments include: When two line segments are collinear, if there is no overlapping area between the two collinear line segments, it is considered that there is a gap between them. The gap between the two collinear line segments is calculated. If the gap exceeds the set value, they are not merged; When two line segments are parallel but not collinear, the distance between the two parallel line segments is calculated. If the distance exceeds the set value, they are not merged. At the same time, the two line segments are mapped on a straight line, and the interval between the two parallel line segments is calculated according to the collinearity rule. If the interval exceeds the set value, they are not merged. When two line segments have an intersection, sort the endpoints of the line segments and take the two middle points. Calculate the distance from these two points to the other line segment. If the distance exceeds the set value, they will not be merged. At the same time, calculate the interval between the intersecting line segments. If the interval exceeds the set value, they will not be merged. When two line segments are neither parallel nor intersecting, the distance between the two line segments is calculated according to the rules for line segments with intersections. If the distance exceeds the set value, they are not merged. At the same time, the two endpoints form a horizontal rectangle. If the two rectangles are in contact, it is considered that there is no gap, otherwise it is considered that there is a gap. The minimum value of the distance between the endpoints is considered to be the gap between the two line segments. If the gap exceeds the set value, they are not merged.

7. The rectangular target positioning method based on line segment fusion according to claim 4 is characterized in that: Methods for calculating the new angle of the fused line according to the intersection relationship of two line segments include: When two line segments are parallel or collinear, the new angle of the fused line is consistent with the original angle of the two line segments; When two line segments are neither parallel nor intersecting, the length ratio of the two line segments that are neither parallel nor intersecting is calculated, the length ratio is used as the weight of the angle, and the weighted sum of the angles is obtained to obtain the new angle of the fused line; When two line segments intersect, calculate the intersection of the two line segments, divide each line segment into two parts according to the intersection point, use the intersection point as the center of each line segment, and use the two parts divided by the intersection point as the radius. Set the angle to one degree to obtain two sectors, calculate the areas of the two sectors, and sum the two areas to obtain the angle weight of the corresponding line segment; normalize the two angle weights and calculate the new angle of the fused line.

8. The rectangular target positioning method based on line segment fusion according to claim 4 is characterized in that: Methods for calculating the necessary passing points of a fused line based on the intersection relationship of two line segments include: When two line segments are collinear, take any point on the two line segments as the necessary point for the fusion line; When two line segments are parallel or non-parallel and non-intersecting, the midpoints of the two line segments are calculated respectively, and the length of the line segment is used as the weight, and the weighted sum of the two midpoints is obtained to obtain the necessary passing point of the fused line; When two line segments intersect, the midpoints of the two line segments are calculated respectively, and the angle weight is used as the midpoint weight. The weighted sum is calculated to obtain the points that the fused line must pass through.

9. The rectangular target positioning method based on line segment fusion according to claim 4, characterized in that: Methods for calculating the starting point coordinates and the end point coordinates of the fused line segment according to the intersection relationship of two line segments include: Calculate the starting coordinates of the merged line segment: Take the minimum value of the x coordinates of the four endpoints as the x coordinate of the starting point of the fused line segment; Take the minimum value of the y coordinates of the four endpoints as the y coordinate of the starting point of the fused line segment; Calculate the coordinates of the end point of the merged line segment: Take the maximum value of the x coordinates of the four endpoints as the x coordinate of the end point of the fused line segment; Take the maximum value of the y coordinates of the four endpoints as the y coordinate of the end point of the merged line segment.

10. The rectangular target positioning method based on line segment fusion according to claim 1, characterized in that: Methods for removing redundant line segments include: Filter the line segments according to their rotation angles, and retain the line segments with angles close to the sides of the corresponding rotated rectangle; Calculate the lengths of the line segments that have been filtered and retained, and compare them to get the line segment with the largest length. Delete line segments whose length is less than 50% of the maximum length; The line segments that have been filtered by angle and length are filtered according to their projection positions on the rectangular area.

11. The rectangular target positioning method based on line segment fusion according to claim 10, characterized in that: Methods for filtering line segments that have been filtered by angle and length include: When the line segment corresponds to the upper side of the rotated rectangle, the mean of the y values ​​of the intersections of the line segment and the left and right sides of the rotated rectangle is calculated, and the line segment corresponding to the maximum mean is retained; When the line segment corresponds to the lower side of the rotated rectangle, calculate the mean of the y values ​​of the intersections of the line segment and the left and right sides of the rotated rectangle, and retain the line segment corresponding to the minimum mean value; When the line segment corresponds to the left side of the rotated rectangle, calculate the x value of the intersection of the line segment and the upper and lower sides of the rotated rectangle, and keep the line segment corresponding to the maximum x value; When the line segment corresponds to the right side of the rotated rectangle, calculate the x value of the intersection of the line segment and the upper and lower sides of the rotated rectangle, and retain the line segment corresponding to the minimum x value.

12. A rectangular target positioning device based on line segment fusion, characterized in that: Used to implement the rectangular target positioning method based on line segment fusion as described in any one of claims 1 to 11.

13. A rectangular target positioning method based on line segment fusion, characterized in that: The input image is operated by image recognition and image processing technology to obtain the positioning contour of the target, including the following steps: Convert the input image into a grayscale image, and perform dynamic threshold segmentation on the grayscale image to obtain a first binary image; Use image enhancement algorithm to enhance the grayscale image and obtain an enhanced image with equalized brightness; Performing adaptive threshold segmentation on the enhanced image to obtain a second binary image, and ANDing the first binary image with the second binary image to obtain a third binary image; Extract the contour features of the third binary image, and remove the contours that do not meet the requirements from the extracted contour features, and the rotated rectangles corresponding to the remaining contours are the preliminary positioning contours; The preliminary positioning contour is divided into four parts according to the four sides of the rotated rectangle, and line segment detection is performed on each part to obtain the corresponding line segment set; By merging line segments and removing redundant line segments, each line segment set is merged separately to obtain a unique line segment corresponding to each edge; Fit the rotating rectangle according to the intersection of the line segments to obtain the target positioning contour; The method for merging similar line segments in each line segment set includes: Determine whether the two line segments are merged based on the angles between the two line segments and the horizontal line; Determine whether two line segments are merged based on their intersection relationship; Calculate the new angle of the fused line according to the intersection relationship of the two line segments; Calculate the points that the fused line must pass through based on the intersection relationship of the two line segments; Calculate the starting point coordinates and the end point coordinates of the fused line segment according to the intersection relationship of the two line segments; Construct a fused line according to the new angle of the fused line and the points that the fused line must pass through, and map the starting point coordinates and the end point coordinates to the fused line to obtain a fused line segment; Methods for determining whether two line segments are merged according to the intersection relationship of the two line segments include: When two line segments are collinear, if there is no overlapping area between the two collinear line segments, it is considered that there is a gap between them. The gap between the two collinear line segments is calculated. If the gap exceeds the set value, they are not merged; When two line segments are parallel but not collinear, the distance between the two parallel line segments is calculated. If the distance exceeds the set value, they are not merged. At the same time, the two line segments are mapped on a straight line, and the interval between the two parallel line segments is calculated according to the collinearity rule. If the interval exceeds the set value, they are not merged. When two line segments have an intersection, sort the endpoints of the line segments and take the two middle points. Calculate the distance from these two points to the other line segment. If the distance exceeds the set value, they will not be merged. At the same time, calculate the interval between the intersecting line segments. If the interval exceeds the set value, they will not be merged. When two line segments are neither parallel nor intersecting, the distance between the two line segments is calculated according to the rules for line segments with intersections. If the distance exceeds the set value, they are not merged. At the same time, the two endpoints form a horizontal rectangle. If the two rectangles are in contact, it is considered that there is no gap, otherwise it is considered that there is a gap. The minimum value of the distance between the endpoints is considered to be the gap between the two line segments. If the gap exceeds the set value, they are not merged.

14. The rectangular target positioning method based on line segment fusion according to claim 13, characterized in that: The method for removing contours that do not meet the requirements from the extracted contour features includes: extracting corresponding contours according to the boundary between the foreground and the background of the third binary image, fitting a minimum circumscribed rotated rectangle according to each extracted contour, and judging whether the aspect ratio of the fitted rotated rectangle is within the preset rectangle aspect ratio range, if not, removing the contour; if so, judging whether the area of ​​the rotated rectangle is within the preset rectangle area range, if not, removing the contour, and if so, taking the contour corresponding to the rotated rectangle as the preliminary positioning contour.

15. The rectangular target positioning method based on line segment fusion according to claim 13, characterized in that: The preliminary positioning contour is divided into four parts according to the four sides of the rotated rectangle, and the method of performing line segment detection on each part includes: Fill the preliminary positioned contour to obtain a contour binary image; The rotated rectangle circumscribing the outline is used as the ROI area for affine sampling to obtain the ROI image; Search inward from the top, bottom, left, and right boundaries of the ROI; Map the ROI position to the original image, find the non-zero pixel points in the original image, and record the non-zero pixel points as the contour points of the edges corresponding to the corresponding parts, and obtain the contour points corresponding to the four edges; For each edge, the Hough line segment detection algorithm is used to obtain the line segment set corresponding to the edge.

16. The rectangular target positioning method based on line segment fusion according to claim 13, characterized in that: Methods for determining whether two line segments are merged according to the angles between the two line segments and the horizontal line include: Calculate the angle difference between the two angles between the two line segments and the horizontal line. If the angle difference exceeds the set threshold, the line segments will not be merged.

17. The rectangular target positioning method based on line segment fusion according to claim 13, characterized in that: Methods for calculating the new angle of the fused line according to the intersection relationship of two line segments include: When two line segments are parallel or collinear, the new angle of the fused line is consistent with the original angle of the two line segments; When two line segments are neither parallel nor intersecting, the length ratio of the two line segments that are neither parallel nor intersecting is calculated, the length ratio is used as the weight of the angle, and the weighted sum of the angles is obtained to obtain the new angle of the fused line; When two line segments intersect, calculate the intersection of the two line segments, divide each line segment into two parts according to the intersection point, use the intersection point as the center of each line segment, and use the two parts divided by the intersection point as the radius. Set the angle to one degree to obtain two sectors, calculate the areas of the two sectors, and sum the two areas to obtain the angle weight of the corresponding line segment; normalize the two angle weights and calculate the new angle of the fused line.

18. The rectangular target positioning method based on line segment fusion according to claim 13, characterized in that: Methods for calculating the necessary passing points of a fused line based on the intersection relationship of two line segments include: When two line segments are collinear, take any point on the two line segments as the necessary point for the fusion line; When two line segments are parallel or non-parallel and non-intersecting, the midpoints of the two line segments are calculated respectively, and the length of the line segment is used as the weight, and the weighted sum of the two midpoints is obtained to obtain the necessary passing point of the fused line; When two line segments intersect, the midpoints of the two line segments are calculated respectively, and the angle weight is used as the midpoint weight. The weighted sum is calculated to obtain the points that the fused line must pass through.

19. The rectangular target positioning method based on line segment fusion according to claim 13, characterized in that: Methods for calculating the starting point coordinates and the end point coordinates of the fused line segment according to the intersection relationship of two line segments include: Calculate the starting coordinates of the merged line segment: Take the minimum value of the x coordinates of the four endpoints as the x coordinate of the starting point of the fused line segment; Take the minimum value of the y coordinates of the four endpoints as the y coordinate of the starting point of the fused line segment; Calculate the coordinates of the end point of the merged line segment: Take the maximum value of the x coordinates of the four endpoints as the x coordinate of the end point of the fused line segment; Take the maximum value of the y coordinates of the four endpoints as the y coordinate of the end point of the merged line segment.

20. The rectangular target positioning method based on line segment fusion according to claim 13, characterized in that: Methods for removing redundant line segments include: Filter the line segments according to their rotation angles, and retain the line segments with angles close to the sides of the corresponding rotated rectangle; Calculate the lengths of the line segments that have been filtered and retained, and compare them to get the line segment with the largest length. Delete line segments whose length is less than 50% of the maximum length; The line segments that have been filtered by angle and length are filtered according to their projection positions on the rectangular area.

21. A rectangular target positioning method based on line segment fusion according to claim 20, characterized in that: Methods for filtering line segments that have been filtered by angle and length include: When the line segment corresponds to the upper side of the rotated rectangle, the mean of the y values ​​of the intersections of the line segment and the left and right sides of the rotated rectangle is calculated, and the line segment corresponding to the maximum mean is retained; When the line segment corresponds to the lower side of the rotated rectangle, calculate the mean of the y values ​​of the intersections of the line segment and the left and right sides of the rotated rectangle, and retain the line segment corresponding to the minimum mean value; When the line segment corresponds to the left side of the rotated rectangle, calculate the x value of the intersection of the line segment and the upper and lower sides of the rotated rectangle, and keep the line segment corresponding to the maximum x value; When the line segment corresponds to the right side of the rotated rectangle, calculate the x value of the intersection of the line segment and the upper and lower sides of the rotated rectangle, and retain the line segment corresponding to the minimum x value.

22. A rectangular target positioning device based on line segment fusion, characterized in that: Used to implement the rectangular target positioning method based on line segment fusion as described in any one of claims 13 to 21.

23. A rectangular target positioning method based on line segment fusion, characterized in that: The input image is operated by image recognition and image processing technology to obtain the positioning contour of the target, including the following steps: Convert the input image into a grayscale image, and perform dynamic threshold segmentation on the grayscale image to obtain a first binary image; Use image enhancement algorithm to enhance the grayscale image and obtain an enhanced image with equalized brightness; Performing adaptive threshold segmentation on the enhanced image to obtain a second binary image, and ANDing the first binary image with the second binary image to obtain a third binary image; Extract the contour features of the third binary image, and remove the contours that do not meet the requirements from the extracted contour features, and the rotated rectangles corresponding to the remaining contours are the preliminary positioning contours; The preliminary positioning contour is divided into four parts according to the four sides of the rotated rectangle, and line segment detection is performed on each part to obtain the corresponding line segment set; By merging line segments and removing redundant line segments, each line segment set is merged separately to obtain a unique line segment corresponding to each edge; Fit the rotating rectangle according to the intersection of the line segments to obtain the target positioning contour; The method for merging similar line segments in each line segment set includes: Determine whether the two line segments are merged based on the angles between the two line segments and the horizontal line; Determine whether two line segments are merged based on their intersection relationship; Calculate the new angle of the fused line according to the intersection relationship of the two line segments; Calculate the points that the fused line must pass through based on the intersection relationship of the two line segments; Calculate the starting point coordinates and the end point coordinates of the fused line segment according to the intersection relationship of the two line segments; Construct a fused line according to the new angle of the fused line and the points that the fused line must pass through, and map the starting point coordinates and the end point coordinates to the fused line to obtain a fused line segment; Methods for calculating the new angle of the fused line according to the intersection relationship of two line segments include: When two line segments are parallel or collinear, the new angle of the fused line is consistent with the original angle of the two line segments; When two line segments are neither parallel nor intersecting, the length ratio of the two line segments that are neither parallel nor intersecting is calculated, the length ratio is used as the weight of the angle, and the weighted sum of the angles is obtained to obtain the new angle of the fused line; When two line segments intersect, calculate the intersection of the two line segments, divide each line segment into two parts according to the intersection point, use the intersection point as the center of each line segment, and use the two parts divided by the intersection point as the radius. Set the angle to one degree to obtain two sectors, calculate the areas of the two sectors, and sum the two areas to obtain the angle weight of the corresponding line segment; normalize the two angle weights and calculate the new angle of the fused line.

24. A rectangular target positioning method based on line segment fusion according to claim 23, characterized in that: The method for removing contours that do not meet the requirements from the extracted contour features includes: extracting corresponding contours according to the boundary between the foreground and the background of the third binary image, fitting a minimum circumscribed rotated rectangle according to each extracted contour, and judging whether the aspect ratio of the fitted rotated rectangle is within the preset rectangle aspect ratio range, if not, removing the contour; if so, judging whether the area of ​​the rotated rectangle is within the preset rectangle area range, if not, removing the contour, and if so, taking the contour corresponding to the rotated rectangle as the preliminary positioning contour.

25. The rectangular target positioning method based on line segment fusion according to claim 23, characterized in that: The preliminary positioning contour is divided into four parts according to the four sides of the rotated rectangle, and the method of performing line segment detection on each part includes: Fill the preliminary positioned contour to obtain a contour binary image; The rotated rectangle circumscribing the outline is used as the ROI area for affine sampling to obtain the ROI image; Search inward from the top, bottom, left, and right boundaries of the ROI; Map the ROI position to the original image, find the non-zero pixel points in the original image, and record the non-zero pixel points as the contour points of the edges corresponding to the corresponding parts, and obtain the contour points corresponding to the four edges; For each edge, the Hough line segment detection algorithm is used to obtain the line segment set corresponding to the edge.

26. A rectangular target positioning method based on line segment fusion according to claim 23, characterized in that: Methods for determining whether two line segments are merged according to the angles between the two line segments and the horizontal line include: Calculate the angle difference between the two angles between the two line segments and the horizontal line. If the angle difference exceeds the set threshold, the line segments will not be merged.

27. The rectangular target positioning method based on line segment fusion according to claim 23, characterized in that: Methods for calculating the necessary passing points of a fused line based on the intersection relationship of two line segments include: When two line segments are collinear, take any point on the two line segments as the necessary point for the fusion line; When two line segments are parallel or non-parallel and non-intersecting, the midpoints of the two line segments are calculated respectively, and the length of the line segment is used as the weight, and the weighted sum of the two midpoints is obtained to obtain the necessary passing point of the fused line; When two line segments intersect, the midpoints of the two line segments are calculated respectively, and the angle weight is used as the midpoint weight. The weighted sum is calculated to obtain the points that the fused line must pass through.

28. The rectangular target positioning method based on line segment fusion according to claim 23, characterized in that: Methods for calculating the starting point coordinates and the end point coordinates of the fused line segment according to the intersection relationship of two line segments include: Calculate the starting coordinates of the merged line segment: Take the minimum value of the x coordinates of the four endpoints as the x coordinate of the starting point of the fused line segment; Take the minimum value of the y coordinates of the four endpoints as the y coordinate of the starting point of the fused line segment; Calculate the coordinates of the end point of the merged line segment: Take the maximum value of the x coordinates of the four endpoints as the x coordinate of the end point of the fused line segment; Take the maximum value of the y coordinates of the four endpoints as the y coordinate of the end point of the merged line segment.

29. The rectangular target positioning method based on line segment fusion according to claim 23, characterized in that: Methods for removing redundant line segments include: Filter the line segments according to their rotation angles, and retain the line segments with angles close to the sides of the corresponding rotated rectangle; Calculate the lengths of the line segments that have been filtered and retained, and compare them to get the line segment with the largest length. Delete line segments whose length is less than 50% of the maximum length; The line segments that have been filtered by angle and length are filtered according to their projection positions on the rectangular area.

30. The rectangular target positioning method based on line segment fusion according to claim 29, characterized in that: Methods for filtering line segments that have been filtered by angle and length include: When the line segment corresponds to the upper side of the rotated rectangle, the mean of the y values ​​of the intersections of the line segment and the left and right sides of the rotated rectangle is calculated, and the line segment corresponding to the maximum mean is retained; When the line segment corresponds to the lower side of the rotated rectangle, calculate the mean of the y values ​​of the intersections of the line segment and the left and right sides of the rotated rectangle, and retain the line segment corresponding to the minimum mean value; When the line segment corresponds to the left side of the rotated rectangle, calculate the x value of the intersection of the line segment and the upper and lower sides of the rotated rectangle, and keep the line segment corresponding to the maximum x value; When the line segment corresponds to the right side of the rotated rectangle, calculate the x value of the intersection of the line segment and the upper and lower sides of the rotated rectangle, and retain the line segment corresponding to the minimum x value.

31. A rectangular target positioning device based on line segment fusion, characterized in that: Used to implement the rectangular target positioning method based on line segment fusion as described in any one of claims 23 to 30.

32. A rectangular target positioning method based on line segment fusion, characterized in that: The input image is operated by image recognition and image processing technology to obtain the positioning contour of the target, including the following steps: Convert the input image into a grayscale image, and perform dynamic threshold segmentation on the grayscale image to obtain a first binary image; Use image enhancement algorithm to enhance the grayscale image and obtain an enhanced image with equalized brightness; Performing adaptive threshold segmentation on the enhanced image to obtain a second binary image, and ANDing the first binary image with the second binary image to obtain a third binary image; Extract the contour features of the third binary image, and remove the contours that do not meet the requirements from the extracted contour features, and the rotated rectangles corresponding to the remaining contours are the preliminary positioning contours; The preliminary positioning contour is divided into four parts according to the four sides of the rotated rectangle, and line segment detection is performed on each part to obtain the corresponding line segment set; By merging line segments and removing redundant line segments, each line segment set is merged separately to obtain a unique line segment corresponding to each edge; Fit the rotating rectangle according to the intersection of the line segments to obtain the target positioning contour; The method for merging similar line segments in each line segment set includes: Determine whether the two line segments are merged based on the angles between the two line segments and the horizontal line; Determine whether two line segments are merged based on their intersection relationship; Calculate the new angle of the fused line according to the intersection relationship of the two line segments; Calculate the points that the fused line must pass through based on the intersection relationship of the two line segments; Calculate the starting point coordinates and the end point coordinates of the fused line segment according to the intersection relationship of the two line segments; Construct a fused line according to the new angle of the fused line and the points that the fused line must pass through, and map the starting point coordinates and the end point coordinates to the fused line to obtain a fused line segment; Methods for calculating the necessary passing points of a fused line based on the intersection relationship of two line segments include: When two line segments are collinear, take any point on the two line segments as the necessary point for the fusion line; When two line segments are parallel or non-parallel and non-intersecting, the midpoints of the two line segments are calculated respectively, and the length of the line segment is used as the weight, and the weighted sum of the two midpoints is obtained to obtain the necessary passing point of the fused line; When two line segments intersect, calculate the intersection of the two line segments, divide each line segment into two parts according to the intersection point, use the intersection point as the center of each line segment, and use the two parts divided by the intersection point as the radius. Set the angle to one degree to obtain two sectors, calculate the areas of the two sectors, sum the two areas to obtain the angle weight of the corresponding line segment, calculate the midpoints of the two line segments respectively, use the angle weight as the midpoint weight, and perform weighted sum calculation to obtain the necessary points for the fused straight line.

33. A rectangular target positioning method based on line segment fusion according to claim 32, characterized in that: The method for removing contours that do not meet the requirements from the extracted contour features includes: extracting corresponding contours according to the boundary between the foreground and the background of the third binary image, fitting a minimum circumscribed rotated rectangle according to each extracted contour, and judging whether the aspect ratio of the fitted rotated rectangle is within the preset rectangle aspect ratio range, if not, removing the contour; if so, judging whether the area of ​​the rotated rectangle is within the preset rectangle area range, if not, removing the contour, and if so, taking the contour corresponding to the rotated rectangle as the preliminary positioning contour.

34. A rectangular target positioning method based on line segment fusion according to claim 32, characterized in that: The preliminary positioning contour is divided into four parts according to the four sides of the rotated rectangle, and the method of performing line segment detection on each part includes: Fill the preliminary positioned contour to obtain a contour binary image; The rotated rectangle circumscribing the outline is used as the ROI area for affine sampling to obtain the ROI image; Search inward from the top, bottom, left, and right boundaries of the ROI; Map the ROI position to the original image, find the non-zero pixel points in the original image, and record the non-zero pixel points as the contour points of the edges corresponding to the corresponding parts, and obtain the contour points corresponding to the four edges; For each edge, the Hough line segment detection algorithm is used to obtain the line segment set corresponding to the edge.

35. A rectangular target positioning method based on line segment fusion according to claim 32, characterized in that: Methods for determining whether two line segments are merged according to the angles between the two line segments and the horizontal line include: Calculate the angle difference between the two angles between the two line segments and the horizontal line. If the angle difference exceeds the set threshold, the line segments will not be merged.

36. A rectangular target positioning method based on line segment fusion according to claim 32, characterized in that: Methods for calculating the starting point coordinates and the end point coordinates of the fused line segment according to the intersection relationship of two line segments include: Calculate the starting coordinates of the merged line segment: Take the minimum value of the x coordinates of the four endpoints as the x coordinate of the starting point of the fused line segment; Take the minimum value of the y coordinates of the four endpoints as the y coordinate of the starting point of the fused line segment; Calculate the coordinates of the end point of the merged line segment: Take the maximum value of the x coordinates of the four endpoints as the x coordinate of the end point of the fused line segment; Take the maximum value of the y coordinates of the four endpoints as the y coordinate of the end point of the merged line segment.

37. A rectangular target positioning method based on line segment fusion according to claim 32, characterized in that: Methods for removing redundant line segments include: Filter the line segments according to their rotation angles, and retain the line segments with angles close to the sides of the corresponding rotated rectangle; Calculate the lengths of the line segments that have been filtered and retained, and compare them to get the line segment with the largest length. Delete line segments whose length is less than 50% of the maximum length; The line segments that have been filtered by angle and length are filtered according to their projection positions on the rectangular area.

38. A rectangular target positioning method based on line segment fusion according to claim 37, characterized in that: Methods for filtering line segments that have been filtered by angle and length include: When the line segment corresponds to the upper side of the rotated rectangle, the mean of the y values ​​of the intersections of the line segment and the left and right sides of the rotated rectangle is calculated, and the line segment corresponding to the maximum mean is retained; When the line segment corresponds to the lower side of the rotated rectangle, calculate the mean of the y values ​​of the intersections of the line segment and the left and right sides of the rotated rectangle, and retain the line segment corresponding to the minimum mean value; When the line segment corresponds to the left side of the rotated rectangle, calculate the x value of the intersection of the line segment and the upper and lower sides of the rotated rectangle, and keep the line segment corresponding to the maximum x value; When the line segment corresponds to the right side of the rotated rectangle, calculate the x value of the intersection of the line segment and the upper and lower sides of the rotated rectangle, and retain the line segment corresponding to the minimum x value.

39. A rectangular target positioning device based on line segment fusion, characterized in that: Used to implement the rectangular target positioning method based on line segment fusion as described in any one of claims 32 to 38.

Images