A method, device, equipment, medium and product for identifying grid line intersections

Abstract

The present application discloses a method, device, equipment, medium and product for identifying grid line intersections, and relates to the field of computer vision or machine vision technology. The method comprises: obtaining a grid line image and graying it to obtain a grayed image; using a region of interest to intercept the grayed image to obtain a region of interest image; normalizing the region of interest image to obtain a normalized image; obtaining a grid line skeleton image according to the normalized image; obtaining a grid line skeleton according to the grid line skeleton image to construct a skeleton edge segment set, and dividing the skeleton edge segment set into a horizontal edge segment set and a vertical edge segment set; using the points common to the horizontal edge segment set and the vertical edge segment set as grid line intersections. The present application extracts the coordinates of grid line intersections through relevant image processing operations, and can accurately identify distorted arc grids.

Description

Technical Field

[0001] The present application relates to the field of computer vision or machine vision technology, and in particular to a method, device, equipment, medium and product for machine vision grid line intersection recognition. Background Art

[0002] With the rapid development of industrial automation and intelligent manufacturing, machine vision technology has become an indispensable part of modern industrial manufacturing. Machine vision captures images through cameras or other visual sensors, and uses computer algorithms to analyze and process these images to achieve functions such as automated detection, measurement, positioning and identification. In the field of industrial manufacturing, machine vision technology is widely used in quality inspection, assembly positioning, product identification and other links, significantly improving production efficiency and product quality.

[0003] In machine vision applications, camera calibration and visual positioning are two core technologies. Camera calibration uses a series of known calibration objects (such as chessboards, grid plates, etc.) to accurately determine the internal and external parameters of the camera in order to convert points in the image coordinate system to the actual physical coordinate system. Visual positioning uses feature points in the image (such as corner points, intersection points, etc.) to determine the position and direction of the target object in three-dimensional space.

[0004] Traditional grid intersection recognition methods are mainly based on straight grids, and are implemented by detecting the intersections of straight grids. However, in practical applications, due to various factors in the manufacturing process (such as camera distortion, etc.), grid lines often become distorted, forming arc grids. This distorted arc grid makes traditional straight grid intersection recognition methods difficult to apply, resulting in reduced accuracy of camera calibration and visual positioning. Summary of the invention

[0005] The purpose of the present application is to provide a grid line intersection recognition method, device, equipment, medium and product, which can effectively extract the intersection coordinates of grid lines from a distorted arc grid.

[0006] To achieve the above objectives, this application provides the following solutions:

[0007] In a first aspect, the present application provides a method for identifying grid line intersections, comprising:

[0008] Acquire the grid line image and perform grayscale processing to obtain a grayscale image;

[0009] Using the region of interest to intercept the grayscale image to obtain a region of interest image;

[0010] Normalizing the image of the region of interest to obtain a normalized image;

[0011] According to the normalized image, a grid line skeleton image is obtained;

[0012] Acquire a grid line skeleton according to the grid line skeleton image to construct a skeleton edge segment set, and divide the skeleton edge segment set into a horizontal edge segment set and a vertical edge segment set;

[0013] The points common to the horizontal edge segment set and the vertical edge segment set are taken as grid line intersection points.

[0014] Optionally, the step of using the region of interest to intercept the grayscale image to obtain the region of interest image specifically includes:

[0015] Binarize the grayscale image to obtain a binary image;

[0016] Perform morphological closing operation on the binary image to obtain a smooth image;

[0017] On the smooth image, identifying and calculating the connected domain of the smooth image, selecting the one with the largest area as the basic region of the connected domain of the smooth image, and taking the area enclosed by the minimum circumscribed rectangle of the basic region as the region of interest;

[0018] The grayscale image is intercepted using the region of interest to obtain the region of interest image.

[0019] Optionally, obtaining a grid line skeleton image according to the normalized image specifically includes:

[0020] Performing Gaussian blur processing on the normalized image to obtain a Gaussian blurred image, and performing histogram adaptive equalization operation on the Gaussian blurred image to obtain a histogram image;

[0021] Perform adaptive threshold segmentation on the histogram image to obtain an adaptive mask;

[0022] The normalized image is subjected to canny edge extraction by using a canny edge detection algorithm to obtain an edge mask image, and the edge mask image is subjected to a dilation operation to obtain an edge mask;

[0023] Perform pixel AND operation on the adaptive mask and the edge mask to obtain a skeleton mask;

[0024] The skeleton mask connected domains are identified and calculated on the skeleton mask, and the skeleton mask connected domains smaller than an area threshold are eliminated to obtain a grid line skeleton image.

[0025] Optionally, acquiring a grid line skeleton according to the grid line skeleton image to construct a skeleton edge segment set, and dividing the skeleton edge segment set into a horizontal edge segment set and a vertical edge segment set, specifically includes:

[0026] Calculating the grid line skeleton image connected domain of the grid line skeleton image, determining whether an intersection is encountered during the traversal of the current grid line skeleton image connected domain, and obtaining a determination result;

[0027] If the judgment result is no, the first point and the last point traversed on the connected domain of the current grid line skeleton image are connected to form an edge segment, which is added to the skeleton edge segment set;

[0028] If the judgment result is yes, the current grid line skeleton image connected domain is disconnected at the intersection, and the first point traversed on the current grid line skeleton image connected domain is connected to the intersection to form an edge segment, which is added to the skeleton edge segment set;

[0029] Traverse the next grid line skeleton image connected domain of the current grid line skeleton image connected domain, return to the step of "calculating the grid line skeleton image connected domain of the grid line skeleton image, determining whether an intersection is encountered during the traversal of the current grid line skeleton image connected domain, and obtaining a determination result", until all grid line skeleton image connected domains are traversed;

[0030] Traverse the skeleton edge segment set, calculate the straight line slope and straight line deviation of each edge segment respectively, and eliminate the edge segments whose straight line deviation exceeds the deviation threshold; calculate the edge segment angle according to the straight line slope, and classify the edge segments whose edge segment angles are in the first angle interval or the second angle interval into the horizontal edge segment set and the vertical edge segment set respectively, and eliminate the remaining edge segments.

[0031] Optionally, the taking the points common to the horizontal edge segment set and the vertical edge segment set as grid line intersections may also include:

[0032] The head endpoint of the current horizontal edge segment in the horizontal edge segment set is taken out, and the first distance from the head endpoints of all other horizontal edge segments except the current horizontal edge segment to the head endpoint of the current horizontal edge segment and the second distance from the tail endpoints of all other horizontal edge segments except the current horizontal edge segment to the head endpoint of the current horizontal edge segment are calculated respectively, and the smaller value of the first distance and the second distance is compared with the preset threshold value to obtain a judgment result; if the judgment result is less than, the current horizontal edge segment is added to the edge segment set to be selected; if the judgment result is not less than, the head endpoint of the next horizontal edge segment of the current horizontal edge segment in the horizontal edge segment set is taken out, and the above steps are repeated until all horizontal edge segments in the horizontal edge segment set are traversed;

[0033] Traverse the set of edge segments to be selected, and remove the edge segments to be selected that are smaller than the horizontal threshold;

[0034] Calculate the distance between each candidate edge segment in the eliminated candidate edge segment set and the current horizontal edge segment, and select the candidate edge segment corresponding to the minimum distance as the best edge segment of the current horizontal edge segment;

[0035] Determine whether there is an inclusion relationship between the current horizontal edge segment and the best edge segment. If there is no inclusion relationship, connect the current horizontal edge segment and the best edge segment to obtain a connected horizontal edge segment, and mark the connected best edge segment; if there is an inclusion relationship, select the next horizontal edge segment of the current horizontal edge segment, and return to the step of "respectively calculating the distance between each candidate edge segment in the eliminated candidate edge segment set and the current horizontal edge segment, and selecting the candidate edge segment corresponding to the minimum distance as the best edge segment of the current horizontal edge segment" until there is no connectable horizontal edge segment in the eliminated candidate edge segment set;

[0036] Remove connected horizontal edge segments whose length after being connected is less than the length threshold.

[0037] Optionally, the method further comprises taking the points common to the horizontal edge segment set and the vertical edge segment set as grid line intersections:

[0038] Calculate the grayscale centroid coordinates of the c×c neighborhood of each grid line intersection, and use the grayscale centroid coordinates as the updated grid line intersection coordinates. The calculation formula of the grayscale centroid coordinates is:

[0039]

[0040]

[0041] Among them, x c is the horizontal coordinate of the updated grid line intersection coordinates, y c is the ordinate of the updated grid line intersection coordinates, x i is the horizontal coordinate of the i-th pixel in the c×c neighborhood, y i is the ordinate of the i-th pixel in the c×c neighborhood, (i=0, 1, 2, 3…, c×c-1), v i represents the gray value of the i-th pixel, x b is the pixel horizontal coordinate of the grid line intersection, y b The pixel ordinate of the grid line intersection.

[0042] In a second aspect, the present application provides a grid line intersection recognition device, comprising:

[0043] A grayscale image acquisition module acquires a grid line image and performs grayscale processing to obtain a grayscale image;

[0044] An area of ​​interest image acquisition module uses the area of ​​interest to intercept the grayscale image to obtain an area of ​​interest image;

[0045] A normalized image acquisition module performs normalization processing on the image of the region of interest to obtain a normalized image;

[0046] A grid line skeleton image acquisition module obtains a grid line skeleton image according to the normalized image;

[0047] A skeleton edge segment extraction module obtains a grid line skeleton according to the grid line skeleton image to construct a skeleton edge segment set, and divides the skeleton edge segment set into a horizontal edge segment set and a vertical edge segment set;

[0048] The grid intersection extraction module takes the points common to the horizontal edge segment set and the vertical edge segment set as grid line intersections.

[0049] In a third aspect, the present application provides a computer device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of any of the above-described grid line intersection identification methods.

[0050] In a fourth aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of any of the above-described grid line intersection identification methods.

[0051] In a fifth aspect, the present application provides a computer program product, including a computer program, which, when executed by a processor, implements the steps of any of the above-mentioned grid line intersection identification methods.

[0052] According to the specific embodiments provided in this application, this application discloses the following technical effects:

[0053] The present application provides a method, device, equipment, medium and product for identifying grid line intersections. By obtaining a grayscale image of a grid line image, intercepting the region of interest of the grayscale image to obtain a region of interest image, and obtaining a normalized image of the region of interest image, the grid line features are effectively highlighted and background interference is suppressed. A grid line skeleton image is further extracted based on the normalized image, and its essential structural information can be retained. A skeleton edge segment set is constructed based on the grid line skeleton image, which is divided into two types of sets: a horizontal edge segment set and a vertical edge segment set, and the points common to the two types of sets are located as grid line intersections. In order to further improve the accuracy of intersection positioning, the method also calculates the grayscale centroid of the intersection and uses it as the final intersection coordinate. The present application not only overcomes the limitations of traditional methods in processing distorted arc grids, but also significantly improves the accuracy and efficiency of intersection recognition, and successfully achieves the goal of efficiently and accurately extracting grid line intersection coordinates from complex grid images. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0055] Figure 1 This is an application environment diagram of a grid line intersection recognition method in one embodiment of the present application;

[0056] Figure 2 A schematic diagram of a flow chart of a method for identifying grid line intersections provided in one embodiment of the present application;

[0057] Figure 3 A schematic diagram of a grid line image provided by an embodiment of the present application;

[0058] Figure 4 A schematic diagram of a grid line skeleton image provided by an embodiment of the present application;

[0059] Figure 5 A horizontal edge segment division effect diagram provided by an embodiment of the present application;

[0060] Figure 6 A vertical edge segment division effect diagram provided by an embodiment of the present application;

[0061] Figure 7 This is a diagram showing the effect of removing the horizontal edge segment connection provided by an embodiment of the present application;

[0062] Figure 8 This is a rendering of the effect after vertical edge segment connections are removed according to an embodiment of the present application;

[0063] Fig. 9 The final grid line intersection extraction effect diagram provided in an embodiment of the present application;

[0064] Fig.10 A schematic diagram of functional modules of a grid line intersection recognition device provided in one embodiment of the present application;

[0065] Fig.11 A schematic diagram of the structure of a computer device provided in one embodiment of the present application. DETAILED DESCRIPTION

[0066] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0067] In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, the present application is further described in detail below in conjunction with the accompanying drawings and specific implementation methods.

[0068] The grid line intersection recognition method provided in the embodiment of the present application can be applied to Figure 1 In the application environment shown. Among them, the terminal 102 communicates with the server 104 through the network. The data storage system can store the data that the server 104 needs to process. The data storage system can be set up separately, integrated on the server 104, or placed on the cloud or other servers. The terminal 102 can send a grid line image to the server 104. After the server 104 receives the grid line image, for the grid line image, the server 104 grayscales the grid line image to obtain a grayscale image; uses the region of interest to intercept the grayscale image to obtain the region of interest image; normalizes the region of interest image to obtain a normalized image; obtains a grid line skeleton image based on the normalized image; obtains a grid line skeleton based on the grid line skeleton image to construct a skeleton edge segment set, and divides the skeleton edge segment set into a horizontal edge segment set and a vertical edge segment set; and uses the points common to the horizontal edge segment set and the vertical edge segment set as the grid line intersection. The server 104 can feedback the obtained grid line intersection to the terminal 102. In addition, in some embodiments, the grid line intersection recognition method can also be implemented by the server 104 or the terminal 102 alone, such as the terminal 102 can directly process the grid line image, or the server 104 can obtain the grid line image from the data storage system and process the grid line image, wherein the obtained grid line image is as follows: Figure 3 shown.

[0069] In an exemplary embodiment, Figure 2 As shown, a grid line intersection recognition method is provided, which is executed by a computer device, and can be executed by a computer device such as a terminal or a server alone, or can be executed by a terminal and a server together. In the embodiment of the present application, the method is applied to Figure 1 The server 104 in the example is used as an example to illustrate the method, which includes the following steps 201 to 206. Among them:

[0070] Step 201, obtain a grid line image and perform grayscale processing to obtain a grayscale image.

[0071] Step 202: Use the region of interest to intercept the grayscale image to obtain a region of interest image.

[0072] Step 203: normalize the image of the region of interest to obtain a normalized image.

[0073] Step 204: Obtain a grid line skeleton image based on the normalized image.

[0074] Step 205 , obtaining a grid line skeleton according to the grid line skeleton image to construct a skeleton edge segment set, and dividing the skeleton edge segment set into a horizontal edge segment set and a vertical edge segment set.

[0075] Step 206: Use the points common to the horizontal edge segment set and the vertical edge segment set as grid line intersection points.

[0076] By implementing the above steps 201 to 206, the intersection points of the mesh with distorted arcs can be identified.

[0077] Wherein, in implementing this implementation mode, step 202 specifically includes:

[0078] The grayscale image is binarized to obtain a binary image T0.

[0079] The binarized image T0 is binarized using the threshold t0, and a morphological closing operation is performed on the binarized image T0 to obtain a smoothed image T1.

[0080] On the smooth image T1, the connected domain of the smooth image is identified and calculated, the one with the largest area is selected as the basic region of the connected domain of the smooth image, and the area enclosed by the minimum circumscribed rectangle of the basic region is selected as the region of interest.

[0081] The grayscale image is intercepted using the region of interest to obtain the region of interest image I0.

[0082] Step 203 specifically includes:

[0083] Each pixel of the region of interest image I0 is traversed to perform pixel threshold restriction, that is, the pixels in the region of interest image I0 whose grayscale values ​​are less than the threshold t1 are set to t1, and the pixels in the region of interest image I0 whose grayscale values ​​are greater than the threshold t2 are set to t2.

[0084] Each pixel of the region of interest image I0 is traversed again, and a normalization operation is performed on each pixel to obtain a normalized image I1.

[0085] The normalized calculation formula is:

[0086] (1)

[0087] Among them, m0 is the grayscale value of the pixel before normalization, and m is the pixel value of the pixel after normalization.

[0088] Step 204 specifically includes:

[0089] The normalized image I1 is subjected to Gaussian blur processing to obtain a Gaussian blurred image T2, and the Gaussian blurred image T2 is subjected to histogram adaptive equalization operation to obtain a histogram image T3.

[0090] Perform adaptive threshold segmentation on the histogram image T3 to obtain an adaptive mask M0.

[0091] The normalized image I1 is subjected to canny edge extraction by using a canny edge detection algorithm to obtain an edge mask image M1, and the edge mask image M1 is subjected to a dilation operation to obtain an edge mask M2.

[0092] A pixel AND operation is performed on the adaptive mask M0 and the edge mask M2 to obtain a skeleton mask M3.

[0093] On the skeleton mask M3, the skeleton mask connected domain is identified and calculated, and the skeleton mask connected domain with an area smaller than the area threshold t3 is eliminated to obtain a grid line skeleton image I2. The schematic diagram of the grid line skeleton image I2 is shown in FIG. Figure 4 shown.

[0094] In another exemplary embodiment of the present application, in order to construct a skeleton edge segment set according to the grid line skeleton image I2 and divide the skeleton edge segment set into a horizontal edge segment set and a vertical edge segment set, the above step 205 is replaced by the following steps 301 to 305:

[0095] Step 301 , calculating the grid line skeleton image connected domain of the grid line skeleton image I2 , determining whether an intersection is encountered during the traversal of the current grid line skeleton image connected domain, and obtaining a determination result.

[0096] Step 302: If the judgment result is no, the first point and the last point traversed on the connected domain of the current grid line skeleton image are connected to form an edge segment, which is added to the skeleton edge segment set.

[0097] Step 303, if the judgment result is yes, the current grid line skeleton image connected domain is disconnected at the intersection, and the first point traversed on the current grid line skeleton image connected domain is connected to the intersection to form an edge segment, which is added to the skeleton edge segment set.

[0098] Step 304, traverse the next grid line skeleton image connected domain of the current grid line skeleton image connected domain, and return to step 301 until all grid line skeleton image connected domains are traversed.

[0099] Step 305, traverse the skeleton edge segment set, calculate the straight line slope and straight line deviation of each edge segment respectively, and eliminate the edge segments whose straight line deviation exceeds the deviation threshold; calculate the edge segment angle according to the straight line slope, and classify the edge segments whose edge segment angles are in the first angle interval or the second angle interval into the horizontal edge segment set and the vertical edge segment set respectively, and eliminate the remaining edge segments.

[0100] In this example, each skeleton edge segment S in the skeleton edge segment set S is traversed j (j=0, 1, 2…n-1, n is the size of the set S), take out the two coordinate points at the head and tail of the skeleton edge segment respectively, and calculate the general straight line equation formed by connecting these two points. The general straight line equation is:

[0101] (2)

[0102] Among them, k is the slope, b is the intercept, A is the horizontal coordinate of any point on the straight line, and B is the vertical coordinate of any point on the straight line.

[0103] At the same time, calculate the skeleton edge segment S j The straight line deviation degree, the calculation method of the deviation degree e is:

[0104] (3)

[0105] Where q is the skeleton edge segment S j The number of coordinate points included, A g , B g The skeleton edge segment S j The horizontal and vertical coordinates of the g-th point in , g=0, 1, 2…q-1.

[0106] The skeleton edge segments with straight line deviation e greater than threshold t4 are eliminated, and the angle a of each skeleton edge segment is calculated according to the slope k corresponding to each skeleton edge segment. The skeleton edge segments corresponding to the angle a between angle threshold t5 and angle threshold t6 are added to the horizontal edge segment set H. The effect diagram of horizontal edge segment division is shown in Figure 5 As shown; the skeleton edge segment corresponding to the angle a between the angle threshold t7 and the angle threshold t8 is added to the vertical edge segment set V, and the vertical edge segment division effect diagram is shown in Figure 6 As shown; and the skeleton edge segments at other angles are removed.

[0107] In another exemplary embodiment of the present application, in order to use the points common to the horizontal edge segment set and the vertical edge segment set as grid line intersections, before the above step 205, steps 401 to 405 are further included, wherein:

[0108] Step 401, take out the head endpoint of the current horizontal edge segment in the horizontal edge segment set, calculate the first distance from the head endpoints of all other horizontal edge segments except the current horizontal edge segment to the head endpoint of the current horizontal edge segment, and the second distance from the tail endpoints of all other horizontal edge segments except the current horizontal edge segment to the head endpoint of the current horizontal edge segment, take the smaller value of the first distance and the second distance and compare it with a preset threshold to obtain a judgment result; if the judgment result is less than, put the current horizontal edge segment into the edge segment set to be selected; if the judgment result is not less than, take out the head endpoint of the next horizontal edge segment of the current horizontal edge segment in the horizontal edge segment set, and repeat the above steps until all horizontal edge segments in the horizontal edge segment set are traversed.

[0109] Step 402: traverse the set of edge segments to be selected, and remove edge segments to be selected that are smaller than the horizontal threshold.

[0110] Step 403 , respectively calculate the distance between each candidate edge segment in the eliminated candidate edge segment set and the current horizontal edge segment, and select the candidate edge segment corresponding to the minimum distance as the best edge segment for the current horizontal edge segment.

[0111] Step 404, determine whether there is an inclusion relationship between the current horizontal edge segment and the best edge segment. If there is no inclusion relationship, connect the current horizontal edge segment and the best edge segment to obtain a connected horizontal edge segment, and mark the connected best edge segment; if there is an inclusion relationship, select the next horizontal edge segment of the current horizontal edge segment, and return to step 403 until there is no connected horizontal edge segment in the set of edge segments to be selected after elimination.

[0112] Step 405: delete the connected horizontal edge segments whose connected length is less than the length threshold.

[0113] Steps 401-405 further specifically include:

[0114] (1) Calculate the parameters (slope, intercept and angle) of the head and tail of each horizontal edge segment. For the head of each horizontal edge segment, first take the head endpoint of the horizontal edge segment as the first point, and the wth point from the head to the tail as the first reference point. Use these two points to construct the general straight line equation according to formula (2), and calculate the angle a corresponding to the slope k. Record the slope k, intercept b and angle a at this time. Similarly, for the tail of each horizontal edge segment, take the tail endpoint as the first point, and the wth point from the tail to the head as the second reference point. Calculate the slope k, intercept b and angle a and record them. Where w∈[20,50], because the points are pixel-level, the closer the distance between the two points, the greater the error. Therefore, the error can be reduced by selecting the wth point from the head to the tail and the wth point from the tail to the head as the first reference point and the second reference point, respectively.

[0115] (2) Remove the horizontal edge segment H f The head endpoint p0 of the H is traversed through other undestroyed horizontal edge segments. For each other horizontal edge segment H r , calculate H r The distance d0 from the head endpoint to p0, and H r The distance d1 from the end point of the tail to p0, if the smaller value of d0 and d1 is less than the preset threshold t9, then H r Join H f The corresponding first to-be-selected edge segment set F1.

[0116] (3) Remove H f The tail endpoint of the horizontal edge segment is the same as operation (2). The distance between the two endpoints of the other undestroyed horizontal edge segments is calculated to obtain H f The corresponding second edge segment set F2 to be selected.

[0117] (4) Eliminate the candidate edge segment sets F1 and F2 by point-line distance. Traverse the first candidate edge segment set F1, and for each first candidate edge segment F in F1 u , calculate H by the straight line parameters obtained by operation (1) f The head end point to F u The distance from the point to the straight line is greater than the horizontal threshold t 10 , then F u Eliminate from the first set of edge segments to be selected F1; and so on, use the same method to screen and eliminate from the second set of edge segments to be selected F2.

[0118] (5) Eliminate the candidate edge sets F1 and F2 after operation (4) by angle. Traverse the first candidate edge segment set F1, for each horizontal edge segment F in F1 o , find F in operation (1) o and H f The corresponding angles are calculated, and the absolute difference s between the two angles is calculated. If s is greater than the angle threshold t 11 Then F o Remove them from F1; and so on, using the same method, screen and remove the second candidate edge set F2.

[0119] Points with large distances and angle deviations are generally not grid lines. Therefore, some messy noise lines are removed through operations (4) and (5).

[0120] (6) Select the first candidate edge segment set F1 and the second candidate edge segment set F2 after operation (5), the distance H f The nearest edge segment is taken as the best edge segment H best .

[0121] (7) Calculate the edge segment inclusion relationship and take H f The head endpoint h0 and the tail endpoint h1, as well as the optimal edge segment H best The head endpoint h2 contains the coordinates C h The calculation method is:

[0122] (4)

[0123] Among them, X0 is the edge segment H f The horizontal coordinate of the head endpoint h0, Y0 is the edge segment H f The ordinate of the head endpoint h0, X1 is the edge segment H f The horizontal coordinate of the tail end point h1, Y1 is the edge segment H f The ordinate of the tail endpoint h1, X2 is the optimal edge segment H best The horizontal coordinate of the head endpoint h2, Y2 is the optimal edge segment H best The vertical coordinate of the head endpoint h2.

[0124] If the coordinate C h At the threshold coordinate t x1 and t x2 If the edge segments are between , then it is considered that the edge segments contain each other, and H is selected. f The next horizontal edge segment, return to operation (6); if the coordinate C h Not at the threshold coordinate t x1 and t x2 If there is no containment relationship between the edge segments, the following edge segment connection operation (8) is performed.

[0125] (8) Connect the horizontal edge segment H f and H best , get the connected horizontal edge segment, calculate the points on the connected horizontal edge segment through Bresenham's line algorithm, and add these points to the horizontal edge segment H in the order of head to tail f In the meantime, H best The points are also added to H f In the best The parameters of the synchronous update steps 401-402 are H f The calculated straight line parameters and H best The corresponding horizontal edge segment is marked for destruction.

[0126] (9) If the horizontal edge segment H f has been marked for destruction, then f Instead of performing the following operations, the current loop traversal is jumped out and the traversal of other horizontal edge segments is performed.

[0127] (10) Repeat the above operations (1) to (9) until there are no two horizontal edge segments that can be connected and meet the threshold condition.

[0128] Similarly, follow the above operations (1)-(10) to connect the vertical edge segments with similar trend directions.

[0129] After deleting the connection, the length is less than the length threshold t 11 The connected horizontal edge segments and the connected vertical edge segments are shown in the following figure. Figure 7 As shown in the figure, the effect after the connected vertical edge segments are removed is as follows Figure 8 shown.

[0130] In another exemplary embodiment of the present application, the points common to the horizontal edge segment set and the vertical edge segment set are used as grid line intersections, further comprising:

[0131] According to the distance threshold d2, it is determined whether there are points with similar distances in the horizontal edge segment set H and the vertical edge segment set V. If so, the x-coordinates and y-coordinates of multiple points are averaged, and the average x-coordinates and y-coordinates of multiple points are taken as the intersection points of the grid lines.

[0132] Calculate the grayscale centroid coordinates of the c×c neighborhood of each grid line intersection, and use the grayscale centroid coordinates as the updated grid line intersection coordinates. The updated grid line intersection coordinate extraction effect diagram is as follows: Fig. 9 As shown in the figure, the calculation formula of the grayscale centroid coordinates is:

[0133] (5)

[0134] (6)

[0135] Among them, x c is the horizontal coordinate of the updated grid line intersection coordinates, y c is the ordinate of the updated grid line intersection coordinates, x i is the horizontal coordinate of the i-th pixel in the c×c neighborhood, y i is the ordinate of the i-th pixel in the c×c neighborhood, (i=0, 1, 2, 3…, c×c-1), v i represents the gray value of the i-th pixel, x b is the pixel horizontal coordinate of the grid line intersection, y b The pixel ordinate of the grid line intersection.

[0136] The present application also provides an application scenario, which applies the above-mentioned grid line intersection recognition method. Specifically: The grid line intersection recognition method provided in this embodiment can be applied in industrial automation detection and quality control scenarios. This scenario usually includes multiple key links such as raw material preparation, production and processing, quality inspection, and finished product packaging and distribution. In the production process, the product enters the production line from the raw material preparation link, and undergoes a series of precise processing and manufacturing steps to form a semi-finished product or finished product. Subsequently, these products enter the quality inspection link, and the grid line image on its surface is captured by a high-definition camera. The grid line intersection recognition method provided in this embodiment can automatically process the captured grid line image, including a series of steps such as image pre-processing, normalization processing, adaptive mask generation, and grid line skeleton extraction, and finally effectively extracts accurate grid line intersection coordinates from the distorted arc grid. These intersection coordinates not only reflect the geometric size and position accuracy of the product, but also have important significance for evaluating product quality, detecting potential defects, and optimizing production processes.

[0137] In the specific implementation process, this method is based on advanced image processing algorithms and machine vision technology, which can intelligently identify and process complex distortions in images to ensure the accuracy and stability of intersection recognition. At the same time, this method also has efficient processing capabilities, which can meet the production line's needs for fast and accurate detection, and improve overall production efficiency and product quality.

[0138] Based on the same inventive concept, the embodiment of the present application also provides a device for implementing the above-mentioned grid line intersection recognition. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above-mentioned method, so the specific limitations in one or more grid line intersection recognition device embodiments provided below can refer to the above-mentioned limitations on the grid line intersection recognition method, and will not be repeated here.

[0139] In an exemplary embodiment, Fig.10 As shown, a grid line intersection recognition device is provided, comprising:

[0140] The grayscale image acquisition module 501 acquires the grid line image and performs grayscale processing to obtain a grayscale image.

[0141] The region of interest image acquisition module 502 uses the region of interest to intercept the grayscale image to obtain the region of interest image.

[0142] The normalized image acquisition module 503 performs normalization processing on the image of the region of interest to obtain a normalized image.

[0143] The grid line skeleton image acquisition module 504 obtains a grid line skeleton image according to the normalized image.

[0144] The skeleton edge segment extraction module 505 acquires a grid line skeleton according to the grid line skeleton image to construct a skeleton edge segment set, and divides the skeleton edge segment set into a horizontal edge segment set and a vertical edge segment set.

[0145] The grid intersection extraction module 506 takes the points common to the horizontal edge segment set and the vertical edge segment set as grid line intersections.

[0146] In an exemplary embodiment, a computer device is provided. The computer device may be a server or a terminal. The internal structure diagram thereof may be as follows: Fig.11 As shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, referred to as I / O) and a communication interface. The processor, the memory and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store video tag processing data. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a grid line intersection recognition method is implemented.

[0147] Those skilled in the art will understand that Fig.11 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0148] In an exemplary embodiment, a computer device is further provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above-mentioned method embodiments when executing the computer program.

[0149] In an exemplary embodiment, a computer-readable storage medium is provided, storing a computer program, and when the computer program is executed by a processor, the steps in the above method embodiments are implemented.

[0150] In an exemplary embodiment, a computer program product is provided, including a computer program, and when the computer program is executed by a processor, the steps in the above method embodiments are implemented.

[0151] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.

[0152] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM may be in various forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM).

[0153] The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. The non-relational database may include a distributed database based on blockchain, etc., but is not limited thereto. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but is not limited thereto.

[0154] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0155] This article uses specific examples to illustrate the principles and implementation methods of this application. The description of the above embodiments is only used to help understand the method and core ideas of this application. At the same time, for those skilled in the art, according to the ideas of this application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting this application.

Claims

1. A method for identifying grid line intersections, characterized in that: The grid line intersection recognition method comprises: Acquire the grid line image and perform grayscale processing to obtain a grayscale image; Using the region of interest to intercept the grayscale image to obtain a region of interest image; Normalizing the image of the region of interest to obtain a normalized image; According to the normalized image, a grid line skeleton image is obtained; Obtaining a grid line skeleton according to the grid line skeleton image to construct a skeleton edge segment set, and dividing the skeleton edge segment set into a horizontal edge segment set and a vertical edge segment set, specifically including: Calculating the grid line skeleton image connected domain of the grid line skeleton image, determining whether an intersection is encountered during the traversal of the current grid line skeleton image connected domain, and obtaining a determination result; If the judgment result is no, the first point and the last point traversed on the connected domain of the current grid line skeleton image are connected to form an edge segment, which is added to the skeleton edge segment set; If the judgment result is yes, the current grid line skeleton image connected domain is disconnected at the intersection, and the first point traversed on the current grid line skeleton image connected domain is connected to the intersection to form an edge segment, which is added to the skeleton edge segment set; Traverse the next grid line skeleton image connected domain of the current grid line skeleton image connected domain, return to the step of "calculating the grid line skeleton image connected domain of the grid line skeleton image, determining whether an intersection is encountered during the traversal of the current grid line skeleton image connected domain, and obtaining a determination result", until all grid line skeleton image connected domains are traversed; Traverse the skeleton edge segment set, calculate the straight line slope and straight line deviation of each edge segment respectively, and remove the edge segments whose straight line deviation exceeds the deviation threshold; calculate the edge segment angle according to the straight line slope, and classify the edge segments whose edge segment angles are in the first angle interval or the second angle interval into the horizontal edge segment set and the vertical edge segment set respectively, and remove the remaining edge segments; The points common to the horizontal edge segment set and the vertical edge segment set are taken as grid line intersection points.

2. The grid line intersection recognition method according to claim 1, characterized in that: The method of using the region of interest to intercept the grayscale image to obtain the region of interest image specifically includes: Binarize the grayscale image to obtain a binary image; Perform morphological closing operation on the binary image to obtain a smooth image; On the smooth image, identifying and calculating the connected domain of the smooth image, selecting the one with the largest area as the basic region of the connected domain of the smooth image, and taking the area enclosed by the minimum circumscribed rectangle of the basic region as the region of interest; The grayscale image is intercepted using the region of interest to obtain the region of interest image.

3. The grid line intersection recognition method according to claim 1, characterized in that: The step of obtaining a grid line skeleton image according to the normalized image specifically includes: Performing Gaussian blur processing on the normalized image to obtain a Gaussian blurred image, and performing histogram adaptive equalization operation on the Gaussian blurred image to obtain a histogram image; Perform adaptive threshold segmentation on the histogram image to obtain an adaptive mask; Perform canny edge extraction on the normalized image to obtain an edge mask image, and perform dilation operation on the edge mask image to obtain an edge mask; Perform pixel AND operation on the adaptive mask and the edge mask to obtain a skeleton mask; The skeleton mask connected domain is identified and calculated on the skeleton mask, and the skeleton mask connected domain with an area smaller than a threshold is eliminated to obtain a grid line skeleton image.

4. The grid line intersection recognition method according to claim 1, characterized in that: The method of taking the points common to the horizontal edge segment set and the vertical edge segment set as grid line intersections may also include: The head endpoint of the current horizontal edge segment in the horizontal edge segment set is taken out, and the first distance from the head endpoints of all other horizontal edge segments except the current horizontal edge segment to the head endpoint of the current horizontal edge segment and the second distance from the tail endpoints of all other horizontal edge segments except the current horizontal edge segment to the head endpoint of the current horizontal edge segment are calculated respectively, and the smaller value of the first distance and the second distance is compared with the preset threshold value to obtain a judgment result; if the judgment result is less than, the current horizontal edge segment is added to the edge segment set to be selected; if the judgment result is not less than, the head endpoint of the next horizontal edge segment of the current horizontal edge segment in the horizontal edge segment set is taken out, and the above steps are repeated until all horizontal edge segments in the horizontal edge segment set are traversed; Traverse the set of edge segments to be selected, and remove the edge segments to be selected that are smaller than the horizontal threshold; Calculate the distance between each candidate edge segment in the eliminated candidate edge segment set and the current horizontal edge segment, and select the candidate edge segment corresponding to the minimum distance as the best edge segment of the current horizontal edge segment; Determine whether there is an inclusion relationship between the current horizontal edge segment and the best edge segment. If there is no inclusion relationship, connect the current horizontal edge segment and the best edge segment to obtain a connected horizontal edge segment, and mark the connected best edge segment; if there is an inclusion relationship, select the next horizontal edge segment of the current horizontal edge segment, and return to the step of "respectively calculating the distance between each candidate edge segment in the eliminated candidate edge segment set and the current horizontal edge segment, and selecting the candidate edge segment corresponding to the minimum distance as the best edge segment of the current horizontal edge segment" until there is no connectable horizontal edge segment in the eliminated candidate edge segment set; Remove connected horizontal edge segments whose length after being connected is less than the length threshold.

5. The grid line intersection recognition method according to claim 1, characterized in that: The method further comprises: taking the points common to the horizontal edge segment set and the vertical edge segment set as the grid line intersection points; and then: Calculate the grayscale centroid coordinates of the c×c neighborhood of each grid line intersection, and use the grayscale centroid coordinates as the updated grid line intersection coordinates. The calculation formula of the grayscale centroid coordinates is: ; ; Among them, x c is the horizontal coordinate of the updated grid line intersection coordinates, y c is the ordinate of the updated grid line intersection coordinates, x i is the horizontal coordinate of the i-th pixel in the c×c neighborhood, y i is the ordinate of the i-th pixel in the c×c neighborhood, i=0, 1, 2, 3…, c×c-1, v i represents the gray value of the i-th pixel, x b is the pixel horizontal coordinate of the grid line intersection, y b The pixel ordinate of the grid line intersection.

6. A grid line intersection recognition device, characterized in that: The grid line intersection recognition device comprises: A grayscale image acquisition module acquires a grid line image and performs grayscale processing to obtain a grayscale image; An area of ​​interest image acquisition module uses the area of ​​interest to intercept the grayscale image to obtain an area of ​​interest image; A normalized image acquisition module performs normalization processing on the image of the region of interest to obtain a normalized image; A grid line skeleton image acquisition module obtains a grid line skeleton image according to the normalized image; The skeleton edge segment extraction module obtains the grid line skeleton according to the grid line skeleton image to construct a skeleton edge segment set, and divides the skeleton edge segment set into a horizontal edge segment set and a vertical edge segment set, specifically including: Calculating the grid line skeleton image connected domain of the grid line skeleton image, determining whether an intersection is encountered during the traversal of the current grid line skeleton image connected domain, and obtaining a determination result; If the judgment result is no, the first point and the last point traversed on the connected domain of the current grid line skeleton image are connected to form an edge segment, which is added to the skeleton edge segment set; If the judgment result is yes, the current grid line skeleton image connected domain is disconnected at the intersection, and the first point traversed on the current grid line skeleton image connected domain is connected to the intersection to form an edge segment, which is added to the skeleton edge segment set; Traverse the next grid line skeleton image connected domain of the current grid line skeleton image connected domain, return to the step of "calculating the grid line skeleton image connected domain of the grid line skeleton image, determining whether an intersection is encountered during the traversal of the current grid line skeleton image connected domain, and obtaining a determination result", until all grid line skeleton image connected domains are traversed; Traverse the skeleton edge segment set, calculate the straight line slope and straight line deviation of each edge segment respectively, and remove the edge segments whose straight line deviation exceeds the deviation threshold; calculate the edge segment angle according to the straight line slope, and classify the edge segments whose edge segment angles are in the first angle interval or the second angle interval into the horizontal edge segment set and the vertical edge segment set respectively, and remove the remaining edge segments; The grid intersection extraction module takes the points common to the horizontal edge segment set and the vertical edge segment set as grid line intersections.

7. A computer device comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the grid line intersection identification method described in any one of claims 1 to 5.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the grid line intersection recognition method described in any one of claims 1 to 5 is implemented.

9. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the grid line intersection recognition method described in any one of claims 1 to 5 is implemented.

Images