Image straight line detection method and device, equipment and medium

Abstract

The embodiment of the invention discloses an image straight line detection method and device, equipment and a medium, and the method comprises the steps: dividing an identified straight line into a transverse line and a vertical line; sorting the vertical lines and sorting the transverse lines; determining information of a vertical line region where each vertical line is located, and taking an image region between two adjacent vertical line regions as a transverse deletable region; determining information of a transverse line region where each transverse line is located, and taking an image region between two adjacent transverse line regions as a longitudinal deletable region; the method comprises the steps of deleting a transverse deletable area and a longitudinal deletable area in an original image, performing straight line detection on a new image obtained after the areas are deleted, mapping end point position information, obtained through detection, of each straight line in the new image into the original image, and obtaining end point position information of the straight line to which each straight line belongs in the original image. By adopting the technical scheme, the detection speed can be effectively improved while the linear detection precision is ensured.

Description

Technical Field

[0001] The embodiments of the present invention relate to the field of image recognition technology, and in particular, to a method, device, equipment and medium for detecting straight lines in an image. Background Art

[0002] Detecting straight lines in images is an important task in the field of computer vision and image processing.

[0003] There are currently two methods for line detection. One is the traditional method, such as Hough transform, probabilistic Hough transform, LSD (line segment detection) and other line detection algorithms; the other is a method based on deep learning. The above two methods have slow detection speeds for larger document images and cannot meet the needs of practical applications, so acceleration processing is required. At present, accelerated detection methods can generally be divided into two categories. One is to quantize or prune deep learning to reduce the amount of model calculation. The other is to accelerate the model through hardware.

[0004] Of the two types of acceleration technologies mentioned above, the former actually sacrifices calculation accuracy while accelerating; the latter requires specific hardware and related engineering development, is costly, and has limited application. Summary of the invention

[0005] The embodiments of the present invention provide a method, device, equipment and medium for detecting straight lines in an image, so as to effectively improve the detection speed while ensuring the straight line detection accuracy.

[0006] In a first aspect, the present invention provides a method for detecting straight lines in an image, comprising:

[0007] Identify the straight lines in the original image and divide the identified straight lines into horizontal lines and vertical lines;

[0008] Sort the vertical lines according to the positions of the horizontal coordinates of the center points of the vertical lines, and sort the horizontal lines according to the positions of the vertical coordinates of the center points of the horizontal lines;

[0009] Determine the vertical line area information of each vertical line according to the endpoint positions of the sorted vertical lines, and use the image area between two adjacent vertical line areas as a horizontal deletable area, wherein the vertical line area information includes the original left boundary position and the original right boundary position of the vertical line area;

[0010] According to the position of the endpoints of the sorted horizontal lines, the horizontal line area information of each horizontal line is determined, and the image area between two adjacent horizontal line areas is used as the vertical deletable area, wherein the horizontal line area information includes the original upper boundary position and the original lower boundary position of the horizontal line area;

[0011] Delete the horizontal and vertical deletable areas in the original image, and perform straight line detection on the new image obtained after the area deletion to obtain the endpoint position information of each straight line in the new image;

[0012] The endpoint position information of each straight line in the new image is mapped to the original image to obtain the endpoint position information of the straight line to which each straight line belongs in the original image.

[0013] Optionally, determining the vertical line region information where each vertical line is located according to the endpoint positions of the sorted vertical lines includes:

[0014] The original left boundary position and the original right boundary position of each vertical line are determined according to the following formula:

[0015] Leftxi=max(0,min(xti,xbi)-th)

[0016] Rightxi=min(w,max(xti,xbi)+th)

[0017] Among them, xti represents the horizontal coordinate of the first endpoint of the i-th vertical line, xbi represents the horizontal coordinate of the second endpoint of the i-th vertical line, th represents a constant, which represents the set multiple of the straight line width, w represents the width of the original image, Leftxi represents the original left boundary position of the i-th vertical line, and Rightxi represents the original right boundary position of the i-th vertical line.

[0018] Optionally, determining the horizontal line region information where each horizontal line is located according to the endpoint positions of the sorted horizontal lines includes:

[0019] Determine the original upper boundary position and the original lower boundary position of each horizontal line according to the following formula:

[0020] Topyi=max(0,min(yli,yri)-th)

[0021] Bottomyi=min(h,max(yli,yri)+th)

[0022] Among them, yli represents the ordinate of the first endpoint of the i-th horizontal line, yri represents the ordinate of the second endpoint of the i-th horizontal line, th represents a constant, which represents the set multiple of the straight line width, h represents the height of the original image, Topyi represents the original upper boundary position of the i-th horizontal line, and Bottomyi represents the original lower boundary position of the i-th horizontal line.

[0023] Optionally, performing straight line detection on the new image obtained after the region is deleted to obtain the endpoint position information of each straight line in the new image includes:

[0024] Determine a new left boundary position and a new right boundary position of each horizontal line, and a new upper boundary position and a new lower boundary position of each vertical line in a new image obtained after the region is deleted;

[0025] According to the new left boundary position and the new right boundary position of each horizontal line, and the new upper boundary position and the new lower boundary position of each vertical line, and based on the deep line detection model, line detection is performed to obtain the endpoint position information of each vertical line and each horizontal line in the new image.

[0026] Optionally, determining a new left boundary position and a new right boundary position of each horizontal line, and a new upper boundary position and a new lower boundary position of each vertical line in a new image obtained after the region is deleted includes:

[0027] According to the following formula, the new left boundary position and the new right boundary position of each vertical line in the new image obtained after the region is deleted are determined:

[0028]

[0029] S1_Rightxi=S1_Leftxi+Rightxi-Leftxi

[0030] Furthermore, according to the following formula, the new upper boundary position and the new lower boundary position of each horizontal line in the new image obtained after the region is deleted are determined:

[0031]

[0032] S1_Bottomyj=S1_Topyj+Bottomyj-Topyj

[0033] Among them, S1_Leftxi and S1_Rightxi respectively represent the horizontal coordinates of the new left boundary and the new right boundary of the i-th vertical line in the new image, Rightxk and Leftxk respectively represent the horizontal coordinates of the original left boundary and the original right boundary of the k-th vertical line in the original image; S1_Topyj and S1_Bottomyj respectively represent the vertical coordinates of the new upper boundary and the new lower boundary of the j-th horizontal line in the new image, Topyk and Bottomyk respectively represent the horizontal coordinates of the original upper boundary and the original lower boundary of the k-th horizontal line in the original image.

[0034] Optionally, the endpoint position information of each straight line in the new image is mapped to the original image to obtain the endpoint position information of the straight line to which each straight line belongs in the original image, including:

[0035] For each target detection line obtained by the depth line detection model, respectively determine the region information where two endpoints of the target detection line are located in the new image, wherein the region information includes first region information corresponding to the horizontal coordinates of each endpoint and second region information corresponding to the vertical coordinates of each endpoint;

[0036] According to the endpoint position information of the target detection line, the area information and its corresponding original boundary position information in the original image, the target detection line is mapped to the original image to obtain the endpoint position information of the line to which the target detection line belongs in the original image.

[0037] Optionally, mapping the target detection line to the original image according to the endpoint position information of the target detection line, the region information and the original boundary position information corresponding to the target detection line in the original image to obtain the endpoint position information of the line to which the target detection line belongs in the original image includes:

[0038] When the target detection line is a vertical line, the first region information corresponding to the horizontal coordinates of the two endpoints of the target detection line in the new image is determined according to the following formula:

[0039] S1_Leftxi≤cps_xp1≤S1_Rightxi

[0040] S1_Leftxi≤cps_xp2≤S1_Rightxi

[0041] Accordingly, the horizontal coordinate of the line to which each target detection line belongs in the original image is determined according to the following formula:

[0042] fin_xp1=Leftxi+cps_xp1-S1_Leftxi

[0043] fin_xp2=Leftxi+cps_xp2-S1_Leftxi

[0044] Furthermore, the second region information corresponding to the ordinates of the two end points of the target detection line in the new image is determined respectively according to the following formula:

[0045] S1_Topyj≤cps_yp1≤S1_Bottomyj

[0046] S1_Topyk≤cps_yp2≤S1_Bottomyk

[0047] Accordingly, the ordinate of the line to which each target detection line belongs in the original image is determined according to the following formula:

[0048] fin_yp1=Topyj+cps_yp1-S1_Topyj

[0049] fin_yp2=Topyk+cps_yp2-S1_Topyk

[0050] Among them, (cps_xp1, cps_yp1) represents the horizontal coordinate and vertical coordinate of the first endpoint of the p-th vertical line in the new image, (cps_xp2, cps_yp2) represents the horizontal coordinate and vertical coordinate of the second endpoint of the p-th vertical line in the new image, (fin_xp1, fin_yp1) represents the horizontal coordinate and vertical coordinate of the first endpoint of the p-th vertical line in the new image mapped to the corresponding straight line in the original image, (fin_xp2, fin_yp2) represents the horizontal coordinate and vertical coordinate of the second endpoint of the p-th vertical line in the new image mapped to the corresponding straight line in the original image, S1_Leftxi represents the horizontal coordinate of the new left boundary of the ith vertical line in the new image, S1_Rightxi represents the horizontal coordinate of the new right boundary of the ith vertical line in the new image; Leftxi represents the horizontal coordinate of the original left boundary of the ith vertical line in the original image; S1_Topy represents the vertical coordinate of the new upper boundary of the j-th horizontal line in the new image, S1_Bo ttom represents the ordinate of the new lower boundary of the j-th horizontal line in the new image; S1_Topyk represents the ordinate of the new upper boundary of the k-th horizontal line in the new image, S1_Bottomyk represents the ordinate of the new lower boundary of the k-th horizontal line in the new image; Topyj represents the ordinate of the original upper boundary of the j-th horizontal line in the original image; Topyk represents the ordinate of the original upper boundary of the k-th horizontal line in the original image;

[0051] When the target detection line is a horizontal line, the second region information corresponding to the ordinates of the two endpoints of the target detection line in the new image is determined according to the following formula:

[0052] S1_Topyi≤cps_yq1≤S1_Bottomyi

[0053] S1_Topyi≤cps_yq2≤S1_Bottomyi

[0054] Accordingly, the ordinate of the line to which each target detection line belongs in the original image is determined according to the following formula:

[0055] fin_yq1=Topyi+cps_yq1-S1_Topyi

[0056] fin_yq2=Topyi+cps_xq2-S1_Topyi

[0057] Furthermore, the first region information corresponding to the horizontal coordinates of the two end points of the target detection line in the new image is determined respectively according to the following formula:

[0058] S1_Leftxj≤cps_xq1≤S1_Rightxj

[0059] S1_Leftxk≤cps_xq2≤S1_Rightxk

[0060] Accordingly, the horizontal coordinate of the line to which each target detection line belongs in the original image is determined according to the following formula:

[0061] fin_xq1=Leftxj+cps_yq1-S1_Leftxj

[0062] fin_xq2=Leftxk+cps_yq2-S1_Leftxk

[0063] Among them, (cps_xq1, cps_yq1) represents the horizontal coordinate and vertical coordinate of the first endpoint of the qth horizontal line in the new image, (cps_xq2, cps_yq2) represents the horizontal coordinate and vertical coordinate of the second endpoint of the qth horizontal line in the new image, (fin_xq1, fin_yq1) represents the horizontal coordinate and vertical coordinate of the first endpoint of the qth horizontal line in the new image mapped to the corresponding straight line in the original image, (fin_xq2, fin_yq2) represents the horizontal coordinate and vertical coordinate of the second endpoint of the qth horizontal line in the new image mapped to the corresponding straight line in the original image, S1_Topyi represents the vertical coordinate of the new upper boundary of the ith horizontal line in the new image, and S1_Bottomyi represents the vertical coordinate of the qth horizontal line in the new image. The ordinate of the new lower boundary of the i-th horizontal line in the new image; Topyi represents the ordinate of the original upper boundary of the ith horizontal line in the original image; Leftxj represents the ordinate of the original left boundary of the j-th vertical line in the original image; Leftxk represents the ordinate of the original left boundary of the k-th vertical line in the original image; S1_Leftxj represents the ordinate of the new left boundary of the j-th vertical line in the new image; S1_Leftxk represents the ordinate of the new left boundary of the k-th vertical line in the new image; S1_Rightxj represents the ordinate of the new right boundary of the j-th vertical line in the new image; S1_Rightxk represents the ordinate of the new right boundary of the k-th vertical line in the new image.

[0064] In a second aspect, an embodiment of the present invention further provides an image straight line detection device, comprising:

[0065] A straight line classification module is configured to identify straight lines in the original image and classify the identified straight lines into horizontal lines and vertical lines;

[0066] A straight line sorting module is configured to sort each vertical line according to the position of the horizontal coordinate of the center point of the vertical line, and to sort each horizontal line according to the position of the vertical coordinate of the center point of the horizontal line;

[0067] a horizontal deletable area determination module, configured to determine the vertical line area information where each vertical line is located according to the endpoint positions of the sorted vertical lines, and to use the image area between two adjacent vertical line areas as the horizontal deletable area, wherein the vertical line area information includes the original left boundary position and the original right boundary position of the vertical line area;

[0068] A vertical deletable area determination module is configured to determine the horizontal line area information where each horizontal line is located according to the endpoint positions of the sorted horizontal lines, and use the image area between two adjacent horizontal line areas as the vertical deletable area, wherein the horizontal line area information includes the original upper boundary position and the original lower boundary position of the horizontal line area;

[0069] A line detection module is configured to delete the horizontal deletable area and the vertical deletable area in the original image, and perform line detection on the new image obtained after the area deletion to obtain the endpoint position information of each line in the new image;

[0070] The mapping module is configured to map the endpoint position information of each straight line in the new image to the original image to obtain the endpoint position information of the straight line to which each straight line belongs in the original image.

[0071] Optionally, the horizontal deletable area determination module includes: a horizontal boundary determination unit and a horizontal deletable area determination unit, wherein the horizontal boundary determination unit is specifically configured to:

[0072] The original left boundary position and the original right boundary position of each vertical line are determined according to the following formula:

[0073] Leftxi=max(0,min(xti,xbi)-th)

[0074] Rightxi=min(w,max(xti,xbi)+th)

[0075] Among them, xti represents the horizontal coordinate of the first endpoint of the i-th vertical line, xbi represents the horizontal coordinate of the second endpoint of the i-th vertical line, th represents a constant, which represents the set multiple of the straight line width, w represents the width of the original image, Leftxi represents the original left boundary position of the i-th vertical line, and Rightxi represents the original right boundary position of the i-th vertical line.

[0076] Optionally, the vertical deletable area determination module includes: a vertical boundary determination unit and a vertical deletable area determination unit, wherein the vertical boundary confirmation unit is specifically configured to: determine the original upper boundary position and the original lower boundary position of each horizontal line according to the following formula:

[0077] Topyi=max(0,min(yli,yri)-th)

[0078] Bottomyi=min(h,max(yli,yri)+th)

[0079] Among them, yli represents the ordinate of the first endpoint of the i-th horizontal line, yri represents the ordinate of the second endpoint of the i-th horizontal line, th represents a constant, which represents the set multiple of the straight line width, h represents the height of the original image, Topyi represents the original upper boundary position of the i-th horizontal line, and Bottomyi represents the original lower boundary position of the i-th horizontal line.

[0080] Optionally, the line detection module includes a region deletion unit and a line detection unit, wherein the line detection unit includes:

[0081] The new boundary position determination subunit is configured to: determine a new left boundary position and a new right boundary position of each horizontal line, and a new upper boundary position and a new lower boundary position of each vertical line in a new image obtained after the region is deleted;

[0082] The line detection subunit is configured to: perform line detection based on the new left boundary position and the new right boundary position of each horizontal line, and the new upper boundary position and the new lower boundary position of each vertical line, and based on the deep line detection model, to obtain the endpoint position information of each vertical line and each horizontal line in the new image.

[0083] Optionally, the new boundary position determination subunit is specifically configured as follows:

[0084] According to the following formula, the new left boundary position and the new right boundary position of each vertical line in the new image obtained after the region is deleted are determined:

[0085]

[0086] S1_Rightxi=S1_Leftxi+Rightxi-Leftxi

[0087] Furthermore, according to the following formula, the new upper boundary position and the new lower boundary position of each horizontal line in the new image obtained after the region is deleted are determined:

[0088]

[0089] S1_Bottomyj=S1_Topyj+Bottomyj-Topyj

[0090] Among them, S1_Leftxi and S1_Rightxi respectively represent the horizontal coordinates of the new left boundary and the new right boundary of the i-th vertical line in the new image, Rightxk and Leftxk respectively represent the horizontal coordinates of the original left boundary and the original right boundary of the k-th vertical line in the original image; S1_Topyj and S1_Bottomyj respectively represent the vertical coordinates of the new upper boundary and the new lower boundary of the j-th horizontal line in the new image, Topyk and Bottomyk respectively represent the horizontal coordinates of the original upper boundary and the original lower boundary of the k-th horizontal line in the original image.

[0091] Optional, mapping modules, including:

[0092] a corresponding line determination unit, configured to determine, for each target detection line obtained by the depth line detection model, region information where two endpoints of the target detection line are located in the new image, the region information including first region information corresponding to the horizontal coordinates of each endpoint and second region information corresponding to the vertical coordinates of each endpoint;

[0093] The straight line mapping unit is configured to map the target detection straight line to the original image according to the endpoint position information of the target detection straight line, the area information and the original boundary position information corresponding to the target detection straight line in the original image, so as to obtain the endpoint position information of the straight line to which the target detection straight line belongs in the original image.

[0094] Optionally, the linear mapping unit is specifically configured as follows:

[0095] When the target detection line is a vertical line, the first region information corresponding to the horizontal coordinates of the two endpoints of the target detection line in the new image is determined according to the following formula:

[0096] S1_Leftxi≤cps_xp1≤S1_Rightxi

[0097] S1_Leftxi≤cps_xp2≤S1_Rightxi

[0098] Accordingly, the horizontal coordinate of the line to which each target detection line belongs in the original image is determined according to the following formula:

[0099] fin_xp1=Leftxi+cps_xp1-S1_Leftxi

[0100] fin_xp2=Leftxi+cps_xp2-S1_Leftxi

[0101] Furthermore, the second region information corresponding to the ordinates of the two end points of the target detection line in the new image is determined respectively according to the following formula:

[0102] S1_Topyj≤cps_yp1≤S1_Bottomyj

[0103] S1_Topyk≤cps_yp2≤S1_Bottomyk

[0104] Accordingly, the ordinate of the line to which each target detection line belongs in the original image is determined according to the following formula:

[0105] fin_yp1=Topyj+cps_yp1-S1_Topyj

[0106] fin_yp2=Topyk+cps_yp2-S1_Topyk

[0107] Among them, (cps_xp1, cps_yp1) represent the horizontal and vertical coordinates of the first endpoint of the p-th vertical line in the new image, (cps_xp2, cps_yp2) represent the horizontal and vertical coordinates of the second endpoint of the p-th vertical line in the new image, (fin_xp1, fin_yp1) represent the horizontal and vertical coordinates of the first endpoint of the p-th vertical line in the new image mapped to the corresponding straight line in the original image, (fin_xp2, fin_yp2) represent the horizontal and vertical coordinates of the second endpoint of the p-th vertical line in the new image mapped to the corresponding straight line in the original image, S1_Leftxi represents the horizontal coordinate of the new left boundary of the ith vertical line in the new image, and S1_Rightxi represents the horizontal coordinate of the i-th vertical line. Leftxi represents the abscissa of the new right boundary of the i-th vertical line in the new image; Leftxi represents the abscissa of the original left boundary of the i-th vertical line in the original image; S1_Topyj represents the ordinate of the new upper boundary of the j-th horizontal line in the new image, and S1_Bottomy represents the ordinate of the new lower boundary of the j-th horizontal line in the new image; S1_Topyk represents the ordinate of the new upper boundary of the k-th horizontal line in the new image, and S1_Bottomyk represents the ordinate of the new lower boundary of the k-th horizontal line in the new image; Topyj represents the ordinate of the original upper boundary of the j-th horizontal line in the original image; Topyk represents the ordinate of the original upper boundary of the k-th horizontal line in the original image;

[0108] When the target detection line is a horizontal line, the second region information corresponding to the ordinates of the two endpoints of the target detection line in the new image is determined according to the following formula:

[0109] S1_Topyi≤cps_yq1≤S1_Bottomyi

[0110] S1_Topyi≤cps_yq2≤S1_Bottomyi

[0111] Accordingly, the ordinate of the line to which each target detection line belongs in the original image is determined according to the following formula:

[0112] fin_yq1=Topyi+cps_yq1-S1_Topyi

[0113] fin_yq2=Topyi+cps_xq2-S1_Topyi

[0114] Furthermore, the first region information corresponding to the horizontal coordinates of the two end points of the target detection line in the new image is determined respectively according to the following formula:

[0115] S1_Leftxj≤cps_xq1≤S1_Rightxj

[0116] S1_Leftxk≤cps_xq2≤S1_Rightxk

[0117] Accordingly, the horizontal coordinate of the line to which each target detection line belongs in the original image is determined according to the following formula:

[0118] fin_xq1=Leftxj+cps_yq1-S1_Leftxj

[0119] fin_xq2=Leftxk+cps_yq2-S1_Leftxk

[0120] Among them, (cps_xq1, cps_yq1) represents the horizontal coordinate and vertical coordinate of the first endpoint of the qth horizontal line in the new image, (cps_xq2, cps_yq2) represents the horizontal coordinate and vertical coordinate of the second endpoint of the qth horizontal line in the new image, (fin_xq1, fin_yq1) represents the horizontal coordinate and vertical coordinate of the first endpoint of the qth horizontal line in the new image mapped to the corresponding straight line in the original image, (fin_xq2, fin_yq2) represents the horizontal coordinate and vertical coordinate of the second endpoint of the qth horizontal line in the new image mapped to the corresponding straight line in the original image, S1_Topyi represents the vertical coordinate of the new upper boundary of the ith horizontal line in the new image, and S1_Bottomyi represents the vertical coordinate of the qth horizontal line in the new image. The ordinate of the new lower boundary of the i-th horizontal line in the new image; Topyi represents the ordinate of the original upper boundary of the ith horizontal line in the original image; Leftxj represents the ordinate of the original left boundary of the j-th vertical line in the original image; Leftxk represents the ordinate of the original left boundary of the k-th vertical line in the original image; S1_Leftxj represents the ordinate of the new left boundary of the j-th vertical line in the new image; S1_Leftxk represents the ordinate of the new left boundary of the k-th vertical line in the new image; S1_Rightxj represents the ordinate of the new right boundary of the j-th vertical line in the new image; S1_Rightxk represents the ordinate of the new right boundary of the k-th vertical line in the new image.

[0121] In a third aspect, an embodiment of the present invention further provides a computing device, including:

[0122] A memory storing executable program code;

[0123] a processor coupled to the memory;

[0124] The processor calls the executable program code stored in the memory to execute the image straight line detection method provided by any embodiment of the present invention.

[0125] In a fourth aspect, an embodiment of the present invention further provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the image straight line detection method provided by any embodiment of the present invention.

[0126] The technical solution provided by the embodiment of the present invention, when detecting the straight line in the original image, first determines the area where each horizontal line and vertical line in the original image is located, and uses the image area between two adjacent vertical line areas as a horizontal deletable area, and uses the image area between two adjacent horizontal line areas as a vertical deletable area. By deleting the vertical deletable area and the horizontal deletable area, the image can be reduced while the straight line resolution and the number of straight lines remain unchanged, that is, by deleting the redundant image area except the straight line area in the original image, when detecting the straight line retained in the obtained new image, the image detection time occupied due to the detection of non-straight line areas is avoided, and the speed of image straight line detection is effectively improved while ensuring the detection accuracy. In addition, by mapping the horizontal lines and vertical lines in the new image after the area is deleted to the original image, the position information of the endpoints of each horizontal line and vertical line in the original image can be obtained, and the accurate detection of the image straight line is realized. BRIEF DESCRIPTION OF THE DRAWINGS

[0127] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0128] Figure 1a A flowchart of a method for detecting straight lines in an image provided by Embodiment 1 of the present invention;

[0129] Figure 1b A schematic diagram of a deletable area in an image provided by the first embodiment of the present invention;

[0130] Figure 2 A structural block diagram of an image straight line detection device provided in Embodiment 2 of the present invention;

[0131] Figure 3 A schematic diagram of the structure of a computing device provided in Embodiment 3 of the present invention. DETAILED DESCRIPTION

[0132] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0133] It should be noted that the terms "including" and "having" and any variations thereof in the embodiments of the present invention and the accompanying drawings are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device including a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products or devices.

[0134] The embodiments of the present invention disclose a method, device, equipment and medium for detecting straight lines in an image, which are described in detail below.

[0135] Embodiment 1

[0136] Figure 1a This is a flow chart of a method for detecting straight lines in an image provided by Embodiment 1 of the present invention. The method can be applied to application scenarios such as document recognition, layout analysis, and table detection. The method provided by this embodiment can be executed by an image straight line detection device, which can be implemented by software and / or hardware. Figure 1a As shown, the method provided in this embodiment specifically includes:

[0137] S110 , identifying straight lines in the original image, and dividing the identified straight lines into horizontal lines and vertical lines.

[0138] Among them, the initial recognition of the straight lines in the original image can adopt the traditional straight line detection method, such as LSD method, Hough transform method, etc. By performing preliminary detection on the straight lines in the original image, the initial line segment set of each straight line in the image can be obtained.

[0139] In this embodiment, for the initial line segment set, the line segments can be divided into horizontal lines and vertical lines according to the angles of the line segments. Specifically, for all detected straight lines, straight lines with an inclination angle greater than 15 degrees and less than 75 degrees can be deleted, and straight lines with a length less than 30 pixels can be deleted. For the remaining straight lines in the initial line segment set, straight lines with an inclination angle less than 45 can be classified as horizontal lines and put into the horizontal line list listH; and straight lines with an inclination angle greater than 45 degrees and less than 90 degrees can be classified as vertical lines and put into the vertical line list listV.

[0140] In this embodiment, the horizontal lines in the horizontal line list and the vertical lines in the vertical line list can be defined by three parameters A, B, and C in the expression Ax+By+C=0.

[0141] Furthermore, in order to facilitate straight line detection, both ends of all straight lines in the horizontal line list listH and the vertical line list listV can be extended to the boundary position of the image, and the horizontal lines in the horizontal line list and the vertical lines in the vertical line list can be merged respectively. For example, the horizontal lines with a distance d less than 3 pixels can be merged into one horizontal line, and the vertical lines with a distance d less than 3 pixels can be merged into one vertical line. Specifically, the distance d of two straight lines A1x+B1y+C1=0 and A2x+B2y+C2=0 is calculated as follows:

[0142]

[0143] The combined straight line can be expressed as: Ax+By+(C1+C2) / 2=0

[0144] Among them, A=(A1+A2) / 2, B=(B1+B2) / 2.

[0145] S120, sorting the vertical lines according to the positions of the horizontal coordinates of the center points of the vertical lines, and sorting the horizontal lines according to the positions of the vertical coordinates of the center points of the horizontal lines.

[0146] For example, each vertical line can be sorted in the order of the horizontal coordinates of the center points of the vertical lines from small to large, and each horizontal line can be sorted in the order of the vertical coordinates of the center points of the horizontal lines from small to large. Alternatively, each vertical line can be sorted in the order of the horizontal coordinates of the center points of the vertical lines from large to small, and each horizontal line can be sorted in the order of the vertical coordinates of the center points of the horizontal lines from large to small, and this embodiment does not limit this. In the following, an example is given according to the sorting order from small to large.

[0147] S130 , determining the vertical line region information where each vertical line is located according to the position of the endpoints of the sorted vertical lines, and taking the image region between two adjacent vertical line regions as a horizontal deletable region.

[0148] The vertical line area information includes the boundary information of the vertical line area, and the boundary information includes the original left boundary position and the original right boundary position of the vertical line area, and the image area from the original left boundary to the original right boundary is the vertical line area where the vertical line is located. In this embodiment, the original left boundary position and the original right boundary position of the area where each vertical line is located can be determined according to the following formula:

[0149] Leftxi=max(0,min(xti,xbi)-th)

[0150] Rightxi=min(w,max(xti,xbi)+th)

[0151] Among them, xti represents the horizontal coordinate of the first endpoint of the i-th vertical line, xbi represents the horizontal coordinate of the second endpoint of the i-th vertical line, th represents a constant, which represents the set multiple of the straight line width, for example, it can be set to 3 to 5 times the straight line width; w represents the width of the original image, Leftxi represents the original left boundary position of the i-th vertical line, and Rightxi represents the original right boundary position of the i-th vertical line.

[0152] The purpose of this embodiment is to perform straight line detection. Therefore, in order to quickly identify the straight line in the image, the image area between two adjacent vertical line areas can be used as a horizontal deletable area. For the sorted vertical lines, the horizontal deletable areas can be determined in sequence as Rightx(i-1) to Leftxi, where i=1...n, and n represents the total number of vertical lines in the original image. In addition, the horizontal deletable area in the image also includes the area from the image boundary with a horizontal coordinate of 0 to the first vertical line, that is, the area from position 0 to Leftx1, and the area from the last vertical line to the image boundary with a horizontal coordinate of w, that is, the area from position Rightxn to w columns.

[0153] For example, Figure 1b A schematic diagram of a deletable area in an image provided in Example 1 of the present invention. The dotted areas A1 and A2 are both vertical line areas. The vertical line area corresponding to the vertical line V1 is composed of the original left boundary Leftx1 to the original right boundary Rightx1 of the area, and the vertical line area corresponding to the vertical line V2 is composed of the original left boundary Leftx2 to the original right boundary Rightx2 of the area. The deletable area between the vertical lines V1 and V2 is the area from Rightx1 to Leftx2.

[0154] S140, determining the horizontal line region information where each horizontal line is located according to the endpoint positions of the sorted horizontal lines, and taking the image region between two adjacent horizontal line regions as a vertical deletable region.

[0155] The horizontal line area information includes horizontal line area boundary information, and the boundary information includes the original upper boundary position and the original lower boundary position of the horizontal line area. The image area between the original upper boundary position and the original lower boundary position is the horizontal line area to which the horizontal line belongs.

[0156] In this embodiment, the original upper boundary position and the original lower boundary position of the region where each horizontal line is located can be determined according to the following formula:

[0157] Topyi=max(0,min(yli,yri)-th)

[0158] Bottomyi=min(h,max(yli,yri)+th)

[0159] Among them, yli represents the ordinate of the first endpoint of the i-th horizontal line, yri represents the ordinate of the second endpoint of the i-th horizontal line, th represents a constant, which represents the set multiple of the straight line width, h represents the height of the original image, Topyi represents the original upper boundary position of the i-th horizontal line, and Bottomyi represents the original lower boundary position of the i-th horizontal line.

[0160] The purpose of this embodiment is to detect straight lines. Therefore, in order to quickly identify straight lines in an image, the image area between two adjacent horizontal line areas can be used as a vertical deletable area. For each sorted horizontal line, the vertical deletable area can be determined in sequence as Bottomy(j-1) to Topyj, where j=1...m, and m represents the total number of horizontal lines in the original image. In addition, the horizontal deletable area in the image also includes the area from the upper boundary of the image with a vertical coordinate of 0 to the first vertical line, that is, the area from 0 to Topy1, and the area from the last horizontal line to the lower boundary of the image with a vertical coordinate of h, that is, the area from Rightyn to h columns.

[0161] For example, Figure 1b A schematic diagram of a horizontal deletable area in an image provided in Example 1 of the present invention. The dotted areas B1 and B2 are both horizontal line areas. The horizontal line area corresponding to the horizontal line H1 is composed of the original left boundary Topy1 to the original right boundary Bottomy1 of the area, and the horizontal line area corresponding to the horizontal line H2 is composed of the original left boundary Topy2 to the original right boundary Bottomy2 of the area. The deletable area between the horizontal line H1 and the horizontal line H2 is the area from Bottomy1 to Topy2.

[0162] It should be noted that this embodiment does not specifically limit the execution order of step S130 and step S140, which may be executed simultaneously or sequentially.

[0163] S150, deleting the horizontal deletable area and the vertical deletable area in the original image, and performing straight line detection on the new image obtained after the area deletion to obtain the endpoint position information of each straight line in the new image.

[0164] It should be noted that, in this embodiment, the horizontal deletable area can be deleted first and then the vertical deletable area can be deleted, or the vertical deletable area can be deleted first and then the horizontal deletable area can be deleted. This embodiment does not limit the order of deleting areas.

[0165] By deleting the non-horizontal line area and the non-vertical line area, the image can be reduced while the straight line resolution and the number of straight lines remain unchanged, that is, by deleting the redundant image area except the straight line area in the original image, when detecting the obtained new image, it is sufficient to identify the straight line area retained in the new image, thereby avoiding the image detection time occupied by detecting the non-straight line area. The technical solution provided in this embodiment improves the speed of image straight line detection while ensuring the accuracy of straight line detection. For example, if a deep learning model is used to detect straight lines in an image, by deleting the non-straight line area in the image, the convolution calculation amount of the model for the deleted area is reduced, and the detection speed is effectively improved.

[0166] In this embodiment, for the new image obtained after the area is deleted, the new left boundary position and the new right boundary position of each horizontal line in the new image, as well as the new upper boundary position and the new lower boundary position of each vertical line can be determined; according to the new left boundary position and the new right boundary position of each horizontal line, as well as the new upper boundary position and the new lower boundary position of each vertical line, and based on the deep line detection model, line detection is performed to obtain the endpoint position information of each vertical line and each horizontal line in the new image.

[0167] Among them, the new left boundary position and the new right boundary position of each horizontal line in the new image can be calculated by the following formula:

[0168]

[0169] S1_Rightxi=S1_Leftxi+Rightxi-Leftxi

[0170] The new upper boundary position and the new lower boundary position of each vertical line in the new image can be calculated by the following formula:

[0171]

[0172] S1_Bottomyj=S1_Topyj+Bottomyj-Topyj

[0173] Among them, S1_Leftxi and S1_Rightxi respectively represent the horizontal coordinates of the new left boundary and the new right boundary of the i-th vertical line in the new image, Rightxk and Leftxk respectively represent the horizontal coordinates of the original left boundary and the original right boundary of the k-th vertical line in the original image; S1_Topyj and S1_Bottomyj respectively represent the vertical coordinates of the new upper boundary and the new lower boundary of the j-th horizontal line in the new image, Topyk and Bottomyk respectively represent the horizontal coordinates of the original upper boundary and the original lower boundary of the k-th horizontal line in the original image.

[0174] S160, mapping the endpoint position information of each straight line in the new image to the original image, to obtain the endpoint position information of the straight line to which each straight line belongs in the original image.

[0175] In this embodiment, after the straight line (including horizontal and vertical lines) in the new image is mapped to the original image, the horizontal coordinates of its endpoints should be within the range of the new left boundary position and the new right boundary position of the straight line corresponding to the horizontal line, and the vertical coordinates of the points after mapping should be within the range of the new upper boundary position and the new lower boundary position of the straight line corresponding to the vertical line. If the horizontal coordinate is not within the range of the new left boundary position and the new right boundary position of the straight line corresponding to the straight line, the straight line is deleted. Alternatively, if the vertical coordinate is no longer within the range of the new upper boundary position and the new lower boundary position of the straight line corresponding to the straight line, the straight line also needs to be deleted. If the horizontal coordinates of the endpoints after mapping are within the range of the new left boundary position and the new right boundary position of the straight line corresponding to the horizontal line, and the vertical coordinates of the points after mapping are within the range of the new upper boundary position and the new lower boundary position of the straight line corresponding to the vertical line, the endpoint position information of the straight line in the original image can be determined based on the linear mapping relationship between the straight line and the corresponding straight line in the original image.

[0176] Exemplarily, for each target detection line obtained by the deep line detection model, the area information where the two endpoints of the target detection line are located in the new image can be determined respectively, and the area information includes first area information corresponding to the horizontal coordinate of each endpoint, and second area information corresponding to the vertical coordinate of each endpoint.

[0177] Specifically, for any accurately detected vertical line p in the new image obtained after region deletion, the coordinates of the two endpoints are (cps_xp1, cps_yp1), (cps_xp2, cps_yp2), first confirm the first region information corresponding to the horizontal coordinates of the two endpoints. Specifically, the following formula can be used to determine whether the horizontal coordinates of the two endpoints simultaneously meet the following conditions:

[0178] S1_Leftxi≤cps_xp1≤S1_Rightxi

[0179] S1_Leftxi≤cps_xp2≤S1_Rightxi

[0180] Delete the straight lines that do not meet the above conditions. If the above conditions are met, map the vertical line to the original image, and obtain the horizontal coordinate of the straight line in the original image:

[0181] fin_xp1=Leftxi+cps_xp1-S1_Leftxi

[0182] fin_xp2=Leftxi+cps_xp2-S1_Leftxi

[0183] Secondly, the second area information corresponding to the ordinates of the two endpoints of the target detection line in the new image can be determined respectively. Specifically, the following formula can be used to confirm whether the ordinates of the two endpoints of the vertical line meet the following conditions:

[0184] S1_Topyj≤cps_yp1≤S1_Bottomyj

[0185] S1_Topyk≤cps_yp2≤S1_Bottomyk

[0186] Delete the straight lines that do not meet the condition. If the condition is met, map the vertical line to the original image, and the vertical coordinates of the mapped straight lines in the original image are:

[0187] fin_yp1=Topyj+cps_yp1-S1_Topypj

[0188] fin_yp2=Topyk+cps_yp2-S1_Topyk

[0189] Among them, (cps_xp1, cps_yp1) represents the horizontal coordinate and vertical coordinate of the first endpoint of the p-th vertical line in the new image, (cps_xp2, cps_yp2) represents the horizontal coordinate and vertical coordinate of the second endpoint of the p-th vertical line in the new image, (fin_xp1, fin_yp1) represents the horizontal coordinate and vertical coordinate of the first endpoint of the p-th vertical line in the new image mapped to the straight line in the original image, (fin_xp2, fin_yp2) represents the horizontal coordinate and vertical coordinate of the second endpoint of the p-th vertical line in the new image mapped to the straight line in the original image, S1_Leftxi represents the horizontal coordinate of the new left boundary of the ith vertical line in the new image, and S1_Rightxi represents the horizontal coordinate of the i-th vertical line. Leftxi represents the abscissa of the new right boundary of the i-th vertical line in the new image; Leftxi represents the abscissa of the original left boundary of the i-th vertical line in the original image; S1_Topyj represents the ordinate of the new upper boundary of the j-th horizontal line in the new image, and S1_Bottomyj represents the ordinate of the new lower boundary of the j-th horizontal line in the new image; S1_Topyk represents the ordinate of the new upper boundary of the k-th horizontal line in the new image, and S1_Bottomyk represents the ordinate of the new lower boundary of the k-th horizontal line in the new image; Topyj represents the ordinate of the original upper boundary of the j-th horizontal line in the original image; Topyk represents the ordinate of the original upper boundary of the k-th horizontal line in the original image;

[0190] In the case where the target detection line is a horizontal line, for the horizontal line q in the new image, the coordinates of the two endpoints are (cps_xq1, cps_yq1), (cps_xq2, cps_yq2), and the second area information corresponding to the vertical coordinates of the two endpoints of the target detection line in the new image can be determined respectively. Specifically, the following formula can be used to confirm whether the vertical coordinates of the two endpoints of the vertical line simultaneously meet the following conditions:

[0191] S1_Topyi≤cps_yq1≤S1_Bottomyi

[0192] S1_Topyi≤cps_yq2≤S1_Bottomyi

[0193] The straight lines that do not meet the condition are deleted. If the condition is met, the horizontal line is mapped to the original image, and the vertical coordinates of the mapped straight line in the original image are obtained, which are:

[0194] fin_yq1=Topyi+cps_y1-S1_Topyi

[0195] fin_yq2=Topyi+cps_x2-S1_Topyi

[0196] Secondly, confirm the first region information corresponding to the horizontal coordinates of the two endpoints. Specifically, the following formula can be used to determine whether the horizontal coordinates of the two endpoints simultaneously meet the following conditions:

[0197] S1_Leftxj≤cps_xq1≤S1_Rightxj

[0198] S1_Leftxk≤cps_xq2≤S1_Rightxk

[0199] Delete the straight lines that do not meet the condition. If the condition is met, the horizontal line is mapped to the original image, and the horizontal coordinates of the straight line in the original image are obtained, which are:

[0200] fin_xq1=Leftxj+cps_yq1-S1_Leftxj

[0201] fin_xq2=Leftxk+cps_yq2-S1_Leftxk

[0202] Among them, (cps_xq1, cps_yq1) represents the abscissa and ordinate of the first endpoint of the qth horizontal line in the new image, (cps_xq2, cps_yq2) represents the abscissa and ordinate of the second endpoint of the qth horizontal line in the new image, (fin_xq1, fin_yq1) represents the abscissa and ordinate of the first endpoint of the qth horizontal line in the new image mapped to the corresponding straight line in the original image, (fin_xq2, fin_yq2) represents the abscissa and ordinate of the second endpoint of the qth horizontal line in the new image mapped to the corresponding straight line in the original image, S1_Topyi represents the ordinate of the new upper boundary of the ith horizontal line in the new image, S1_Bottomyi represents the ordinate of the new lower boundary of the ith horizontal line in the new image; Topyi represents the ordinate of the original upper boundary of the ith horizontal line in the original image; Leftxj represents the abscissa of the original left boundary of the jth vertical line in the original image; Leftxk represents the abscissa of the original left boundary of the kth vertical line in the original image; S1_Leftxj represents the abscissa of the new left boundary of the jth vertical line in the new image; S1_Leftxk represents the abscissa of the new left boundary of the kth vertical line in the new image; S1_Rightxj represents the abscissa of the new right boundary of the jth vertical line in the new image; S1_Rightxk represents the abscissa of the new right boundary of the kth vertical line in the new image.

[0203] By performing straight line mapping according to the above formula, the endpoint coordinates of the horizontal and vertical lines in the new image in the original image can be obtained, thereby achieving accurate detection of the endpoint positions of the straight lines in the image.

[0204] In this embodiment, when detecting the straight lines in the original image, the areas where the horizontal and vertical lines in the original image are located are first determined, and the image area between two adjacent vertical line areas is used as a horizontal deletable area, and the image area between two adjacent horizontal line areas is used as a vertical deletable area. By deleting the vertical deletable area and the horizontal deletable area, the image can be reduced while the straight line resolution and the number of straight lines remain unchanged, that is, by deleting the redundant image areas except the straight line areas in the original image, when detecting the straight lines retained in the new image, the image detection time occupied by detecting the non-straight line areas is avoided, and the speed of straight line detection in the image is improved while ensuring the detection accuracy. In addition, by mapping the horizontal and vertical lines in the new image after the area is deleted to the original image, the position information of the endpoints of each horizontal and vertical line in the original image can be obtained, and the accurate detection of the straight lines in the image is realized.

[0205] Embodiment 2

[0206] Figure 2 A structural block diagram of an image straight line detection device provided in the second embodiment of the present invention, such as Figure 2 As shown, the device includes: a straight line classification module 210, a straight line sorting module 220, a horizontal deletable area determination module 230, a vertical deletable area determination module 240, a straight line detection module 250 and a mapping module 260, wherein:

[0207] A line classification module 210 is configured to identify straight lines in the original image and classify the identified straight lines into horizontal lines and vertical lines;

[0208] The straight line sorting module 220 is configured to sort the vertical lines according to the positions of the horizontal coordinates of the center points of the vertical lines, and to sort the horizontal lines according to the positions of the vertical coordinates of the center points of the horizontal lines;

[0209] The horizontal deletable area determination module 230 is configured to determine the vertical line area information where each vertical line is located according to the endpoint positions of the sorted vertical lines, and use the image area between two adjacent vertical line areas as the horizontal deletable area, wherein the vertical line area information includes the original left boundary position and the original right boundary position of the vertical line area;

[0210] The vertical deletable area determination module 240 is configured to determine the horizontal line area information where each horizontal line is located according to the endpoint positions of the sorted horizontal lines, and use the image area between two adjacent horizontal line areas as the vertical deletable area, wherein the horizontal line area information includes the original upper boundary position and the original lower boundary position of the horizontal line area;

[0211] The line detection module 250 is configured to delete the horizontal deletable area and the vertical deletable area in the original image, and perform line detection on the new image obtained after the area deletion to obtain the endpoint position information of each line in the new image;

[0212] The mapping module 260 is configured to map the endpoint position information of each straight line in the new image to the original image to obtain the endpoint position information of the straight line to which each straight line belongs in the original image.

[0213] Optionally, the horizontal deletable area determination module 230 includes: a horizontal boundary determination unit and a horizontal deletable area determination unit, wherein the horizontal boundary determination unit is specifically configured to:

[0214] The original left boundary position and the original right boundary position of the area where each vertical line is located are determined according to the following formula:

[0215] Leftxi=max(0,min(xti,xbi)-th)

[0216] Rightxi=min(w,max(xti,xbi)+th)

[0217] Among them, xti represents the horizontal coordinate of the first endpoint of the i-th vertical line, xbi represents the horizontal coordinate of the second endpoint of the i-th vertical line, th represents a constant, which represents the set multiple of the straight line width, w represents the width of the original image, Leftxi represents the original left boundary position of the i-th vertical line, and Rightxi represents the original right boundary position of the i-th vertical line.

[0218] Optionally, the vertical deletable area determination module 240 includes: a vertical boundary determination unit and a vertical deletable area determination unit, wherein the vertical boundary confirmation unit is specifically configured to: determine the remaining original upper boundary position and original lower boundary position of each horizontal line according to the following formula:

[0219] Topyi=max(0,min(yli,yri)-th)

[0220] Bottomyi=min(h,max(yli,yri)+th)

[0221] Among them, yli represents the ordinate of the first endpoint of the i-th horizontal line, yri represents the ordinate of the second endpoint of the i-th horizontal line, th represents a constant, which represents the set multiple of the straight line width, h represents the height of the original image, Topyi represents the original upper boundary position of the i-th horizontal line, and Bottomyi represents the original lower boundary position of the i-th horizontal line.

[0222] Optionally, the line detection module 250 includes a region deletion unit and a line detection unit, wherein the line detection unit includes:

[0223] The new boundary position determination subunit is configured to: determine a new left boundary position and a new right boundary position of each horizontal line, and a new upper boundary position and a new lower boundary position of each vertical line in a new image obtained after the region is deleted;

[0224] The line detection subunit is configured to: perform line detection based on the new left boundary position and the new right boundary position of each horizontal line, and the new upper boundary position and the new lower boundary position of each vertical line, and based on the deep line detection model, to obtain the endpoint position information of each vertical line and each horizontal line in the new image.

[0225] Optionally, the new boundary position determination subunit is specifically configured as follows:

[0226] According to the following formula, the new left boundary position and the new right boundary position of each vertical line in the new image obtained after the region is deleted are determined:

[0227]

[0228] S1_Rightxi=S1_Leftxi+Rightxi-Leftxi

[0229] Furthermore, according to the following formula, the new upper boundary position and the new lower boundary position of each horizontal line in the new image obtained after the region is deleted are determined:

[0230]

[0231] S1_Bottomyj=S1_Topyj+Bottomyj-Topyj

[0232] Among them, S1_Leftxi and S1_Rightxi respectively represent the horizontal coordinates of the new left boundary and the new right boundary of the i-th vertical line in the new image, Rightxk and Leftxk respectively represent the horizontal coordinates of the original left boundary and the original right boundary of the k-th vertical line in the original image; S1_Topyj and S1_Bottomyj respectively represent the vertical coordinates of the new upper boundary and the new lower boundary of the j-th horizontal line in the new image, Topyk and Bottomyk respectively represent the horizontal coordinates of the original upper boundary and the original lower boundary of the k-th horizontal line in the original image.

[0233] Optionally, the mapping module 260 includes:

[0234] a corresponding line determination unit, configured to determine, for each target detection line obtained by the depth line detection model, region information where two endpoints of the target detection line are located in the new image, the region information including first region information corresponding to the horizontal coordinates of each endpoint and second region information corresponding to the vertical coordinates of each endpoint;

[0235] The straight line mapping unit is configured to map the target detection straight line to the original image according to the endpoint position information of the target detection straight line, the area information and the original boundary position information corresponding to the target detection straight line in the original image, so as to obtain the endpoint position information of the straight line to which the target detection straight line belongs in the original image.

[0236] Optionally, the linear mapping unit is specifically configured as follows:

[0237] When the target detection line is a vertical line, the first region information corresponding to the horizontal coordinates of the two endpoints of the target detection line in the new image is determined according to the following formula:

[0238] S1_Leftxi≤cps_xp1≤S1_Rightxi

[0239] S1_Leftxi≤cps_xp2≤S1_Rightxi

[0240] Accordingly, the horizontal coordinate of the line to which each target detection line belongs in the original image is determined according to the following formula:

[0241] fin_xp1=Leftxi+cps_xp1-S1_Leftxi

[0242] fin_xp2=Leftxi+cps_xp2-S1_Leftxi

[0243] Furthermore, the second region information corresponding to the ordinates of the two end points of the target detection line in the new image is determined respectively according to the following formula:

[0244] S1_Topyj≤cps_yp1≤S1_Bottomyj

[0245] S1_Topyk≤cps_yp2≤S1_Bottomyk

[0246] Accordingly, the ordinate of the line to which each target detection line belongs in the original image is determined according to the following formula:

[0247] fin_yp1=Topyj+cps_yp1-S1_Topyj

[0248] fin_yp2=Topyk+cps_yp2-S1_Topyk

[0249] Among them, (cps_xp1, cps_yp1) represents the horizontal coordinate and vertical coordinate of the first endpoint of the p-th vertical line in the new image, (cps_xp2, cps_yp2) represents the horizontal coordinate and vertical coordinate of the second endpoint of the p-th vertical line in the new image, (fin_xp1, fin_yp1) represents the horizontal coordinate and vertical coordinate of the first endpoint of the p-th vertical line in the new image mapped to the corresponding straight line in the original image, (fin_xp2, fin_yp2) represents the horizontal coordinate and vertical coordinate of the second endpoint of the p-th vertical line in the new image mapped to the corresponding straight line in the original image, S1_Leftxi represents the horizontal coordinate of the new left boundary of the ith vertical line in the new image, and S1_Rightxi represents the horizontal coordinate of the i-th vertical line. Leftxi represents the abscissa of the new right boundary of the ith vertical line in the new image; Leftxi represents the abscissa of the original left boundary of the ith vertical line in the original image; S1_Topy represents the ordinate of the new upper boundary of the jth horizontal line in the new image, and S1_Bottomyj represents the ordinate of the new lower boundary of the jth horizontal line in the new image; S1_Topyk represents the ordinate of the new upper boundary of the kth horizontal line in the new image, and S1_Bottomyk represents the ordinate of the new lower boundary of the kth horizontal line in the new image; Topyj represents the ordinate of the original upper boundary of the jth horizontal line in the original image; Topyk represents the ordinate of the original upper boundary of the kth horizontal line in the original image;

[0250] When the target detection line is a horizontal line, the second region information corresponding to the ordinates of the two endpoints of the target detection line in the new image is determined according to the following formula:

[0251] S1_Topyi≤cps_yq1≤S1_Bottomyi

[0252] S1_Topyi≤cps_yq2≤S1_Bottomyi

[0253] Accordingly, the ordinate of the line to which each target detection line belongs in the original image is determined according to the following formula:

[0254] fin_yq1=Topyi+cps_yq1-S1_Topyi

[0255] fin_yq2=Topyi+cps_xq2-S1_Topyi

[0256] Furthermore, the first region information corresponding to the horizontal coordinates of the two end points of the target detection line in the new image is determined respectively according to the following formula:

[0257] S1_Leftxj≤cps_xq1≤S1_Rightxj

[0258] S1_Leftxk≤cps_xq2≤S1_Rightxk

[0259] Accordingly, the horizontal coordinate of the line to which each target detection line belongs in the original image is determined according to the following formula:

[0260] fin_xq1=Leftxj+cps_yq1-S1_Leftxj

[0261] fin_xq2=Leftxk+cps_yq2-S1_Leftxk

[0262] Among them, (cps_xq1, cps_yq1) represents the horizontal coordinate and vertical coordinate of the first endpoint of the qth horizontal line in the new image, (cps_xq2, cps_yq2) represents the horizontal coordinate and vertical coordinate of the second endpoint of the qth horizontal line in the new image, (fin_xq1, fin_yq1) represents the horizontal coordinate and vertical coordinate of the first endpoint of the qth horizontal line in the new image mapped to the corresponding straight line in the original image, (fin_xq2, fin_yq2) represents the horizontal coordinate and vertical coordinate of the second endpoint of the qth horizontal line in the new image mapped to the corresponding straight line in the original image, S1_Topyi represents the vertical coordinate of the new upper boundary of the ith horizontal line in the new image, and S1_Bottomyi represents The ordinate of the new lower boundary of the ith horizontal line in the new image; Topyi represents the ordinate of the original upper boundary of the ith horizontal line in the original image; Leftxj represents the abscissa of the original left boundary of the jth vertical line in the original image; Leftxk represents the abscissa of the original left boundary of the kth vertical line in the original image; S1_Leftxj represents the abscissa of the new left boundary of the jth vertical line in the new image; S1_Leftxk represents the abscissa of the new left boundary of the kth vertical line in the new image; S1_Rightxj represents the abscissa of the new right boundary of the jth vertical line in the new image; S1_Rightxk represents the abscissa of the new right boundary of the kth vertical line in the new image.

[0263] Embodiment 3

[0264] See also Figure 3 , Figure 3 This is a schematic diagram of the structure of a computing device provided in Embodiment 3 of the present invention. Figure 3 As shown, the computing device may include:

[0265] A memory 701 storing executable program codes;

[0266] a processor 702 coupled to the memory 701;

[0267] The processor 702 calls the executable program code stored in the memory 701 to execute the image straight line detection method provided by any embodiment of the present invention.

[0268] An embodiment of the present invention discloses a computer-readable storage medium storing a computer program, wherein the computer program enables a computer to execute the image straight line detection method provided by any embodiment of the present invention.

[0269] In various embodiments of the present invention, it should be understood that the size of the serial numbers of the above-mentioned processes does not mean the necessary order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0270] In the embodiments provided by the present invention, it should be understood that "B corresponding to A" means that B is associated with A, and B can be determined according to A. However, it should also be understood that determining B according to A does not mean determining B only according to A, and B can also be determined according to A and / or other information.

[0271] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0272] If the above integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-accessible memory. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product, which is stored in a memory and includes several requests for a computer device (which can be a personal computer, a server or a network device, etc., specifically a processor in a computer device) to perform some or all of the steps of the above methods of various embodiments of the present invention.

[0273] Those skilled in the art can understand that all or part of the steps in the various methods of the above embodiments can be completed by instructing related hardware through a program, and the program can be stored in a computer-readable storage medium, and the storage medium includes a read-only memory (ROM), a random access memory (RAM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), a one-time programmable read-only memory (OTPROM), an electronically erasable rewritable read-only memory (EEPROM), a compact disc (CD-ROM) or other optical disc storage, magnetic disk storage, magnetic tape storage, or any other computer-readable medium that can be used to carry or store data.

[0274] Those skilled in the art can understand that the accompanying drawings are only schematic diagrams of an embodiment, and the modules or processes in the accompanying drawings are not necessarily required to implement the present invention.

[0275] Those skilled in the art can understand that the modules in the device in the embodiment can be distributed in the device in the embodiment according to the description of the embodiment, or can be changed accordingly and located in one or more devices different from the embodiment. The modules in the above embodiment can be combined into one module, or can be further divided into multiple sub-modules.

[0276] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for detecting straight lines in an image, characterized in that: include: Identify the straight lines in the original image and divide the identified straight lines into horizontal lines and vertical lines; Sort the vertical lines according to the positions of the horizontal coordinates of the center points of the vertical lines, and sort the horizontal lines according to the positions of the vertical coordinates of the center points of the horizontal lines; Determine the vertical line area information where each vertical line is located according to the endpoint positions of the sorted vertical lines, and use the image area between two adjacent vertical line areas as a horizontal deletable area, wherein the vertical line area information includes the original left boundary position and the original right boundary position of the vertical line area; Determine the horizontal line area information of each horizontal line according to the endpoint positions of the sorted horizontal lines, and use the image area between two adjacent horizontal line areas as a vertical deletable area, wherein the horizontal line area information includes the original upper boundary position and the original lower boundary position of the horizontal line area; Deleting the horizontal deletable area and the vertical deletable area in the original image, and performing straight line detection on a new image obtained after the area deletion to obtain endpoint position information of each straight line in the new image; The endpoint position information of each straight line in the new image is mapped to the original image to obtain the endpoint position information of the straight line to which each straight line belongs in the original image.

2. The method according to claim 1, characterized in that The step of determining the vertical line region information where each vertical line is located according to the endpoint positions of the sorted vertical lines includes: The original left boundary position and the original right boundary position of the vertical line area where each vertical line is located are determined according to the following formula: Leftxi=max(0,min(xti,xbi)-th) Rightxi=min(w,max(xti,xbi)+th) Among them, xti represents the horizontal coordinate of the first endpoint of the i-th vertical line, xbi represents the horizontal coordinate of the second endpoint of the i-th vertical line, th represents a constant, which represents the set multiple of the straight line width, w represents the width of the original image, Leftxi represents the original left boundary position of the i-th vertical line, and Rightxi represents the original right boundary position of the i-th vertical line.

3. The method according to claim 2, characterized in that The step of determining the horizontal line region information where each horizontal line is located according to the endpoint positions of the sorted horizontal lines includes: The original upper boundary position and the original lower boundary position of the horizontal line area where each horizontal line is located are determined according to the following formula: Topyi=max(0,min(yli,yri)-th) Bottomyi=min(h,max(yli,yri)+th) Among them, yli represents the ordinate of the first endpoint of the i-th horizontal line, yri represents the ordinate of the second endpoint of the i-th horizontal line, th represents a constant, which represents the set multiple of the straight line width, h represents the height of the original image, Topyi represents the original upper boundary position of the i-th horizontal line, and Bottomyi represents the original lower boundary position of the i-th horizontal line.

4. The method according to any one of claims 1 to 3, characterized in that: The performing of straight line detection on the new image obtained after the region deletion to obtain the endpoint position information of each straight line in the new image includes: Determine a new left boundary position and a new right boundary position of each horizontal line, and a new upper boundary position and a new lower boundary position of each vertical line in a new image obtained after the region is deleted; According to the new left boundary position and the new right boundary position of each horizontal line, and the new upper boundary position and the new lower boundary position of each vertical line, and based on the deep line detection model, line detection is performed to obtain the endpoint position information of each vertical line and each horizontal line in the new image.

5. The method according to any one of claims 1 to 4, characterized in that: The new left boundary position and the new right boundary position of each horizontal line, and the new upper boundary position and the new lower boundary position of each vertical line in the new image obtained after the determined area is deleted include: According to the following formula, the new left boundary position and the new right boundary position of each vertical line in the new image obtained after the region is deleted are determined: S1_Rightxi=S1_Leftxi+Rightxi-Leftxi Furthermore, according to the following formula, the new upper boundary position and the new lower boundary position of each horizontal line in the new image obtained after the region is deleted are determined: S1_Bottomyj=S1_Topyj+Bottomyj-Topyj Among them, S1_Leftxi and S1_Rightxi respectively represent the horizontal coordinates of the new left boundary and the new right boundary of the i-th vertical line in the new image, Rightxk and Leftxk respectively represent the horizontal coordinates of the original left boundary and the original right boundary of the k-th vertical line in the original image; S1_Topyj and S1_Bottomyj respectively represent the vertical coordinates of the new upper boundary and the new lower boundary of the j-th horizontal line in the new image, Topyk and Bottomyk respectively represent the horizontal coordinates of the original upper boundary and the original lower boundary of the k-th horizontal line in the original image.

6. The method according to claim 4, characterized in that Mapping the endpoint position information of each straight line in the new image to the original image to obtain the endpoint position information of the straight line to which each straight line belongs in the original image includes: For each target detection line obtained by the depth line detection model, respectively determine the region information where two endpoints of the target detection line are located in the new image, wherein the region information includes first region information corresponding to the horizontal coordinates of each endpoint and second region information corresponding to the vertical coordinates of each endpoint; According to the endpoint position information of the target detection line, the area information and its corresponding original boundary position information in the original image, the target detection line is mapped to the original image to obtain the endpoint position information of the line to which the target detection line belongs in the original image.

7. The method according to any one of claims 1 to 6, characterized in that: The step of mapping the target detection line to the original image based on the endpoint position information of the target detection line, the region information and the original boundary position information corresponding to the target detection line in the original image to obtain the endpoint position information of the line to which the target detection line belongs in the original image includes: When the target detection line is a vertical line, the first region information corresponding to the horizontal coordinates of the two endpoints of the target detection line in the new image is determined according to the following formula: S1_Leftxi≤cps_xp1≤S1_Rightxi S1_Leftxi≤cps_xp2≤S1_Rightxi Accordingly, the horizontal coordinate of the line to which each target detection line belongs in the original image is determined according to the following formula: fin_xp1=Leftxi+cps_xp1-S1_Leftxi fin_xp2=Leftxi+cps_xp2-S1_Leftxi Furthermore, the second region information corresponding to the ordinates of the two end points of the target detection line in the new image is determined respectively according to the following formula: S1_Topyj≤cps_yp1≤S1_Bottomyj S1_Topyk≤cps_yp2≤S1_Bottomyk Accordingly, the ordinate of the line to which each target detection line belongs in the original image is determined according to the following formula: fin_yp1=Topyj+cps_yp1-S1_Topyj fin_yp2=Topyk+cps_yp2-S1_Topyk Among them, (cps_xp1, cps_yp1) represent the horizontal coordinate and vertical coordinate of the first endpoint of the p-th vertical line in the new image, (cps_xp2, cps_yp2) represent the horizontal coordinate and vertical coordinate of the second endpoint of the p-th vertical line in the new image, (fin_xp1, fin_yp1) represent the horizontal coordinate and vertical coordinate of the first endpoint of the p-th vertical line in the new image mapped to the corresponding straight line in the original image, (fin_xp2, fin_yp2) represent the horizontal coordinate and vertical coordinate of the second endpoint of the p-th vertical line in the new image mapped to the corresponding straight line in the original image, S1_Leftxi represents the horizontal coordinate of the new left boundary of the ith vertical line in the new image, and S1_Rightxi represents the horizontal coordinate of the i-th vertical line. Leftxi represents the horizontal coordinate of the new right boundary of the vertical line in the new image; Leftxi represents the horizontal coordinate of the original left boundary of the ith vertical line in the original image; S1_Topyj represents the vertical coordinate of the new upper boundary of the jth horizontal line in the new image, and S1_Bottomyj represents the vertical coordinate of the new lower boundary of the jth horizontal line in the new image; S1_Topyk represents the vertical coordinate of the new upper boundary of the kth horizontal line in the new image, and S1_Bottomyk represents the vertical coordinate of the new lower boundary of the kth horizontal line in the new image; Topyj represents the vertical coordinate of the original upper boundary of the jth horizontal line in the original image; Topyk represents the vertical coordinate of the original upper boundary of the kth horizontal line in the original image; When the target detection line is a horizontal line, the second region information corresponding to the ordinates of the two endpoints of the target detection line in the new image is determined according to the following formula: S1_Topyi≤cps_yq1≤S1_Bottomyi S1_Topyi≤cps_yq2≤S1_Bottomyi Accordingly, the ordinate of the line to which each target detection line belongs in the original image is determined according to the following formula: fin_yq1=Topyi+cps_yq1-S1_Topyi fin_yq2=Topyi+cps_xq2-S1_Topyi Furthermore, the first region information corresponding to the horizontal coordinates of the two end points of the target detection line in the new image is determined respectively according to the following formula: S1_Leftxj≤cps_xq1≤S1_Rightxj S1_Leftxk≤cps_xq2≤S1_Rightxk Accordingly, the horizontal coordinate of the line to which each target detection line belongs in the original image is determined according to the following formula: fin_xq1=Leftxj+cps_yq1-S1_Leftxj fin_xq2=Leftxk+cps_yq2-S1_Leftxk Among them, (cps_xq1, cps_yq1) represents the horizontal coordinate and vertical coordinate of the first endpoint of the qth horizontal line in the new image, (cps_xq2, cps_yq2) represents the horizontal coordinate and vertical coordinate of the second endpoint of the qth horizontal line in the new image, (fin_xq1, fin_yq1) represents the horizontal coordinate and vertical coordinate of the first endpoint of the qth horizontal line in the new image mapped to the corresponding straight line in the original image, (fin_xq2, fin_yq2) represents the horizontal coordinate and vertical coordinate of the second endpoint of the qth horizontal line in the new image mapped to the corresponding straight line in the original image, S1_Topyi represents the vertical coordinate of the new upper boundary of the ith horizontal line in the new image, and S1_Bottomyi represents the vertical coordinate of the qth horizontal line in the new image. The ordinate of the new lower boundary of the i-th horizontal line in the new image; Topyi represents the ordinate of the original upper boundary of the ith horizontal line in the original image; Leftxj represents the ordinate of the original left boundary of the j-th vertical line in the original image; Leftxk represents the ordinate of the original left boundary of the k-th vertical line in the original image; S1_Leftxj represents the ordinate of the new left boundary of the j-th vertical line in the new image; S1_Leftxk represents the ordinate of the new left boundary of the k-th vertical line in the new image; S1_Rightxj represents the ordinate of the new right boundary of the j-th vertical line in the new image; S1_Rightxk represents the ordinate of the new right boundary of the k-th vertical line in the new image.

8. A device for detecting straight lines in an image, characterized in that: include: A straight line classification module is configured to identify straight lines in the original image and classify the identified straight lines into horizontal lines and vertical lines; A straight line sorting module is configured to sort each vertical line according to the position of the horizontal coordinate of the center point of the vertical line, and to sort each horizontal line according to the position of the vertical coordinate of the center point of the horizontal line; a horizontal deletable area determination module, configured to determine the vertical line area information where each vertical line is located according to the endpoint positions of the sorted vertical lines, and use the image area between two adjacent vertical line areas as the horizontal deletable area, wherein the vertical line area information includes the original left boundary position and the original right boundary position of the vertical line area; A vertical deletable area determination module is configured to determine the horizontal line area information where each horizontal line is located according to the endpoint positions of the sorted horizontal lines, and use the image area between two adjacent horizontal line areas as the vertical deletable area, wherein the horizontal line area information includes the original upper boundary position and the original lower boundary position of the horizontal line area; A line detection module is configured to delete the horizontal deletable area and the vertical deletable area in the original image, and perform line detection on a new image obtained after the area deletion to obtain endpoint position information of each straight line in the new image; The mapping module is configured to map the endpoint position information of each straight line in the new image to the original image to obtain the endpoint position information of the straight line to which each straight line belongs in the original image.

9. A computing device, characterized in that The computing device comprises: one or more processors; a storage device for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors implement the image straight line detection method as described in any one of claims 1-7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the image straight line detection method as described in any one of claims 1 to 7 is implemented.

Images