Table line detection method, device, computer device and storage medium

Abstract

The present application relates to a table line detection method, apparatus, computer equipment, storage medium, and computer program product, which can be used in the financial field or other fields. The method comprises: acquiring a table line image, performing a multi-scale transformation on the table line image to obtain a multi-scale image set, obtaining a fused table line image based on the multi-scale image set through region growing and region fusion, and extracting the table lines from the fused table line image through a straight line detection algorithm. By adopting this method, the multi-scale image set obtained by performing a multi-scale transformation on the table line image can show clearer table line features. By performing region growing and region fusion, the multi-scale image set with different features can be fused. This method of extracting table lines based on the straight line detection algorithm combined with multi-scale transformation can improve the accuracy of table line detection.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to a table line detection method, apparatus, computer equipment, storage medium, and computer program product. Background Art

[0002] Tables are essential for recording data, and table images play a crucial role in recording information in daily life. However, with the explosive growth of daily information, standard tables are no longer sufficient. T-shaped tables and unordered tables are increasingly used in work and life. Table image recognition focuses on detecting table lines. Due to the increasing diversity of tables, the connections between table lines are no longer strong. Furthermore, table images often exhibit skew, which impacts the accuracy of table line detection to varying degrees.

[0003] Currently, the table line detection method generally adopts pixel-based table line detection. However, pixel-based table line detection has high requirements on sample quality. For T-shaped tables and unclear table lines, there is a problem of low table line detection accuracy. Summary of the Invention

[0004] Based on this, it is necessary to provide a table line detection method, device, computer equipment, computer-readable storage medium and computer program product that can improve the accuracy of table line detection in order to address the problem of low accuracy of traditional table line detection.

[0005] In a first aspect, the present application provides a table line detection method. The method comprises:

[0006] Get table line image;

[0007] Perform multi-scale transformation on the table line image to obtain a multi-scale image set;

[0008] According to the multi-scale image set, a fused tabular line image is obtained through region growing and region fusion;

[0009] Through the straight line detection algorithm, the table lines are extracted from the fused table image.

[0010] In one embodiment, performing a multi-scale transformation on the table line image to obtain a multi-scale image set includes:

[0011] Get the pixel value of the table line image;

[0012] Perform convolution operation on the pixel values of the table line image and the Gaussian kernel function to obtain the scale space of the table line image;

[0013] Update the standard deviation of the Gaussian kernel function and return to the step of performing a convolution operation on the pixel values of the table line image and the Gaussian kernel function to obtain the scale space of the table line image;

[0014] The corresponding multi-scale images in different scale spaces are collected to obtain a multi-scale image set.

[0015] In one embodiment, obtaining a fused table line image by region growing and region fusion based on the multi-scale image set includes:

[0016] According to the multi-scale image set, a growing region of the multi-scale image set is obtained by region growing;

[0017] According to the growth area of the multi-scale image set, the fused tabular line image is obtained through regional fusion.

[0018] In one embodiment, obtaining a growing region of the multi-scale image set by region growing according to the multi-scale image set includes:

[0019] Get the gradient value of a multi-scale image set;

[0020] Difference is calculated between the gradient value of the current pixel point and the gradient value of the adjacent pixel points in the multi-scale image to obtain the gradient difference;

[0021] If the gradient difference is less than or equal to the preset threshold, the current pixel point is combined with the adjacent pixel point to obtain a combined pixel point;

[0022] Assigning the combined pixel to the current pixel, returning to the step of subtracting the gradient value of the current pixel from the gradient value of the adjacent pixel in the multi-scale image to obtain the gradient difference;

[0023] According to the combined pixel points, the growth area of the multi-scale image is obtained;

[0024] The growth regions of all multi-scale images are aggregated to obtain the growth regions of the multi-scale image set.

[0025] In one embodiment, obtaining a fused table line image by regional fusion based on the growth region of the multi-scale image set includes:

[0026] According to the growth region of the multi-scale image set, a regional angle of the growth region is obtained, and a target growth region is selected in the growth region;

[0027] The difference between the regional angle of the growth area of the multi-scale image set and the regional angle of the target growth area is calculated to obtain the regional angle difference;

[0028] If the regional angle difference is less than or equal to the first preset angle threshold, a screened growth region is obtained;

[0029] The screened growth areas are fused to obtain a fused table line image.

[0030] In one embodiment, extracting table lines from the fused table image using a line detection algorithm includes:

[0031] According to the fused table line image, the line segments in the fused table line image are obtained by using a straight line detection algorithm;

[0032] Get the length of the line segment, the angle of the line segment, and the error warning value of the line segment;

[0033] Take the weighted average of the length of each line segment and the error warning value of the corresponding line segment to obtain the weighted average value corresponding to each line segment;

[0034] Select the line segment corresponding to the largest weighted average value to obtain the target line segment;

[0035] Difference between the angle of the line segment and the angle of the target line segment to obtain the line segment angle difference;

[0036] If the line segment angle difference is less than or equal to the second preset angle threshold, obtaining a first table line segment set;

[0037] If the difference between the line segment angle difference and π / 2 is less than or equal to the second preset angle threshold, obtaining a second table line segment set;

[0038] The first table line segment set and the second table line segment set are aggregated to obtain table lines.

[0039] In a second aspect, the present application further provides a table line detection device. The device comprises:

[0040] An image acquisition module, used for acquiring table line images;

[0041] A multi-scale transformation module is used to perform multi-scale transformation on the table line image to obtain a multi-scale image set;

[0042] An image set fusion module is used to obtain a fused table line image based on a multi-scale image set through region growing and region fusion;

[0043] The table line extraction module is used to extract table lines from the fused table image through a straight line detection algorithm.

[0044] In a third aspect, the present application further provides a computer device. The computer device includes a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the following steps are performed:

[0045] Acquire a table line image; perform multi-scale transformation on the table line image to obtain a multi-scale image set; obtain a fused table line image through region growing and region fusion based on the multi-scale image set; and extract the table lines from the fused table image through a line detection algorithm.

[0046] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the following steps:

[0047] Acquire a table line image; perform multi-scale transformation on the table line image to obtain a multi-scale image set; obtain a fused table line image through region growing and region fusion based on the multi-scale image set; and extract the table lines from the fused table image through a line detection algorithm.

[0048] In a fifth aspect, the present application further provides a computer program product. The computer program product includes a computer program that, when executed by a processor, implements the following steps:

[0049] Acquire a table line image; perform multi-scale transformation on the table line image to obtain a multi-scale image set; obtain a fused table line image through region growing and region fusion based on the multi-scale image set; and extract the table lines from the fused table image through a line detection algorithm.

[0050] The above-mentioned table line detection method, apparatus, computer device, storage medium, and computer program product obtain a table line image, perform a multi-scale transformation on the table line image, obtain a multi-scale image set, and then obtain a fused table line image based on the multi-scale image set through region growing and region fusion. The table lines are then extracted from the fused table line image using a line detection algorithm. In the above-mentioned scheme, the multi-scale image set obtained by performing a multi-scale transformation on the table line image can exhibit clearer table line features. The multi-scale image set with different features can be fused through region growing and region fusion. The table lines in the fused table line image are extracted using a line detection algorithm. The accuracy of the detection of tilted table lines is still high. This method of extracting table lines based on the line detection algorithm combined with multi-scale transformation can improve the accuracy of table line detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] Figure 1 A diagram showing an application environment of a table line detection method in one embodiment;

[0052] Figure 2 1 is a flow chart of a table line detection method according to an embodiment;

[0053] Figure 3Schematic diagram of the process of table line detection step in one embodiment;

[0054] Figure 4 is a flow chart of a table line detection method according to another embodiment;

[0055] Figure 5 Schematic diagram of a sub-process of S640 in one embodiment;

[0056] Figure 6 1 is a schematic diagram of the process of region growing and region fusion in one embodiment;

[0057] Figure 7 is a structural block diagram of a table line detection device in one embodiment;

[0058] Figure 8 FIG. 1 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0059] In order to make the purpose, technical solutions and advantages of this application more clear, the following further describes this application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0060] The table line detection method provided in the embodiment of the present application can be applied to Figure 1 In the application environment shown. The terminal 102 communicates with the server 104 through the network. The data storage system can store the data that the server 104 needs to process. The data storage system can be integrated on the server 104, or it can be placed on the cloud or other network servers. The terminal 102 obtains the table line image, performs multi-scale transformation on the table line image, obtains a multi-scale image set, obtains a fused table line image based on the multi-scale image set through regional growth and regional fusion, and extracts the table lines from the fused table image through a straight line detection algorithm. The terminal 102 can be, but is not limited to, various personal computers, laptops, smart phones, tablets, Internet of Things devices and portable wearable devices. The Internet of Things devices can be smart speakers, smart TVs, smart air conditioners, smart car-mounted devices, etc. Portable wearable devices can be smart watches, smart bracelets, head-mounted devices, etc. The server 104 can be implemented with an independent server or a server cluster consisting of multiple servers.

[0061] In one embodiment, Figure 2 As shown, a table line detection method is provided, which is applied to Figure 1 Taking the terminal 102 in FIG. 1 as an example, the method includes the following steps:

[0062] S200, obtaining a table line image.

[0063] Among them, tables are important carriers of data records, generally composed of parallel and vertical line segments. Table line images are images containing table lines. The core of table image recognition lies in the detection of table lines. The type of table can be a T-type table or an unordered table. This application does not specifically limit the type of table used. The angle of the table lines in the table line image can be parallel or inclined. This application does not specifically limit the angle of the table lines in the table line image. The application scenarios of table line images can include financial statement images, transfer slip images, and voucher images. This application does not limit the application scenarios of table line images.

[0064] Specifically, a table line image is acquired.

[0065] S400, performing multi-scale transformation on the table line image to obtain a multi-scale image set.

[0066] The multi-scale nature of an image refers to its varying resolutions. The larger the image scale, the lower the resolution and the greater the blur. Images exhibit feature invariance at multiple scales. The same key points detected from the same image at different scales are the same. Performing a multi-scale transformation on an image yields a collection of images at different scales. When extracting image features, using fixed-scale feature detection will result in a biased detection result toward that scale, while missing many features at other scales. Detecting at multiple scales allows for the detection of features at different scales.

[0067] Specifically, a multi-scale transformation is performed on the table line image to obtain a multi-scale image set.

[0068] S600 , obtaining a fused table line image through region growing and region fusion according to the multi-scale image set.

[0069] Among them, region growing refers to the process of developing groups of pixels or regions into larger regions, starting from a set of seed points. Region growth from these points is achieved by merging adjacent pixels with similar properties to each seed point, such as intensity, grayscale, texture color, etc. into this region. Region fusion is the process of combining an image or image regions in an image set that meet specific conditions. After the images in the image set have undergone region growing, the image regions that meet specific conditions are combined together to obtain a fused image.

[0070] Specifically, according to the multi-scale image set, a fused table line image is obtained through region growing and region fusion.

[0071] S800, through the straight line detection algorithm, extracts the table lines from the fused table image.

[0072] Among them, line detection is a method used in image detection technology to detect straight lines in an image. Common line detection algorithms include Hough_line, LSD (Line Segment Detector), FLD (FastLine Detector), EDlines (Edge Drawing lines), LSWMS (LineSegment using Weighted Mean-Shift), CannyLines line detection algorithm, MCMLSD (Markov Chain Edge Segment Detector) line detection algorithm and LSM (Log Structured MergeTrees). This application does not specifically limit the type of line detection algorithm used. Through the line detection algorithm, table lines can be extracted from the fused table image.

[0073] Specifically, table lines are extracted from the fused table image through a straight line detection algorithm.

[0074] In the above-mentioned table line detection method, a table line image is obtained, and a multi-scale transformation is performed on the table line image to obtain a multi-scale image set. Based on the multi-scale image set, a fused table line image is obtained through region growing and region fusion. The table lines are extracted from the fused table line image through a line detection algorithm. In the above-mentioned embodiment, the multi-scale image set obtained by performing a multi-scale transformation on the table line image can show clearer table line features. Through region growing and region fusion, the multi-scale image set with different features can be fused. The table lines in the fused table line image are extracted through a line detection algorithm. The accuracy of tilted table line detection is still high. This method of extracting table lines based on a line detection algorithm combined with multi-scale transformation can improve the accuracy of table line detection.

[0075] In one embodiment, Figure 3 As shown, the table line image is multi-scale transformed to obtain a multi-scale image set including:

[0076] S420, obtaining pixel values of the table line image;

[0077] S440, performing a convolution operation on the pixel values of the table line image and the Gaussian kernel function to obtain a scale space of the table line image;

[0078] S460, updating the standard deviation of the Gaussian kernel function, and returning to the step of performing a convolution operation on the pixel values of the table line image and the Gaussian kernel function to obtain the scale space of the table line image;

[0079] S480: Collect corresponding multi-scale images in different scale spaces to obtain a multi-scale image set.

[0080] In this embodiment, the pixel is the calculation unit of the image. The higher the resolution of the image, the more pixels it contains and the clearer the image. Based on the obtained image, the pixel value of the image can be obtained. The scale space method incorporates the traditional single-scale visual information processing technology into a dynamic analysis framework with constantly changing scales. Therefore, it is easier to obtain the essential characteristics of the image. The purpose of generating the scale space is to simulate the multi-scale characteristics of the image data, and the Gaussian convolution kernel is the only linear kernel to achieve scale transformation. The Gaussian kernel function (Gaussian kernel), also known as the radial basis function, is a scalar function that is radially symmetric. The core idea of the Gaussian kernel function is to map each sample point to an infinite-dimensional feature space, so that the originally linearly inseparable data can be linearly separable. The Gaussian kernel function is defined as:

[0081]

[0082] Get the pixel value I(x,y) of the table line image, and compare the pixel value of the table line image with the Gaussian kernel function G σ Perform convolution operation to obtain the scale space I of the table line image σ (x,y), where I σ (x,y)=I(x,y)*G σ (x, y), σ is the standard deviation of the Gaussian kernel function, that is, the scale of the image, update the standard deviation of the Gaussian kernel function, return to the step of convolving the pixel values of the table line image with the Gaussian kernel function to obtain the scale space of the table line image, that is, update the scale of the image, obtain the updated Gaussian kernel function, bring the updated Gaussian kernel function into the scale space calculation formula, obtain scale spaces corresponding to different scales, each scale space corresponds to a scale of the table line image, collect the multi-scale images corresponding to different scale spaces, and obtain a multi-scale image set.

[0083] The solution of the above embodiment obtains the pixel values of the table line image, performs a convolution operation on the pixel values of the table line image and the Gaussian kernel function to obtain the scale space of the table line image, updates the standard deviation of the Gaussian kernel function, obtains a multi-scale space, and aggregates the corresponding multi-scale images in different scale spaces to obtain a multi-scale image set. This image processing method based on multi-scale space transformation can present the characteristics of table line images at multiple scales, which is conducive to the detection of features in table line images of different scales and improves the accuracy of table line detection.

[0084] In one embodiment, Figure 4 As shown, according to the multi-scale image set, through region growing and region fusion, the fused tabular line image is obtained, including:

[0085] S620, obtaining a growing region of the multi-scale image set by region growing according to the multi-scale image set;

[0086] S640 , obtaining a fused table line image by performing regional fusion according to the growth region of the multi-scale image set.

[0087] In this embodiment, region growing is performed on a multiscale image set to obtain a growth region for the multiscale image set. Specifically, region growing is performed on the table line images at each scale in the multiscale image set to obtain a growth region for each scale table line. The growth regions of all scale table lines are then aggregated to obtain the growth region for the multiscale image set. Images exhibit different characteristics at different scales, and growth regions at a single scale are easily affected by noise. Based on the growth regions of the multiscale image set, region fusion is performed to obtain a fused table line image. Specifically, the growth regions of table line images at different scales are fused at the same location to obtain the fused table line image.

[0088] The solution of the above embodiment obtains the growth area of the multi-scale image set through region growing based on the multi-scale image set, and obtains the fused table line image through region fusion based on the growth area of the multi-scale image set. It can fuse the different feature areas in the multi-scale table line image set. The obtained fused table line image has clear features compared with the table line image before the multi-scale transformation, which is conducive to the detection of the table line features and improves the accuracy of table line detection.

[0089] In one embodiment, obtaining a growth region of a multiscale image set through region growing based on a multiscale image set includes: obtaining a gradient value of the multiscale image set; subtracting the gradient value of a current pixel point in the multiscale image from the gradient values of adjacent pixels to obtain a gradient difference; if the gradient difference is less than or equal to a preset threshold, combining the current pixel point and the adjacent pixels to obtain a combined pixel point; assigning the combined pixel point to the current pixel point, and returning to the step of subtracting the gradient value of the current pixel point in the multiscale image from the gradient values of adjacent pixels to obtain the gradient difference; obtaining a growth region of the multiscale image based on the combined pixel point; and aggregating the growth regions of all multiscale images to obtain the growth region of the multiscale image set.

[0090] In this embodiment, a gradient value of a multiscale image set is obtained. Specifically, based on the obtained multiscale image set, a gradient value at each pixel in each scaled table line image is obtained. A gradient vector at each pixel can also be obtained. A current pixel in the multiscale image is selected, and there are eight adjacent pixels around the current pixel. The gradient value of the current pixel in the multiscale image is subtracted from the gradient value of each adjacent pixel to obtain a plurality of gradient differences. If the gradient difference is less than or equal to a preset threshold, the current pixel and the adjacent pixel are combined to obtain a combined pixel. The step of subtracting the gradient value of the current pixel in the multiscale image from the gradient values of the adjacent pixels to obtain the gradient difference is returned. Pixels whose gradient difference is less than or equal to the preset threshold are continuously combined to obtain a growth region of the multiscale image. The growth regions of all multiscale images are aggregated to obtain the growth region of the multiscale image set.

[0091] The solution of the above embodiment obtains the gradient value of the multi-scale image set, calculates the difference between the gradient value of the current pixel point in the multi-scale image and the gradient value of the adjacent pixel point, determines the size of the gradient difference and the preset threshold, combines the pixel points whose gradient difference is less than or equal to the preset threshold, executes the loop, obtains the combined pixel points, and then obtains the growth area of the multi-scale image set. This method performs pixel point aggregation by determining the size of the gradient difference between the current pixel point and the adjacent pixel point and the preset threshold. The obtained growth area of the multi-scale image set has clear features compared to the table line image before the multi-scale transformation, which is conducive to the detection of the table line features and improves the accuracy of table line detection.

[0092] In one embodiment, Figure 5 As shown, according to the growth area of the multi-scale image set, through regional fusion, the fused tabular line image is obtained, including:

[0093] S642, obtaining a region angle of the growth region according to the growth region of the multi-scale image set, and selecting a target growth region in the growth region;

[0094] S644, calculating the difference between the regional angle of the growth region of the multi-scale image set and the regional angle of the target growth region to obtain a regional angle difference value;

[0095] S646, if the regional angle difference is less than or equal to the first preset angle threshold, obtaining a screened growth region;

[0096] S648, fusing the screened growth areas to obtain a fused table line image.

[0097] In this embodiment, based on the growth area of the multi-scale image set, the regional angle of the growth area is obtained, and the target growth area is selected from the growth area. The regional angle of each multi-scale image is subtracted from the regional angle of the target growth area, and the obtained regional angle difference is compared with the preset first angle threshold. The growth area with a regional angle difference greater than the first preset angle threshold is regarded as an irrelevant area and eliminated, and the growth area with a regional angle difference less than or equal to the first preset angle threshold is screened out to obtain the screened growth area. After the screened growth area is fused, the final fused table line image is obtained.

[0098] The solution of the above embodiment obtains the regional angle of the growth area of the multi-scale image set and selects the target growth area. According to the difference between the regional angle of the growth area of the multi-scale image set and the regional angle of the target growth area, the growth area whose regional angle difference is less than or equal to the first preset angle threshold is screened out and fused. The obtained fused table line image has clear features compared with the table line image before multi-scale transformation, which is conducive to the detection of table line features and improves the accuracy of table line detection.

[0099] In one embodiment, extracting table lines from the fused table image through a straight line detection algorithm includes: obtaining line segments in the fused table line image through a straight line detection algorithm based on the fused table line image; obtaining the length of the line segment, the angle of the line segment, and the error warning value of the line segment; taking a weighted average of the length of each line segment and the error warning value of the corresponding line segment to obtain the weighted average value corresponding to each line segment; selecting the line segment corresponding to the largest weighted average value to obtain the target line segment; subtracting the angle of the line segment from the angle of the target line segment to obtain the line segment angle difference; if the line segment angle difference is less than or equal to a second preset angle threshold, obtaining a first table line segment set; if the difference between the line segment angle difference and π / 2 is less than or equal to the second preset angle threshold, obtaining a second table line segment set; combining the first table line segment set and the second table line segment set to obtain table lines.

[0100] In this embodiment, based on the fused table line image, a straight line detection algorithm is used to detect the line segments in the fused table line image, obtain the length and angle of the line segments, construct a circumscribed rectangle for each area in the fused table line image, and calculate the error warning value (NFA) of each line segment. When the error warning value of the line segment is less than or equal to the preset warning value threshold, the line segment is determined to be a straight line segment, otherwise it is not a straight line segment. Since the table lines in the table line image are generally parallel or perpendicular, first determine the most likely line segment as the target table line, then use the target table line as the baseline and compare it with other straight line segments to determine whether the other straight line segments are approximately parallel or approximately perpendicular to the target table line, thereby obtaining the filtered table line. Specifically, a preset weight is set for the error warning value and line segment length of each line segment. According to the preset weight, a weighted average is taken for the error warning value and line segment length of each line segment, and the line segment corresponding to the largest weighted average is selected as the target line segment. Calculate the difference between the angle of each line segment and the angle of the target line segment, determine the line segment angle difference and compare it with the second preset angle threshold, group the line segments whose line segment angle difference is less than or equal to the second preset angle threshold as the first table line segment set, the first table line segment set is also the line segments that are approximately parallel to the target table line, group the line segments whose angle difference is less than or equal to π / 2 as the second table line segment set, the second table line segment set is also the line segments that are approximately perpendicular to the target table line, group the first table line segment set and the second table line segment set to obtain the table line.

[0101] The scheme of the above embodiment uses a straight line detection algorithm to extract line segments from the fused table line image, and takes a weighted average of the length of each line segment and the error warning value of the corresponding line segment, and selects the line segment corresponding to the largest weighted average as the target line segment. According to the angle difference between the line segment and the target line segment, if the line segment angle difference is less than or equal to the second preset angle threshold, a first table line segment set approximately parallel to the target table line is obtained; if the difference between the line segment angle difference and π / 2 is less than or equal to the second preset angle threshold, a second table line segment set approximately perpendicular to the target table line is obtained, and the first table line segment set and the second table line segment set are combined to obtain the table line. Based on multi-scale transformation, the table line obtained by this method has clear features, which is conducive to the detection of table line features, can effectively reduce missed detections, and has a high detection accuracy for scenes with inclined table lines and non-closed table lines, thereby improving the accuracy of table line detection.

[0102] To illustrate the table line detection method and effect in this solution in detail, the following is a most detailed embodiment:

[0103] For the common financial report image detection scenario in the financial field, the financial report includes table lines, obtain the financial report image I and the pixel I(x,y) value of the financial report image, and compare the pixel value of the financial report image with the Gaussian kernel function G σ Perform convolution operation to obtain the scale space I of the financial statement image σ (x,y), where I σ (x,y)=I(x,y)*G σ (x, y), the Gaussian kernel standard deviation σ can be set to multiple different values to generate multiple financial statement images of different scales, collect the corresponding multi-scale images in different scale spaces, obtain the multi-scale image set, and obtain the gradient value at the pixel point (x, y) in the multi-scale image

[0104]

[0105]

[0106]

[0107]

[0108] Among them, I x is the horizontal gradient, I y is the vertical gradient and LLA is the gradient vector.

[0109] The gradient value of the current pixel point in the multi-scale image is subtracted from the gradient value of the adjacent pixel point to obtain a gradient difference. If the gradient difference is less than or equal to a preset threshold, the current pixel point and the adjacent pixel point are combined to obtain a combined pixel point, and the combined pixel point is assigned to the current pixel point. The step of subtracting the gradient value of the current pixel point in the multi-scale image from the gradient value of the adjacent pixel point to obtain the gradient difference is returned. Based on the combined pixel point, a growth region of the multi-scale image is obtained, the growth regions of all multi-scale images are aggregated to obtain the growth region of the multi-scale image set, and based on the growth region of the multi-scale image set, a regional angle θ of the growth region is obtained:

[0110]

[0111] Among them, S x is the horizontal pixel length of the growing area, S y is the vertical pixel length of the growing area.

[0112] Select the target growth area in the growth area, and set the regional angle θ of the growth area of the multi-scale image set i The angle θ with the target growth area RCalculate the difference to obtain the regional angle difference. If the regional angle difference is less than or equal to the first preset angle threshold τ, the screened growth area is obtained.

[0113] |θ R -θ i |≤τ

[0114] The screened growth areas are fused to obtain the fused financial statement image. Figure 6 Shown is a schematic diagram of the process of region growing and region fusion.

[0115] According to the fused financial statement image, the line segments in the fused financial statement image are obtained through the line detection algorithm. A bounding rectangle is constructed for each area in the fused financial statement image, and the error warning value (NFA) of each line segment is calculated. If the NFA is less than or equal to the preset warning value threshold ε, it is recorded as: NFA≤ε, then the line segment is determined to be a straight line segment. Get the length of the line segment and the angle of the line segment, and the length of each line segment and the corresponding line segment error warning value NFA i Do weighted averaging to obtain the weighted average value p corresponding to each line segment i , where f1 and f2 are the preset weights of NFA and line segment length respectively,

[0116]

[0117] Select the line segment corresponding to the largest weighted average value, obtain the target line segment L, and set the angle θ of the line segment j Angle θ with the target segment L Calculate the difference to obtain the line segment angle difference. If the line segment angle difference is less than or equal to the second preset angle threshold δ, obtain the first table line segment set.

[0118] |θ L -θ j |≤δ

[0119] If the difference between the line segment angle difference and π / 2 is less than or equal to the second preset angle threshold, a second table line segment set is obtained.

[0120] |θ L -θ i -π / 2|≤δ

[0121] The first table line segment set and the second table line segment set are aggregated to obtain table lines.

[0122] The method of the present application can also be used for transfer bill image detection scenarios in the financial field. The transfer bill includes table lines. Based on the transfer bill image, a multi-scale image set is obtained through multi-scale transformation. A fused transfer bill image is obtained through region growing and region fusion. The table lines are extracted from the fused transfer bill image using a line detection algorithm. It should be noted that the table line detection method and device disclosed in this application can also be used for table line detection in other scenarios, and the scenarios to which the table line detection method and device disclosed in this application are applicable are not limited.

[0123] The above-mentioned table line detection method obtains a table line image, performs a multi-scale transformation on the table line image, obtains a multi-scale image set, obtains a fused table line image based on the multi-scale image set through region growing and region fusion, and extracts the table lines from the fused table line image through a line detection algorithm. In the above-mentioned embodiment, the multi-scale image set obtained by performing a multi-scale transformation on the table line image can show clearer table line features. Through region growing and region fusion, the multi-scale image set with different features can be fused. The table lines in the fused table line image can be extracted through a line detection algorithm. The accuracy of the detection of tilted table lines is still high. This method of extracting table lines based on the line detection algorithm combined with multi-scale transformation can improve the accuracy of table line detection.

[0124] It should be understood that, although the various steps in the flowcharts involved in the various embodiments described above are displayed in sequence according to the instructions of the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and these steps can be executed in other orders. Moreover, at least a portion of the steps in the flowcharts involved in the various embodiments described above can include multiple steps or multiple stages, and these steps or stages are not necessarily executed and completed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a portion of steps or stages in other steps.

[0125] Based on the same inventive concept, embodiments of the present application also provide a table line detection device for implementing the aforementioned table line detection method. The solution provided by this device is similar to the solution described in the aforementioned method. Therefore, the specific limitations of one or more table line detection device embodiments provided below can be found in the above-described limitations of the table line detection method and will not be further elaborated here.

[0126] In one embodiment, Figure 7As shown, a table line detection device 100 is provided, comprising: an image acquisition module 120, a multi-scale transformation module 140, an image set fusion module 160 and a table line extraction module 180, wherein:

[0127] The image acquisition module 120 is used to acquire the table line image.

[0128] The multi-scale transformation module 140 is used to perform multi-scale transformation on the table line image to obtain a multi-scale image set.

[0129] The image set fusion module 160 is used to obtain a fused table line image through region growing and region fusion based on the multi-scale image set.

[0130] The table line extraction module 180 is used to extract table lines from the fused table image using a line detection algorithm.

[0131] The above-mentioned table line detection device obtains a table line image, performs a multi-scale transformation on the table line image, obtains a multi-scale image set, and obtains a fused table line image based on the multi-scale image set through region growing and region fusion. The table lines are extracted from the fused table line image through a line detection algorithm. In the above-mentioned embodiment, the multi-scale image set obtained by performing a multi-scale transformation on the table line image can show clearer table line features. The multi-scale image set with different features can be fused through region growing and region fusion. The table lines in the fused table line image are extracted through a line detection algorithm. The accuracy of the detection of tilted table lines is still high. This method of extracting table lines based on the line detection algorithm and combined with multi-scale transformation can improve the accuracy of table line detection.

[0132] In one embodiment, the multi-scale transformation module 140 is also used to obtain pixel values of the table line image; perform a convolution operation on the pixel values of the table line image and the Gaussian kernel function to obtain the scale space of the table line image; update the standard deviation of the Gaussian kernel function, and return to the step of performing a convolution operation on the pixel values of the table line image and the Gaussian kernel function to obtain the scale space of the table line image; and aggregate the corresponding multi-scale images under different scale spaces to obtain a multi-scale image set.

[0133] In one embodiment, the image set fusion module 160 is further configured to obtain a growing region of the multi-scale image set by region growing based on the multi-scale image set; and obtain a fused table line image by region fusion based on the growing region of the multi-scale image set.

[0134] In one embodiment, the image set fusion module 160 is further used to obtain a gradient value of a multi-scale image set; subtract the gradient value of a current pixel point in the multi-scale image from the gradient value of an adjacent pixel point to obtain a gradient difference; if the gradient difference is less than or equal to a preset threshold, combine the current pixel point and the adjacent pixel point to obtain a combined pixel point; assign the combined pixel point to the current pixel point, and return to the step of subtracting the gradient value of the current pixel point in the multi-scale image from the gradient value of an adjacent pixel point to obtain a gradient difference; obtain a growth area of the multi-scale image based on the combined pixel point; and aggregate the growth areas of all multi-scale images to obtain the growth area of the multi-scale image set.

[0135] In one embodiment, the image set fusion module 160 is also used to obtain the regional angle of the growth area based on the growth area of the multi-scale image set, and select the target growth area in the growth area; subtract the regional angle of the growth area of the multi-scale image set from the regional angle of the target growth area to obtain the regional angle difference; if the regional angle difference is less than or equal to the first preset angle threshold, the screened growth area is obtained; the screened growth area is fused to obtain a fused table line image.

[0136] In one embodiment, the table line extraction module 180 is also used to obtain line segments in the fused table line image through a straight line detection algorithm based on the fused table line image; obtain the length of the line segment, the angle of the line segment and the error warning value of the line segment; take a weighted average of the length of each line segment and the error warning value of the corresponding line segment to obtain the weighted average value corresponding to each line segment; select the line segment corresponding to the largest weighted average value to obtain the target line segment; calculate the difference between the angle of the line segment and the angle of the target line segment to obtain the line segment angle difference; if the line segment angle difference is less than or equal to the second preset angle threshold, obtain a first table line segment set; if the difference between the line segment angle difference and π / 2 is less than or equal to the second preset angle threshold, obtain a second table line segment set; and combine the first table line segment set and the second table line segment set to obtain a table line.

[0137] Each module in the table line detection device can be implemented in whole or in part through software, hardware, or a combination thereof. Each module can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in the computer device in the form of software, so that the processor can call and execute the corresponding operations of each module.

[0138] In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as follows: Figure 8As shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, abbreviated as I / O) and a communication interface. The processor, memory and input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and computer program in the non-volatile storage medium. The database of the computer device is used to store table line images and multi-scale image sets. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a table line detection method is implemented.

[0139] Those skilled in the art will understand that Figure 8 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0140] In one embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the computer program, the following steps are implemented:

[0141] Acquire a table line image; perform multi-scale transformation on the table line image to obtain a multi-scale image set; obtain a fused table line image through region growing and region fusion based on the multi-scale image set; and extract the table lines from the fused table image through a line detection algorithm.

[0142] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the following steps are implemented:

[0143] Acquire a table line image; perform multi-scale transformation on the table line image to obtain a multi-scale image set; obtain a fused table line image through region growing and region fusion based on the multi-scale image set; and extract the table lines from the fused table image through a line detection algorithm.

[0144] In one embodiment, a computer program product is provided, comprising a computer program, which, when executed by a processor, implements the following steps:

[0145] Acquire a table line image; perform multi-scale transformation on the table line image to obtain a multi-scale image set; obtain a fused table line image through region growing and region fusion based on the multi-scale image set; and extract the table lines from the fused table image through a line detection algorithm.

[0146] It should be noted that the table line detection method and device disclosed in the present application can be used for table line detection in the financial field, and can also be used for table line detection in any field other than the financial field. The application field of the table line detection method and device disclosed in the present application is not limited. The user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data need to comply with relevant laws, regulations and standards of relevant countries and regions.

[0147] Those skilled in the art will appreciate that all or part of the processes in the above-mentioned embodiment methods can be implemented by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, database or other media used in the embodiments provided in this application may include at least one of non-volatile and volatile memory. Non-volatile memory may include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory may include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in the various embodiments provided herein may include at least one of a relational database and a non-relational database. Non-relational databases may include, but are not limited to, distributed databases based on blockchains. The processor involved in the various embodiments provided herein may be, but are not limited to, a general-purpose processor, a central processing unit, a graphics processing unit, a digital signal processor, a programmable logic unit, a data processing logic unit based on quantum computing, and the like.

[0148] The technical features of the above embodiments can be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0149] The above-described embodiments merely represent several implementation methods of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present application. It should be noted that a person of ordinary skill in the art may make various modifications and improvements without departing from the spirit of the present application, and these modifications and improvements fall within the scope of protection of the present application. Therefore, the scope of protection of the present application shall be determined by the appended claims.

Claims

1. A table line detection method, characterized in that: The method comprises: S1: Get table line image; S2: performing a multi-scale transformation on the table line image to obtain a multi-scale image set; S3: Difference the gradient value of the current pixel in the multi-scale image with the gradient value of the adjacent pixel to obtain the gradient difference; S4: combining the current pixel point and the adjacent pixel point whose gradient difference is less than or equal to a preset threshold to obtain a combined pixel point; S5: Assigning the combined pixel point to the current pixel point, and returning to execute steps S3 to S5 above until all pixel points whose gradient difference is less than or equal to the preset threshold are combined; Obtaining a growth region of the multi-scale image according to the combined pixel points; Aggregating the growth regions of all the multi-scale images to obtain the growth region of the multi-scale image set; According to the growth region of the multi-scale image set, obtaining the region angle of the growth region, and selecting a target growth region in the growth region; Difference is calculated between the regional angle of the growth region of the multi-scale image set and the regional angle of the target growth region to obtain a regional angle difference value; Merging the growth regions whose regional angle difference is less than or equal to a first preset angle threshold to obtain a fused table line image; According to the fused table line image, obtaining line segments in the fused table line image by a line detection algorithm; Taking a weighted average of the length of each line segment and the error warning value of the corresponding line segment, and selecting the line segment corresponding to the largest weighted average as the target line segment; A first table line segment set in which the angle difference between the line segment and the target line segment is less than or equal to a second preset angle threshold and a second table line segment set in which the difference between the angle difference and π / 2 is less than or equal to the second preset angle threshold are collected to obtain a table line.

2. The table line detection method according to claim 1, characterized in that: The performing multi-scale transformation on the table line image to obtain a multi-scale image set comprises: S21: Obtain pixel values of the table line image; S22: performing a convolution operation on the pixel values of the table line image and a Gaussian kernel function to obtain a scale space of the table line image; S23: Update the standard deviation of the Gaussian kernel function, and return to execute the above steps S22 to S23 until the corresponding multi-scale images in different scale spaces are collected to obtain a multi-scale image set.

3. The table line detection method according to claim 1, characterized in that: Also includes: Obtaining gradient values of the multi-scale image set.

4. The table line detection method according to claim 1, characterized in that: Also includes: Obtaining the length of the line segment, the angle of the line segment, and an error warning value of the line segment; The weighted average of the length of each line segment and the error warning value of the corresponding line segment, and selecting the line segment corresponding to the largest weighted average as the target line segment, includes: Taking a weighted average of the length of each line segment and the error warning value of the corresponding line segment to obtain a weighted average value corresponding to each line segment; The line segment corresponding to the largest weighted average value is selected to obtain a target line segment.

5. A table line detection device, the table line detection device being used to implement the table line detection method according to any one of claims 1 to 4, characterized in that: The device comprises: An image acquisition module, used for acquiring table line images; A multi-scale transformation module, configured to perform multi-scale transformation on the table line image to obtain a multi-scale image set; An image set fusion module, configured to obtain a fused table line image by region growing and region fusion according to the multi-scale image set; The table line extraction module is used to extract the table lines from the fused table image using a straight line detection algorithm.

6. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 4 are implemented.

7. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 4 are implemented.

8. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 4 are implemented.

Images