A form structure extraction method, device and equipment and a storage medium

Abstract

This invention discloses a method, apparatus, device, and storage medium for extracting form structure. The method includes: acquiring a target form image; obtaining a binary image from the target form image using edge detection and morphological algorithms; automatically locating the starting position of the form in the binary image; performing cell convolution on the binary image based on the starting position to obtain cell positions; updating the starting position using the obtained cell positions; repeatedly using cell convolution to divide the remaining cell positions in the form, and marking the parallel and vertical relationships between cells; and analyzing the hierarchical relationships of the cells in the form to obtain the form structure. By utilizing the feedback from image convolution, the form cells are automatically split, and the hierarchical relationships between cells are analyzed, thereby extracting the form structure.

Description

Technical Field

[0001] This invention relates to a method, apparatus, device, and storage medium for extracting form structures, belonging to the field of data processing technology. Background Technology

[0002] Forms are composed of cells of varying sizes and semantic levels. Each cell contains some text content. When the text is too close to the cell boundary, simple edge segmentation algorithms (such as Canny) are difficult to effectively extract the cell boundary. Furthermore, deep learning algorithms such as TableOCR rely on GPUs to perform table recognition calculations, making it difficult to deploy these algorithms quickly and efficiently on CPU platforms. Summary of the Invention

[0003] Image convolution typically uses square kernels to filter and process image patch features, with the weights in the kernel determining the processing result. Deformable convolution, by learning coordinate compensation, controls the shape and size of the kernel, thereby capturing features with more extreme aspect ratios. Various cell shapes are well-suited for using deformable kernels to process image features, thus enabling the capture of different cell features within a form using convolution computation.

[0004] Objective: In view of at least one of the above technical problems, the present invention provides a method, apparatus, device and storage medium for extracting form structure, which utilizes the feedback of convolution calculation to automatically determine the boundaries of different cells in the form, and effectively and quickly segment the form without relying on a deep learning framework.

[0005] Technical solution: To solve the above technical problems, the technical solution adopted by the present invention is as follows:

[0006] In a first aspect, the present invention provides a method for extracting form structures, the method comprising:

[0007] Obtain the target form image; wherein, the form image is an image containing a form structure;

[0008] Binary images are obtained from the target form image using edge detection and morphological algorithms;

[0009] Automatically locate the starting position of the form in the binary image;

[0010] Based on the starting position of the form, perform a cell convolution operation on the binary image to obtain the cell position;

[0011] Update the starting position using the obtained cell position, repeatedly use cell convolution operation to divide the remaining cell positions in the form, and mark the parallel and vertical relationships between cells;

[0012] Based on the parallel and vertical relationships between cells, the hierarchical relationship of cells in the form is analyzed to obtain the form structure.

[0013] In some embodiments, a binary image is obtained from the target form image using edge detection and morphological algorithms, including:

[0014] Edge detection is performed on the target form image to obtain an edge-detected image;

[0015] Enhanced feature maps are obtained by using morphological algorithms to enhance the features of edge detection images.

[0016] The enhanced feature map is converted into a binary image using a filtering mechanism.

[0017] In some embodiments, the edge detection employs the Sobel edge detection algorithm.

[0018] In some embodiments, if the target form image is a color form image, before performing edge detection on the target form image to obtain an edge detection image, the color form image is further converted into a grayscale image.

[0019] In some embodiments, a filtering mechanism is used to convert the enhanced feature map into a binary image, including:

[0020]

[0021] in This represents the pixel value in a binary image, while This represents the pixel value of the enhanced feature map, and the subscript (x, y) represents the coordinates of the pixel in the image. The filtering mechanism sets the pixel value less than or equal to 8 in the enhanced feature map to 0, and sets the pixel value greater than 8 in the enhanced feature map to 1.

[0022] In some embodiments, automatically locating the starting position of the form in the binary image includes:

[0023] A morphological opening operation with a specific window size is used to obtain information on all long lines in the form; wherein the long lines include horizontal and vertical lines with a length greater than Q;

[0024] Get the endpoint coordinates of all long lines in the form based on the information of all long lines in the form;

[0025] Select the coordinates of the endpoints of all long lines in the form that are closest to the top left corner of the target form image as the starting position of the form;

[0026] The method employs morphological opening operations with a specific window size to retrieve long lines from a form, including:

[0027] The first window is set as the sampling window for the morphological opening operation, which captures horizontal lines with a length greater than Q in the binary image from the horizontal direction; where the length of the first window is Q and the height is 1.

[0028] The second window is set as the sampling window for the morphological opening operation, which captures vertical lines with a length greater than Q in the binary image from the vertical direction; the length of the second window is 1 and the height is Q.

[0029] By summarizing all the captured horizontal and vertical line information, we can obtain information on all long straight lines in the form.

[0030] In some embodiments, the cell convolution operation includes:

[0031] Based on the starting position, the current receptive field and its size are determined on the binary image by locating the cell boundaries horizontally and vertically; wherein the receptive field size includes the width and height of the receptive field.

[0032] Based on the receptive field size, obtain the corresponding convolutional kernel; wherein the receptive field size includes the width and height of the receptive field;

[0033] The convolution result is obtained by performing a convolution operation based on the receptive field and the corresponding convolution kernel.

[0034] The cell confidence score is obtained by dividing the convolution result by the perimeter of the receptive field window; wherein the perimeter of the receptive field window is determined based on the width and height of the receptive field.

[0035] If the confidence score of a cell is greater than the first threshold, then the current receptive field is considered to be a cell.

[0036] In some embodiments, determining the current receptive field and its size on the binary image based on the starting position by horizontally locating cell boundaries and vertically locating cell boundaries includes:

[0037] Construct the initial receptive field based on the starting position and the initial value of the receptive field size;

[0038] S4-2. Horizontally position the cell boundaries to determine the width of the receptive field:

[0039] Based on the initial receptive field, the width of the receptive field is continuously expanded horizontally to the right until the horizontal boundary of the current cell is reached, at which point the horizontal expansion stops. The result is a receptive field width that meets the criteria, and the receptive field is updated. Specifically, this includes:

[0040] Set the pixels of the bottom row of the receptive field to 1, and execute the first loop step until the width of the receptive field that meets the standard is obtained. The first loop step includes:

[0041] S4-2-1. Perform average pooling on all pixels on the right boundary of the current receptive field using the first pooling window to obtain the first average pooling value: wherein the height of the first pooling window is the height of the current receptive field and the width is 1.

[0042] If the first average pooling value is not greater than the second threshold, the receptive field width is increased by 1, the pixel value of the right boundary of the receptive field is updated, and S4-2-1 is returned.

[0043] If the first average pooling value is greater than the second threshold, execute S4-2-2;

[0044] S4-2-2: Obtain the corresponding convolution kernel based on the current receptive field size; perform a convolution operation based on the receptive field and the corresponding convolution kernel to obtain the convolution result; divide the convolution result by the perimeter of the receptive field window to obtain the cell confidence score; if the cell confidence score is not greater than the first threshold, increment the receptive field width by 1, update the pixel value of the right boundary of the receptive field, and return to S4-2-1; if the cell confidence score is greater than the first threshold, consider the right boundary of the current receptive field to be the true right boundary of the cell, terminate the first loop step, and return the width of the current receptive field.

[0045] S4-3. Vertically locate the cell boundaries to determine the height of the receptive field;

[0046] Based on the receptive field obtained in S4-2, the height of the receptive field is continuously expanded vertically downwards until the vertical boundary of the current cell is reached, at which point the vertical expansion stops, the height of the receptive field that meets the criteria is returned, and the receptive field of the current cell size is updated; specifically including:

[0047] Set the pixels in the right row of the updated receptive field to 1, and execute the second loop step until the height of the receptive field that meets the standard is obtained, wherein the second loop step includes:

[0048] S4-3-1. Perform average pooling on all pixels at the bottom boundary of the current receptive field using the second pooling window to obtain the second average pooling value, wherein the width of the second pooling window is the width of the current receptive field and the height is 1.

[0049] If the second average pooling value is not greater than the third threshold, increment the receptive field height by 1, update the pixel value at the bottom boundary of the receptive field, and return to S4-3-1;

[0050] If the second average pooling value is greater than the third threshold, execute S4-3-2;

[0051] S4-3-2. Obtain the corresponding convolution kernel based on the current receptive field size; perform a convolution operation based on the receptive field and the corresponding convolution kernel to obtain the convolution result; divide the convolution result by the perimeter of the receptive field window to obtain the cell confidence score.

[0052] If the cell confidence score is not greater than the first threshold, increment the receptive field height by 1, update the pixel value of the bottom boundary of the receptive field, and return S4-3-1;

[0053] If the cell confidence score is greater than the first threshold, the right boundary of the current receptive field is considered to be the true bottom boundary of the cell, the second loop step is terminated, and the height of the current receptive field is returned.

[0054] In some embodiments, the starting position is updated using the obtained cell position, the cell convolution operation is repeatedly used to divide the remaining cell positions in the form, and the parallel and vertical relationships between cells are marked, including:

[0055] S5-1, Find the right linked cell

[0056] The top-right coordinate of the first cell B1 (top-left corner of the form) is used as the new starting position and included in the convolution operation of cell S4; if a new cell is returned, the new cell is recorded as B. 1-left1 This is the right-linked cell of cell B1; if it does not return, then terminate S5-1 and proceed to S5-2;

[0057] B 1-left1 Substitute the convolution operation in cell S4 to obtain more right-linked cells and repeat this type of operation until no results are returned. Find all cells parallel to cell B1 by repeating the operation in S5-1.

[0058] S5-2 Finding the cell with the next link

[0059] The bottom left coordinate of the first cell B1 in the top left corner of the form is used as the new starting position and is put into the convolution operation of cell S4 for calculation; if a new cell is returned, the new cell B2 is recorded as the lower link cell of cell B1; if no cell is returned, S5-2 is terminated.

[0060] After obtaining cell B2, repeat step S5-1 to find the right linked cell, in order to obtain the right linked cell that is parallel to cell B2.

[0061] By repeating the operations of S5-1 to find the right-linked cell and S5-2 to find the bottom-linked cell, the positions of all cells in the form are captured from both the right and down directions, and the parallel and vertical relationships between cells are marked.

[0062] In some embodiments, the hierarchical relationship of cells in the form is analyzed based on the parallel and vertical relationships between cells to obtain the form structure, including:

[0063] For parallel cells, if the information difference between two cells is less than the fourth threshold, the two parallel cells are determined to be at the same level; if the information difference between two cells is greater than the fourth threshold, and the length of cell A is shorter than the length of cell B, then cell A is the superior of cell B.

[0064] For the cells above and below, if B i There are no parallel cells, and B i+1 If there are parallel cells, then the default is B. i For all B i+1 The parent of the parallel cell; if B i and B i+1 If there are parallel cells, then B i and B i+1 Equal level;

[0065] The method for obtaining the information gap between the two cells includes:

[0066] By using a convolution kernel with all weights equal to 1 corresponding to the cell size, the cells are convolved to obtain the information content of the corresponding cells;

[0067] For parallel cells, calculate the information gap between the two cells based on the amount of information in each cell.

[0068] Furthermore, for parallel cells, methods for obtaining the information gap between two cells include:

[0069] By using a convolution kernel with all weights equal to 1 corresponding to the cell size, the cells are convolved to obtain the information content of the corresponding cells;

[0070] Based on the different cell sizes, construct a convolution kernel with all weights equal to 1:

[0071]

[0072] By performing a convolution on each cell using this kernel, the information content of the cell can be obtained:

[0073]

[0074] Where R conv1 Unit represents the amount of information contained in a cell. (x,y) Represents the pixels in each cell;

[0075] For parallel cells, calculate the information gap between the two cells based on the amount of information in each cell.

[0076] In parallel cells, observing from left to right, let R be the information content of the adjacent left cell A. conv1-A The information content of cell B on the right is R.conv1-B Calculate the information gap between cells:

[0077]

[0078] Dist info W represents the information gap between cell A and cell B. A and W B These are the widths of cell A and cell B, respectively.

[0079] Secondly, the present invention provides a form structure extraction device, the device comprising:

[0080] The acquisition module is used to acquire a target form image; wherein the form image is an image containing a form structure;

[0081] The image processing module is used to obtain a binary image from the target form image using edge detection and morphological algorithms;

[0082] The positioning module is used to automatically locate the starting position of the form in the binary image;

[0083] The cell convolution operation module is used to perform cell convolution operation on the binary image according to the starting position of the form to obtain the cell position;

[0084] The repositioning module is used to update the starting position using the obtained cell position, repeatedly use cell convolution operation to divide the remaining cell positions in the form, and mark the parallel and vertical relationships between cells;

[0085] The form structure building unit is used to analyze the hierarchical relationship of cells in the form based on the parallel and vertical relationships between cells, and to obtain the form structure.

[0086] Thirdly, the present invention provides an apparatus comprising,

[0087] Memory;

[0088] processor;

[0089] as well as

[0090] Computer programs;

[0091] The computer program is stored in the memory and configured to be executed by the processor to implement the method described in the first aspect above.

[0092] Fourthly, the present invention provides a storage medium having a computer program stored thereon, which, when executed by a processor, implements the method described in the first aspect.

[0093] Beneficial Effects: The form structure extraction method, apparatus, device, and storage medium provided by this invention have the following advantages: This invention utilizes the feedback from convolution calculations to automatically determine the boundaries of different cells in a form, effectively and quickly segmenting the form without relying on a deep learning framework. By utilizing the feedback from image convolution, it automatically splits the form's cells and assists in analyzing the hierarchical relationships between cells, thereby extracting the form structure. Attached Figure Description

[0094] Figure 1 This is a schematic diagram illustrating an application scenario of a form structure extraction method according to an embodiment of the present invention;

[0095] Figure 2 This is a schematic diagram of a method for extracting form structure according to an embodiment of the present invention;

[0096] Figure 3 This is a schematic diagram of a data processing apparatus according to an embodiment of the present invention;

[0097] Figure 4 This is a schematic diagram of an electronic device according to an embodiment of the present invention;

[0098] Figure 5 This is a schematic diagram of a binary image according to an embodiment of the present invention;

[0099] Figure 6 This is a schematic diagram illustrating the hierarchical relationship between cells in a form according to an embodiment of the present invention. Detailed Implementation

[0100] The present invention will be further described below with reference to the accompanying drawings and embodiments. The following embodiments are only used to more clearly illustrate the technical solutions of the present invention, and should not be used to limit the scope of protection of the present invention.

[0101] like Figure 1 As shown, before detailing the embodiments of the present invention, a specific scenario example is given to illustrate the scenarios in which the technical solution of the present invention can be used. The method of the present invention can be configured in... Figure 1 The server in the system can communicate with multiple clients.

[0102] These multiple clients can be configured with data acquisition devices to collect form images from each client. The clients then transmit the collected form images to a storage device via the network.

[0103] The server is used to extract the form structure from form images stored in the storage device. Specifically, the server can obtain a target form image from the storage device based on the form structure extraction command sent by the client. The form image is an image containing a form structure. Based on the obtained target form image, the server uses edge detection and morphological algorithms to obtain a binary image. The server automatically locates the starting position of the form in the binary image. Based on the starting position, the server performs a cell convolution operation on the binary image to obtain the cell positions. The server updates the starting position using the obtained cell positions, repeats the cell convolution operation to divide the remaining cell positions in the form, and marks the parallel and vertical relationships between cells. Based on the parallel and vertical relationships between cells, the server analyzes the hierarchical relationship of the cells in the form to obtain the form structure.

[0104] The server can send the form structure of the target form image to the target client, that is, to the client that issues the extraction command for the form structure, so that the user can view it.

[0105] The aforementioned server can be an electronic device with a certain computing power. For example, a server can be a distributed system server, or a system with multiple processors, memory, network communication modules, etc., working together. A server can also be a cloud server, or an intelligent cloud computing server or intelligent cloud host with artificial intelligence technology. A server can also be a server cluster formed by several servers. Alternatively, with the development of science and technology, a server can also be a new technological means capable of realizing the corresponding functions of the implementation method described in the specification. For example, it can be a new form of "server" based on quantum computing.

[0106] The aforementioned client can be an electronic device with network access capabilities. Specifically, for example, the terminal can be a desktop computer, tablet computer, laptop computer, smartphone, etc. Alternatively, the terminal can also be software that can run on the electronic device.

[0107] The aforementioned network can be any type of network, which can use any of the various available protocols (including but not limited to TCP / IP, SNA, IPX, etc.) to support data communication. One or more networks can be a Local Area Network (LAN), an Ethernet-based network, a Token Ring network, a Wide Area Network (WAN), the Internet, a Virtual Network, a Virtual Private Network (VPN), an Intranet, an Extranet, a Public Switched Telephone Network (PSTN), an Infrared Network, a Wireless Network (e.g., Bluetooth, Wi-Fi), and / or any combination of these and / or other networks.

[0108] Example 1

[0109] Firstly, such as Figure 2As shown, this embodiment provides a method for extracting form structure, including:

[0110] S1. Obtain the target form image; wherein, the form image is an image containing a form structure;

[0111] S2. Obtain a binary image from the target form image using edge detection and morphological algorithms;

[0112] In this embodiment, obtaining a binary image from the target form image using edge detection and morphological algorithms includes:

[0113] S201. Perform edge detection on the target form image to obtain an edge detection image;

[0114] S202. Use morphological algorithms to enhance the features of the edge detection image to obtain an enhanced feature map;

[0115] S203. Use a filtering mechanism to convert the enhanced feature map into a binary image.

[0116] Furthermore, in S201, the edge detection adopts the Sobel edge detection algorithm, but other edge detection methods (such as Canny) can also be used.

[0117] Furthermore, in some embodiments, if the target form image is a color form image, before performing edge detection on the target form image to obtain an edge detection image, the color form image is further converted into a grayscale image.

[0118] Therefore, in this embodiment, the input color form image first needs to be converted into a grayscale image using an image library such as OpenCV, and then Sobel edge detection is used to extract the outline structure of the form in the grayscale image. Since the desired feature is a regional form feature, Sobel detection performs better than other edge detection algorithms.

[0119] Then, the edge detection image is enhanced using the closing operation in morphology, making the lines in the form more cohesive. Finally, the enhanced feature map is converted into a binary image (e.g., ...) through certain filtering methods. Figure 5 ).

[0120] In some embodiments, S203, converting the enhanced feature map into a binary image using a filtering mechanism, includes:

[0121]

[0122] in This represents the pixel value in a binary image, while This represents the pixel value of the enhanced feature map, and the subscript (x, y) represents the coordinates of the pixel in the image. The filtering mechanism sets the pixel value less than or equal to 8 in the enhanced feature map to 0, and sets the pixel value greater than 8 in the enhanced feature map to 1.

[0123] Sobel edge detection spreads pixel values ​​to neighboring areas, and the filtering mechanism removes noise points that may exist in the original image, ensuring that subsequent convolution calculations can proceed.

[0124] S3. Automatically locate the starting position of the form in the binary image;

[0125] In some embodiments, S3, automatically locating the starting position of the form in the binary image, includes:

[0126] Use morphological opening operations with a specific window size to obtain information about all long lines in the form;

[0127] Get the endpoint coordinates of all long lines in the form based on the information of all long lines in the form;

[0128] Select the coordinates of the endpoints of all long lines in the form that are closest to the top left corner of the target form image as the starting position of the form.

[0129] Furthermore, the morphological opening operation with a specific window size is used to obtain long lines in the form, including:

[0130] The method employs morphological opening operations with a specific window size to retrieve long lines from a form, including:

[0131] The first window is set as the sampling window for the morphological opening operation, which captures horizontal lines with a length greater than Q in the binary image from the horizontal direction; where the length of the first window is Q and the height is 1.

[0132] The second window is set as the sampling window for the morphological opening operation, which captures vertical lines with a length greater than Q in the binary image from the vertical direction; the length of the second window is 1 and the height is Q.

[0133] By summarizing all the captured horizontal and vertical line information, we can obtain information on all long straight lines in the form.

[0134] In this embodiment, Q is set to 800.

[0135] Because it's an automatic algorithm, it's necessary to locate the starting position of the form within the image, specifically the top left corner. Here, a morphological opening operation with a specific window size is used to capture long straight lines within the form. Two windows are created, one with a length of 800 and a height of 1, and the other with a length of 1 and a height of 800. The morphological opening operation is used to capture lines within the form. Setting the window size to 800 is to filter out interference from long text; a window that is too small would capture straight lines within the text, causing positioning errors.

[0136] The first window is set as the sampling window for the morphological opening operation, capturing horizontal lines longer than 800 from the binary image of the form. Then, the second window is set as the sampling window, and the morphological opening operation is used again to capture vertical lines longer than 800. The extracted line information from the two operations is summarized to obtain information on all long lines in the form. The coordinates of the endpoints of all lines are obtained, and the coordinate with the shortest distance relative to the top-left corner of the image is taken as the starting position of the form.

[0137] Although the horizontal and vertical opening operations can outline the straight lines of a form, text interference and vertical form divisions can make the obtained straight line information incomplete. Therefore, the opening operation can only obtain the form boundaries under ideal conditions and is only used here to locate the starting position of the form.

[0138] S4. Based on the starting position of the form, perform a cell convolution operation on the binary image to obtain the cell position;

[0139] By repeatedly performing cell convolution operations based on different starting positions, cells of different positions and sizes can be obtained.

[0140] S4-1 Cell Convolution

[0141] First, we need to define the cell convolution that runs through the following operations. The purpose of cell convolution is to capture cell border information and filter the text information in the middle.

[0142] Based on the receptive field size, obtain the corresponding convolution kernel. The size of the convolution kernel is the same as that of the receptive field. The convolution kernel is represented by a matrix as follows:

[0143]

[0144] The weights of all four edges of the convolution kernel are set to 1, and the weights of the rest in the middle are set to 0.

[0145] The weights of the convolution kernel are set to a fixed value, which ensures that the effective boundary pixels remain at 1, while the text pixels in the middle become 0.

[0146] The convolution result R is calculated based on the receptive field and the corresponding convolution kernel. conv :

[0147] R conv =∑Field (x,y) Kernel (x,y)

[0148] Where Field (x,y) This represents the pixel value at different locations within the receptive field.

[0149] Dividing the convolution result by the perimeter of the receptive field window yields the cell confidence score.

[0150]

[0151] If Confidence is greater than 0.95, the current receptive field is considered to be a cell.

[0152] In some embodiments, the cell convolution operation in step S4 includes:

[0153] Based on the starting position, the current receptive field and its size are determined on the binary image by horizontally locating the cell boundary using S4-2 and vertically locating the cell boundary using S4-3; wherein the size of the receptive field includes the width and height of the receptive field.

[0154] Based on the receptive field size, obtain the corresponding convolutional kernel; wherein the receptive field size includes the width and height of the receptive field;

[0155] The convolution result is obtained by performing a convolution operation based on the receptive field and the corresponding convolution kernel.

[0156] The cell confidence score is obtained by dividing the convolution result by the perimeter of the receptive field window; wherein the perimeter of the receptive field window is determined based on the width and height of the receptive field.

[0157] If the confidence score of a cell is greater than the first threshold (0.95), then the current receptive field is considered to be a cell.

[0158] In some embodiments, determining the current receptive field and its size on the binary image based on the starting position by horizontally locating cell boundaries and vertically locating cell boundaries includes:

[0159] Construct the initial receptive field based on the starting position and the initial value of the receptive field size;

[0160] S4-2. Horizontally position the cell boundaries to determine the width of the receptive field:

[0161] Based on the initial receptive field, the width of the receptive field is continuously expanded horizontally to the right until the horizontal boundary of the current cell is reached, at which point the horizontal expansion stops. The result is a receptive field width that meets the criteria, and the receptive field is updated. Specifically, this includes:

[0162] Here, the receptive field height is set to 50 by default (basically all cell heights are greater than 50), and the initial width of the receptive field is also set to 50. The receptive field expands horizontally, and the pixels in the bottom row of the receptive field are set to 1 (the reason for setting the bottom row to 1 is that the current operation continuously expands the receptive field horizontally until the boundary of the current cell is reached. Since the horizontal expansion is based on the top-left vertex, followed by the vertical expansion, in this step, during horizontal expansion, the actual pixels in the bottom row of the image corresponding to the receptive field are mostly 0. To ensure the convolution meets the requirements, they are all directly set to 1). The initial receptive field is an image window with a starting position at the top-left corner, a width of 50, and a height of 50.

[0163] Cell convolution can be used to determine whether the current receptive field is a cell, but in order to reduce the amount of convolution calculation operations, the boundaries of the valid cells need to be known in advance.

[0164] Set the pixels of the bottom row of the receptive field to 1, and execute the first loop step until the width of the receptive field that meets the standard is obtained. The first loop step includes:

[0165] S4-2-1. Perform average pooling on all pixels at the right boundary of the current receptive field using the first pooling window to obtain the first average pooling value R. avg-pooling-right The height of the first pooling window is the current receptive field height (50 here), and the width is 1.

[0166]

[0167] Where R avg-pooling-right This is the first average pooling value. For each pixel within the right boundary of the current receptive field;

[0168] If R avg-pooling-right If the value is not greater than the second threshold (0.90), increase the receptive field width by 1 (the receptive field size becomes 51*50), update the pixel value at the right boundary of the receptive field, and return to S4-2-1; repeat the above steps until R... avg-pooling-right If the value exceeds the second threshold, execute S4-2-2;

[0169] S4-2-2: Obtain the corresponding convolution kernel based on the current receptive field size; perform a convolution operation based on the receptive field and the corresponding convolution kernel to obtain the convolution result; divide the convolution result by the perimeter of the receptive field window to obtain the cell confidence score; if the cell confidence score is not greater than the first threshold, increment the receptive field width by 1, update the pixel value of the right boundary of the receptive field, and return to S4-2-1; if the cell confidence score is greater than the first threshold, consider the right boundary of the current receptive field to be the true right boundary of the cell, terminate the first loop step, return the current width of the receptive field, and proceed to S4-3.

[0170] S4-3. Vertically locate the cell boundaries to determine the height of the receptive field;

[0171] Based on the receptive field obtained in S4-2, the height of the receptive field is continuously expanded vertically downwards until the vertical boundary of the current cell is reached, at which point the vertical expansion stops, the height of the receptive field that meets the criteria is returned, and the receptive field of the current cell size is updated; specifically including:

[0172] Set the pixels in the right row of the updated receptive field to 1, and execute the second loop step until the height of the receptive field that meets the standard is obtained, wherein the second loop step includes:

[0173] S4-3-1. Perform average pooling on all pixels at the bottom boundary of the current receptive field using the second pooling window to obtain the second average pooling value R. avg-pooling-bottom The width of the second pooling window is the current receptive field width, and the height is 1.

[0174]

[0175] Where R avg-pooling-bottom This is the second average pooling value. For each pixel in the bottom boundary of the current receptive field;

[0176] If R avg-pooling-bottom If the value is not greater than the third threshold (0.75), increment the receptive field height by 1, update the pixel value at the bottom boundary of the receptive field, and return to S4-3-1;

[0177] If R avg-pooling-bottom If the value exceeds the third threshold, execute S4-3-2;

[0178] S4-3-2. Obtain the corresponding convolution kernel based on the current receptive field size; perform a convolution operation based on the receptive field and the corresponding convolution kernel to obtain the convolution result; divide the convolution result by the perimeter of the receptive field window to obtain the cell confidence score.

[0179] If the cell confidence score is not greater than the first threshold, increment the receptive field height by 1, update the pixel value of the bottom boundary of the receptive field, and return S4-3-1;

[0180] If the cell confidence score is greater than the first threshold, the right boundary of the current receptive field is considered to be the true bottom boundary of the cell, the second loop step is terminated, and the height of the current receptive field is returned.

[0181] In this implementation, after obtaining the actual right boundary in S4-2, it is necessary to further obtain the bottom boundary of the cell vertically. At this time, the width W of the receptive field is fixed, that is, the width of the receptive field returned by S4-2, and the initial value of the height is set to 50, so the size of the receptive field becomes W*50. Similar to S4-2, average pooling is performed on the pixels of the bottom boundary of the receptive field using a window with a width of W and a height of 1:

[0182] If R avg-pooling-bottom If the value is greater than 0.75, the bottom boundary of the current receptive field is considered to be the boundary of the real cell; otherwise, the receptive field height is increased by 1, the receptive field size becomes W*51, the pixel value of the bottom boundary is updated, and the calculation of S4-3 is repeated until the standard of 0.75 is met.

[0183] Returning the receptive field size that meets the criteria, obtaining the corresponding convolution kernel, and substituting it into S4-1 for calculation, then determining if it is a cell. If it is a cell, continue with operation S5, at which point the boundary information of a complete cell has been obtained; otherwise, continue trying to increase the receptive field height. If a suitable boundary cannot be found, the operation terminates.

[0184] After the first cell convolution operation, the position of the first cell located in the top left corner of the form is obtained.

[0185] S5. Update the starting position using the obtained cell position, repeat the cell convolution operation to divide the remaining cell positions in the form, and mark the parallel and vertical relationships between cells;

[0186] In some embodiments, in step S5, the starting position is updated using the obtained cell position, the cell convolution operation is repeatedly used to divide the remaining cell positions in the form, and the parallel and vertical relationships between cells are marked, including:

[0187] S5-1, Find the right linked cell

[0188] The top-right coordinate of the first cell B1 in the top-left corner of the form is used as the new starting position and is added to the convolution operation in cell S4 for calculation.

[0189] If a new cell is returned, denote the new cell as B. 1-left1 This is the right-linked cell of cell B1; if it does not return, then terminate S5-1 and proceed to S5-2;

[0190] B 1-left1Substitute the convolution operation in cell S4 to obtain more right-linked cells and repeat this type of operation until no results are returned. Find all cells parallel to cell B1 by repeating the operation in S5-1.

[0191] S5-2 Finding the cell with the next link

[0192] The bottom left coordinate of the first cell B1 in the top left corner of the form is used as the new starting position and is put into the convolution operation of cell S4 for calculation; if a new cell is returned, the new cell B2 is recorded as the lower link cell of cell B1; if no cell is returned, S5-2 is terminated.

[0193] After obtaining cell B2, repeat step S5-1 to find the right linked cell to obtain the right linked cell parallel to cell B2 (there may not be one).

[0194] By repeating the operations of S5-1 to find the right-linked cell and S5-2 to find the bottom-linked cell, the positions of all cells in the form are captured from both the right and down directions, and the parallel and vertical relationships between cells are marked.

[0195] S6. Based on the parallel and vertical relationships between cells, analyze the hierarchical relationship of cells in the form to obtain the form structure.

[0196] In some embodiments, S6, based on the parallel and vertical relationships between cells, analyzes the hierarchical relationship of cells in the form to obtain the form structure, including:

[0197] S6-1. For parallel cells, calculate the information content of each cell, and calculate the information gap (Dist) between the two cells based on the information content of the two cells. info ;

[0198] Based on the different cell sizes, construct a convolution kernel with all weights equal to 1:

[0199]

[0200] By performing a convolution on each cell using this kernel, the information content of the cell can be obtained:

[0201] R conv1 =∑Unit (x,y) Kernel-1 (x,y)

[0202] Where R conv1 Unit represents the amount of information contained in a cell. (x,y) Represents the pixels in each cell.

[0203] In parallel cells, observing from left to right, let R be the information content of the adjacent left cell A.conv1-A The information content of cell B on the right is R. conv1-B Calculate the information gap between cells:

[0204]

[0205] Dist info W represents the information gap between cell A and cell B. A and W B These are the widths of cell A and cell B, respectively.

[0206] S6-2 Constructing the Form Structure

[0207] For parallel cells, if the information gap between two cells is Dist info If the information difference between the two parallel cells is less than 0.3, then the two parallel cells are considered to be at the same level; if the information difference between the two cells is less than 0.3, then the two parallel cells are considered to be at the same level. info If the value is greater than 0.3, and the length of cell A is shorter than the length of cell B, then cell A is the parent of cell B; if Dist... info If the value is greater than 0.3 and the length of cell A is not shorter than the length of cell B, then the relationship is unknown.

[0208] For the cells above and below, if B i There are no parallel cells, and B i+1 If there are parallel cells, then the default is B. i For all B i+1 The parent of a parallel cell; such as B. i There are no parallel cells, and B i+1 If there are no parallel cells, the relationship is unknown; if B i and B i+1 If there are parallel cells, then B i and B i+1 Equal in rank.

[0209] Using the above two criteria, the hierarchical relationship of cells in a form can be automatically analyzed (e.g., ...). Figure 6 (As shown).

[0210] Note: It should be noted that in step S4, S4-1 explains the principle of cell convolution, but no actual convolution is performed at this time. Cell convolution is only performed when the receptive field is the size of the cell. In this embodiment, the initial receptive field is 50*50 (or other sizes). Based on this initial receptive field, the horizontal receptive field is continuously expanded in S4-2. The horizontal expansion stops when the horizontal boundary of the cell is reached (the horizontal boundary is actually the right edge of the cell). In S4-3, the vertical receptive field is continuously expanded to obtain the vertical boundary of the cell (the vertical boundary is actually the bottom edge of the cell). At this point, the receptive field is the size of the cell. Cell convolution is performed on the receptive field of this cell size to obtain the cell confidence score. If the threshold is met, it means that the current "cell" is an actual cell, which can also be a cell that can be seen by the naked eye in the image, rather than a "pseudo cell" caused by noise from edge detection output.

[0211] For table images, especially those with a lot of text, the output after edge detection often contains a lot of noise and the contours are distorted, making it impossible to accurately extract the table boundaries. This noise may be due to the text being too dense or the table itself being too dense. Therefore, in this embodiment, we construct the receptive field of the real cell and then perform convolution (it should be noted that the convolution kernel is a convolution kernel with 1s on all sides and 0s in the center to preserve the boundary information of the cell and eliminate interference from the internal text) to extract the cells, thereby eliminating noise and achieving accurate cell extraction.

[0212] Example 2

[0213] Secondly, based on Example 1, such as Figure 3 As shown, this embodiment provides a form structure extraction device, including:

[0214] The acquisition module is used to acquire a target form image; wherein the form image is an image containing a form structure;

[0215] The image processing module is used to obtain a binary image from the target form image using edge detection and morphological algorithms;

[0216] The positioning module is used to automatically locate the starting position of the form in the binary image;

[0217] The cell convolution operation module is used to perform cell convolution operation on the binary image according to the starting position of the form to obtain the cell position;

[0218] The repositioning module is used to update the starting position using the obtained cell position, repeatedly use cell convolution operation to divide the remaining cell positions in the form, and mark the parallel and vertical relationships between cells;

[0219] The form structure building unit is used to analyze the hierarchical relationship of cells in the form based on the parallel and vertical relationships between cells, and to obtain the form structure.

[0220] Example 3

[0221] Thirdly, based on Example 1, such as Figure 4 As shown, this embodiment provides a device, including,

[0222] Memory;

[0223] processor;

[0224] as well as

[0225] Computer programs;

[0226] The computer program is stored in the memory and configured to be executed by the processor to implement the method described in Embodiment 1.

[0227] Example 4

[0228] Fourthly, based on Embodiment 1, this embodiment provides a storage medium on which a computer program is stored, and when the computer program is executed by a processor, it implements the method described in Embodiment 1.

[0229] Example 5

[0230] Fifthly, based on Embodiment 1, this embodiment proposes a hardware system for extracting a form structure using the method described in Embodiment 1. This method is applied on the server side.

[0231] A terminal can be an electronic device with network access capabilities. For example, a terminal can be a desktop computer, tablet computer, laptop computer, smartphone, digital assistant, shopping guide terminal, television set, etc.

[0232] A server can be an electronic device with a certain computing power. For example, a server can be a distributed system server, or a system with multiple processors, memory, network communication modules, etc., working together. A server can also be a cloud server, or an intelligent cloud computing server or intelligent cloud host with artificial intelligence technology. A server can also be a server cluster formed by several servers. Alternatively, with the development of science and technology, a server can also be a new technological means capable of realizing the corresponding functions of the implementation method described in the manual. For example, it can be a new form of "server" based on quantum computing.

[0233] Example 6:

[0234] Sixthly, based on Embodiment 1, this embodiment also provides a computer program product containing instructions that, when executed by a computer, cause the computer to perform a form structure extraction method as described in Embodiment 1.

[0235] It is understood that the specific examples in this document are only intended to help those skilled in the art better understand the embodiments described herein, and are not intended to limit the scope of the invention.

[0236] It is understood that in the various embodiments described in this specification, the sequence number of each process does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments described in this specification.

[0237] It is understood that the various implementation methods described in this specification can be implemented individually or in combination, and the implementation methods in this specification are not limited in this respect.

[0238] Unless otherwise stated, all technical and scientific terms used in the embodiments of this specification have the same meaning as commonly understood by one of ordinary skill in the art. The terminology used in this specification is for the purpose of describing particular embodiments only and is not intended to limit the scope of this specification. The term "and / or" as used in this specification includes any and all combinations of one or more of the associated listed items. The singular forms "a," "the," and "the" as used in the embodiments of this specification and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise.

[0239] It is understood that the processor in the embodiments of this specification can be an integrated circuit chip with signal processing capabilities. In implementation, each step of the above method embodiments can be completed by integrated logic circuits in the processor's hardware or by instructions in software form. The processor can be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this specification. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of this specification can be directly implemented by a hardware decoding processor, or by a combination of hardware and software modules in the decoding processor. The software modules can reside in random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. This storage medium is located in memory; the processor reads information from the memory and, in conjunction with its hardware, completes the steps of the above methods.

[0240] It is understood that the memory in the embodiments of this specification may be volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. Non-volatile memory may be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. Volatile memory may be random access memory (RAM). It should be noted that the memory in the systems and methods described herein is intended to include, but is not limited to, these and any other suitable types of memory.

[0241] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this specification.

[0242] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the aforementioned method implementations, and will not be repeated here.

[0243] In the several embodiments provided in this specification, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0244] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment, depending on actual needs.

[0245] In addition, the functional units in the various embodiments of this specification can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0246] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solutions of this specification, in essence, or the parts that contribute to the prior art, or parts of the technical solutions, can be embodied in the form of software products. These computer software products are stored in a storage medium and include several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this specification. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0247] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the technical principles of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A method for extracting form structure, characterized in that, The method includes: Obtain the target form image; wherein, the form image is an image containing a form structure; A binary image is obtained from the target form image using edge detection and morphological algorithms; Automatically locate the starting position of the form in the binary image; Based on the starting position of the form, a cell convolution operation is performed on the binary image to obtain the cell position. The cell convolution operation includes: determining the current receptive field and its size on the binary image by horizontally and vertically locating the cell boundaries based on the starting position; wherein the receptive field size includes the width and height of the receptive field; obtaining the corresponding convolution kernel based on the receptive field size; performing a convolution operation based on the receptive field and the corresponding convolution kernel to obtain the convolution result; dividing the convolution result by the perimeter of the receptive field window to obtain the cell confidence score; wherein the perimeter of the receptive field window is determined based on the width and height of the receptive field; if the cell confidence score is greater than a first threshold, the current receptive field is considered a cell. Update the starting position using the obtained cell position, repeatedly use cell convolution operation to divide the remaining cell positions in the form, and mark the parallel and vertical relationships between cells; Based on the parallel and vertical relationships between cells, the hierarchical relationship of cells in the form is analyzed to obtain the form structure.

2. The method for extracting form structure according to claim 1, characterized in that, Obtain a binary image from the target form image using edge detection and morphological algorithms, including: Edge detection is performed on the target form image to obtain an edge-detected image; Enhanced feature maps are obtained by using morphological algorithms to enhance the features of edge detection images. The enhanced feature map is converted into a binary image using a filtering mechanism.

3. The method for extracting form structure according to claim 2, characterized in that, The edge detection uses the Sobel edge detection algorithm; And / or, if the target form image is a color form image, before performing edge detection on the target form image to obtain an edge detection image, the method further includes converting the color form image into a grayscale image; And / or, using a filtering mechanism to convert the enhanced feature map into a binary image, including: in This represents the pixel value in a binary image, while This represents the pixel value of the enhanced feature map, and the subscript (x, y) represents the coordinates of the pixel in the image. The filtering mechanism sets the pixel value less than or equal to 8 in the enhanced feature map to 0, and sets the pixel value greater than 8 in the enhanced feature map to 1.

4. The method for extracting form structure according to claim 1, characterized in that, Automatically locating the starting position of the form in the binary image includes: A morphological opening operation with a specific window size is used to obtain information on all long lines in the form; wherein the long lines include horizontal and vertical lines with a length greater than Q; Get the endpoint coordinates of all long lines in the form based on the information of all long lines in the form; Select the coordinates of the endpoints of all long lines in the form that are closest to the top left corner of the target form image as the starting position of the form; The method of using a morphological opening operation with a specific window size to obtain long lines in a form includes: The first window is set as the sampling window for the morphological opening operation, which captures horizontal lines with a length greater than Q in the binary image from the horizontal direction; where the length of the first window is Q and the height is 1. The second window is set as the sampling window for the morphological opening operation, which captures vertical lines with a length greater than Q in the binary image from the vertical direction; the length of the second window is 1 and the height is Q. By summarizing all the captured horizontal and vertical line information, we can obtain information on all long straight lines in the form.

5. The method for extracting form structure according to claim 1, characterized in that, Based on the starting position, the current receptive field and its size are determined on the binary image by locating the cell boundaries horizontally and vertically, including: Construct the initial receptive field based on the starting position and the initial value of the receptive field size; S4-2. Horizontally locate cell boundaries and determine the width of the receptive field: Based on the initial receptive field, continuously expand the width of the receptive field horizontally to the right until it reaches the horizontal boundary of the current cell, then stop expanding horizontally, return to obtain the width of the receptive field that meets the standard, and update the receptive field. Specifically, this includes: Set the pixels of the bottom row of the receptive field to 1, and execute the first loop step until the width of the receptive field that meets the standard is obtained. The first loop step includes: S4-2-1. Perform average pooling on all pixels on the right boundary of the current receptive field using the first pooling window to obtain the first average pooling value: wherein the height of the first pooling window is the height of the current receptive field and the width is 1. If the first average pooling value is not greater than the second threshold, the receptive field width is increased by 1, the pixel value of the right boundary of the receptive field is updated, and S4-2-1 is returned. If the first average pooling value is greater than the second threshold, execute S4-2-2; S4-2-2: Obtain the corresponding convolution kernel based on the current receptive field size; perform a convolution operation based on the receptive field and the corresponding convolution kernel to obtain the convolution result; divide the convolution result by the perimeter of the receptive field window to obtain the cell confidence score; if the cell confidence score is not greater than the first threshold, increment the receptive field width by 1, update the pixel value of the right boundary of the receptive field, and return to S4-2-1; if the cell confidence score is greater than the first threshold, consider the right boundary of the current receptive field to be the true right boundary of the cell, terminate the first loop step, and return the width of the current receptive field. S4-3. Vertically locate the cell boundary and determine the height of the receptive field; based on the receptive field obtained in S4-2, continuously expand the height of the receptive field vertically downwards until it reaches the vertical boundary of the current cell, then stop vertical expansion, return the height of the receptive field that meets the standard, and update the receptive field of the current cell size; specifically including: Set the pixels in the right row of the updated receptive field to 1, and execute the second loop step until the height of the receptive field that meets the standard is obtained, wherein the second loop step includes: S4-3-1. Perform average pooling on all pixels at the bottom boundary of the current receptive field using the second pooling window to obtain the second average pooling value, wherein the width of the second pooling window is the width of the current receptive field and the height is 1. If the second average pooling value is not greater than the third threshold, increment the receptive field height by 1, update the pixel value at the bottom boundary of the receptive field, and return to S4-3-1; If the second average pooling value is greater than the third threshold, execute S4-3-2; S4-3-2. Obtain the corresponding convolution kernel based on the current receptive field size; perform a convolution operation based on the receptive field and the corresponding convolution kernel to obtain the convolution result; divide the convolution result by the perimeter of the receptive field window to obtain the cell confidence score. If the cell confidence score is not greater than the first threshold, increment the receptive field height by 1, update the pixel value of the bottom boundary of the receptive field, and return S4-3-1; If the cell confidence score is greater than the first threshold, the right boundary of the current receptive field is considered to be the true bottom boundary of the cell, the second loop step is terminated, and the height of the current receptive field is returned.

6. The method for extracting form structure according to claim 1, characterized in that, Update the starting position using the obtained cell position, repeatedly use cell convolution operations to divide the remaining cell positions in the form, and mark the parallel and vertical relationships between cells, including: S5-1, Find the right-linked cell: Take the top-right coordinate of the first cell B1 (the top-left corner of the form) as the new starting position and include it in the cell convolution operation for calculation; if a new cell is returned, denote the new cell as B1. 1-left1 , is the right-linked cell of cell B1; If it does not return, then terminate S5-1 and proceed to S5-2; B 1-left1 The convolution operation is used to obtain more right-linked cells and this type of operation is repeated until no results are returned. All cells parallel to cell B1 are found by repeating the operation in S5-1. S5-2 Find the linked cell below: The bottom left coordinate of the first cell B1 in the top left corner of the form is used as the new starting position and is put into the convolution operation of cell S4 for calculation; if a new cell is returned, the new cell B2 is recorded as the lower link cell of cell B1; if no cell is returned, S5-2 is terminated. After obtaining cell B2, repeat step S5-1 to find the right linked cell, in order to obtain the right linked cell that is parallel to cell B2. By repeating the operations of S5-1 to find the right-linked cell and S5-2 to find the bottom-linked cell, the positions of all cells in the form are captured from both the right and down directions, and the parallel and vertical relationships between cells are marked.

7. The method for extracting form structure according to claim 1, characterized in that, Based on the parallel and vertical relationships between cells, the hierarchical relationships of cells in the form are analyzed to obtain the form structure, including: For parallel cells, if the information difference between two cells is less than the fourth threshold, the two parallel cells are determined to be at the same level; if the information difference between two cells is greater than the fourth threshold, and the length of cell A is shorter than the length of cell B, then cell A is the superior of cell B. For the cells above and below, if B i There are no parallel cells, and B i+1 If there are parallel cells, then the default is B. i For all B i+1 The parent of the parallel cell; if B i and B i+1 If both have parallel cells, then B i and B i+1 Equal level; The method for obtaining the information gap between the two cells includes: By using a convolution kernel with all weights equal to 1 corresponding to the cell size, the cells are convolved to obtain the information content of the corresponding cells; For parallel cells, calculate the information gap between the two cells based on the amount of information in each cell.

8. The method for extracting form structure according to claim 7, characterized in that, Using a convolution kernel with all weights equal to 1 corresponding to the cell size, the cells are convolved to obtain the information content of the corresponding cells, including: Based on the different cell sizes, construct a convolution kernel with all weights equal to 1: By performing a convolution on each cell using this kernel, the information content of the cell can be obtained: R conv1 =∑Unit (x,y) Kernel-1 (x,y) Where R conv1 Unit represents the amount of information contained in a cell. (x,y) Represents the pixels in each cell; And / or, for parallel cells, calculate the information gap between the two cells based on the amount of information in each cell, including: In parallel cells, observing from left to right, let R be the information content of the adjacent left cell A. conv1-A The information content of cell B on the right is R. conv1-B Calculate the information gap between cells: Dist info W represents the information gap between cell A and cell B. A and W B These are the widths of cell A and cell B, respectively.

9. A form structure extraction device, characterized in that, The device includes: The acquisition module is used to acquire a target form image; wherein the form image is an image containing a form structure; The image processing module is used to obtain a binary image from the target form image using edge detection and morphological algorithms; The positioning module is used to automatically locate the starting position of the form in the binary image; A cell convolution operation module is used to perform cell convolution operations on the binary image based on the starting position of the form to obtain the cell position. The cell convolution operation includes: determining the current receptive field and its size on the binary image by horizontally and vertically locating the cell boundaries based on the starting position; wherein the receptive field size includes the width and height of the receptive field; obtaining the corresponding convolution kernel based on the receptive field size; performing a convolution operation based on the receptive field and the corresponding convolution kernel to obtain the convolution result; dividing the convolution result by the perimeter of the receptive field window to obtain the cell confidence score; wherein the perimeter of the receptive field window is determined based on the width and height of the receptive field; if the cell confidence score is greater than a first threshold, the current receptive field is considered a cell. The repositioning module is used to update the starting position using the obtained cell position, repeatedly use cell convolution operation to divide the remaining cell positions in the form, and mark the parallel and vertical relationships between cells; The form structure building unit is used to analyze the hierarchical relationship of cells in the form based on the parallel and vertical relationships between cells, and to obtain the form structure.

Images