Filling option analysis method and device, electronic equipment and storage medium

Abstract

The invention discloses a full-filling option analysis method and device and electronic equipment, and belongs to the technical field of image processing. The method comprises the following steps: performing table structure analysis on an image of a target full-filled information card to obtain structure information of cells in the image; obtaining cell position information corresponding to each filling option in the image based on the structure information and a mapping relationship between pre-configured filling options of the target filling information card and a table structure; and analyzing the content of the image area corresponding to the filling option based on the cell position information to obtain an analysis result of each filling option in the target filling information card. According to the method, the analysis accuracy of the full-filling options in the full-filling information card image is effectively improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, in particular to a fill-in option analysis method and device, an electronic device, and a computer readable storage medium. BACKGROUND

[0002] A fill-in information card is a physical carrier for representing a selection item by drawing a specified pattern on a paper form or a selection card. With the development of OCR technology and the promotion of office automation, there are more and more application requirements for using a computer to automatically analyze fill-in items in a paper document. For example, using computer technology to analyze a test paper answer sheet or a form selection. Due to different uses, the specifications and formats of various fill-in information cards are different, and different fill-in personnel and different handwriting habits result in great difficulty in recognizing and analyzing fill-in information cards.

[0003] In the prior art, common fill-in information card analysis methods include an OCR recognition method and a customized software and hardware template method. The OCR recognition method recognizes and analyzes handwritten fill-in options. This method has a strict and single requirement for the shape of handwritten fill-in, and the recognition accuracy is not high for fill-in methods such as check marks, solid triangles, and hollow circles. The phenomenon of loss, omission, and the like often occurs, resulting in distorted statistical results. In addition, a large amount of handwriting data of fill-in personnel needs to be collected for training against different handwriting styles of different people, which is extremely costly and unsustainable. The customized software and hardware template method can only solve a single requirement, and is severely dependent on templates. When the demand changes and the template needs to be updated, a great cost is required for customization and development.

[0004] It can be seen that the fill-in option analysis method in the prior art still needs to be improved. SUMMARY

[0005] The embodiments of the present application provide a fill-in option analysis method and device, which helps to improve the fill-in option analysis accuracy and reduce the cost.

[0006] In a first aspect, the embodiments of the present application provide a fill-in option analysis method, including:

[0007] performing form structure analysis on an image of a target fill-in information card to obtain structure information of a cell in the image;

[0008] obtain cell position information corresponding to each fill-in option in the image based on the structure information, and a preconfigured mapping relationship between fill-in options and form structures of the target fill-in information card;

[0009] analyze the content of an image region corresponding to the fill-in option based on the cell position information to obtain an analysis result of each fill-in option in the target fill-in information card.

[0010] In a second aspect, an embodiment of the present application provides a filling option analysis device, comprising:

[0011] a cell structure information acquisition module configured to perform table structure analysis on an image of a target filling information card to acquire structure information of a cell in the image;

[0012] a filling option position information acquisition module configured to acquire cell position information corresponding to each filling option in the image based on the structure information and a mapping relationship between preconfigured filling options of the target filling information card and table structures;

[0013] a filling option analysis module configured to analyze content of an image region corresponding to the filling option based on the cell position information to obtain an analysis result of each filling option in the target filling information card.

[0014] In a third aspect, an embodiment of the present application further discloses an electronic device, comprising a memory, a processor, and a computer program stored in the memory and capable of running on the processor, and the processor implements the filling option analysis method according to the embodiment of the present application when executing the computer program.

[0015] In a fourth aspect, an embodiment of the present application provides a computer readable storage medium, which stores a computer program, and the program is executed by a processor to implement the steps of the filling option analysis method disclosed in the embodiment of the present application.

[0016] The filling option analysis method disclosed in the embodiment of the present application performs table structure analysis on an image of a target filling information card to acquire structure information of a cell in the image, acquires cell position information corresponding to each filling option in the image based on the structure information and a mapping relationship between preconfigured filling options of the target filling information card and table structures, and analyzes content of an image region corresponding to the filling option based on the cell position information to obtain an analysis result of each filling option in the target filling information card, thereby effectively improving the analysis accuracy of filling options in a filling information card image.

[0017] The above description is only a summary of the technical solutions of the present application. In order to more clearly understand the technical means of the present application, the embodiments of the present application can be implemented according to the content of the specification, and in order to make the above and other purposes, features and advantages of the present application more obvious and easy to understand, the following specific embodiments of the present application are described. BRIEF DESCRIPTION OF DRAWINGS

[0018] In order to make the purposes, technical solutions, and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are some but not all of the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by a person of ordinary skill in the art without creative work fall within the protection scope of the present application.

[0019] Figure 1 is a step flow chart of the filling option analysis method disclosed by the embodiments of the present application;

[0020] Figure 2 is an image schematic diagram of the filling information card in the filling option analysis method disclosed by the embodiments of the present application;

[0021] Figure 3 is an effect schematic diagram of the morphological processing of the image shown in Figure 2 by the filling option analysis method disclosed by the embodiments of the present application to obtain a first mask image;

[0022] Figure 4 is a table image schematic diagram extracted from the filling information card image shown in Figure 2 by the filling option analysis method disclosed by the embodiments of the present application;

[0023] Figure 5 is a structure schematic diagram of the filling option analysis device disclosed by the embodiments of the present application;

[0024] Figure 6 a block diagram of an electronic device for performing the method according to the present application is schematically shown; and

[0025] Figure 7 a storage unit for holding or carrying program code for implementing the method according to the present application is schematically shown. DETAILED DESCRIPTION

[0026] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are some but not all of the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by a person of ordinary skill in the art without creative work fall within the protection scope of the present application.

[0027] The specific implementation of the present application will be described in detail below with reference to the flow schematic diagram shown in Figure 2 .

[0028] With reference to Figure 1 , a filling option analysis method disclosed by the embodiments of the present application comprises steps 110 to 130.

[0029] In step 110, table structure analysis is performed on the image of the target fill-in information card to obtain structure information of the cells in the image.

[0030] In the embodiments of the present application, the target fill-in information card is a fill-in information card in the currently collected image. The target fill-in information card is a table form information card.

[0031] In some optional embodiments, the table structure analysis performed on the image of the target fill-in information card to obtain the structure information of the cells in the image includes sub-step S1 to sub-step S5.

[0032] In sub-step S1, morphological processing is performed on the image of the target fill-in information card to obtain a first mask image.

[0033] Morphological operation is a shape-based image processing technique that changes the morphology and characteristics of an image by performing specific operations on the structure element and the image. The structure element is a small, pre-defined shape, usually rectangular, circular or elliptical, which can be matched with the pixels in the image. Morphological operation includes dilation, erosion, opening operation, closing operation, etc. The opening operation can remove noise and smooth the edges of the image. The opening operation is a combination process of first performing erosion operation on the image and then performing dilation operation, which can be used to eliminate small objects or details and can smooth the boundaries of objects.

[0034] In some embodiments of the present application, when the image of the target fill-in information card is morphologically processed to obtain a first mask image, the image is opened with a horizontal rectangular kernel and a vertical rectangular kernel, respectively, and then the results of the two opening operations are executed with an AND operation to obtain the first mask image. Wherein, the opening operation of the image with the horizontal rectangular kernel obtains a mask profile mainly with horizontal lines, and the opening operation of the image with the vertical rectangular kernel obtains a mask profile mainly with vertical lines. The result of the AND operation of the two opening operations is a mask image with horizontal and vertical lines intersecting each other. The horizontal and vertical lines intersecting each other can be used as the table structure information of the target fill-in information card. Figure 2 For example, as shown in the image, after morphological processing, the obtained first mask image is as shown in Figure 3

[0035] Optionally, the first mask image can be represented by a two-dimensional matrix, and the elements in the matrix correspond to the pixel values of the pixel points in the first mask image.

[0036] After the AND operation of the mask images processed by the horizontal and vertical rectangular kernels, respectively, the approximately accurate table intersection lines can be obtained, which can reduce the picture resolution and reduce the calculation amount of subsequent operations.

[0037] ​Sub-step S2, a second mask image of the rectangular table in the image is obtained by polygon approximation processing of the first mask image.

[0038] According to the first mask image, the position coordinates of the pixel points of the lines, fillings and table texts in the target fill-in information card can be determined by using the method in the prior art. The polygons in the image can be obtained by polygon approximation processing of the first mask image. Then, the polygons are screened, and the rectangles in the image are obtained. The rectangles in the image are the tables corresponding to the fill-in options.

[0039] In some optional embodiments, the first point and the last point in the pixel point matrix of the first mask image, i.e., the start point and the end point, can be connected to form a straight line, and the distance from each pixel point in the matrix to the straight line is calculated in sequence, compared with a threshold value, and the pixel points less than the threshold value are screened out, so as to obtain the remaining pixel points and the mask shape composed of the remaining pixel points. Then, the mask shape is judged, and the non-rectangular mask shapes are further screened out, so as to obtain the rectangular mask shape as the mask information of the accurate table. The threshold value is usually set to half of the minimum value of the difference between the horizontal and vertical coordinates of the start point and the end point, i.e., 0.5*min{|x1-x2|,|y1-y2|}, wherein x1 and y1 represent the horizontal and vertical coordinates of the start point, and x2 and y2 represent the horizontal and vertical coordinates of the end point.

[0040] The polygon approximation processing of the first mask image to obtain the second mask image of each rectangular table can also use other prior art, which will not be listed one by one in the embodiments of the present application.

[0041] Sub-step S3, based on the second mask image, the image of the rectangular table is extracted from the image.

[0042] The polygon approximation processing of the first mask image to obtain the second mask image of each rectangular table can also use other prior art, which will not be listed one by one in the embodiments of the present application.

[0043] Further, the area of the rectangular table in the image is determined according to the coordinates of the four vertices of the rectangular table in the second mask image, and the image of the area is intercepted to obtain the image of the rectangular table. The image shown in Figure 2 is processed by using the above steps, and the image of the rectangular table obtained is as shown in Figure 4 .

[0044] Sub-step S4, the image of the rectangular table is corrected to obtain a corrected image of the table.

[0045] Optionally, the image of the rectangular table is corrected to obtain a corrected image of the table, including: based on the second mask image, performing table trend adjustment processing on the image of the rectangular table to obtain an image of a horizontal table; determining the size of one or more types of elements in the image of the horizontal table: cell, horizontal line segment, connected text and discrete matrix; according to a preset element priority, searching for a target element from the elements, wherein the size of the target element and the size of the horizontal table satisfy a preset condition; in the case of finding the target element, determining the inclination angle of the horizontal table according to the inclination angle of the target element; in the case of not finding the target element, determining the inclination angle of the horizontal table according to the inclination angle distribution of the element with the highest priority; and correcting the image of the horizontal table based on the inclination angle of the horizontal table.

[0046] Optionally, based on the second mask image, the image of the rectangular table is subjected to table trend adjustment processing to obtain an image of a horizontal table, including: based on the aspect ratio of the circumscribed rectangle of the rectangular table in the second mask image, determining whether the rectangular table is a horizontal table or a vertical table; in the case of the rectangular table being a vertical table, rotating the image of the rectangular table by 90 degrees to obtain an image of a horizontal table. For example, when the aspect ratio is greater than 1, it can be considered that the element distribution direction in the rectangular table is mainly horizontal, that is, the rectangular table is a horizontal table; otherwise, it can be considered that the element distribution direction in the rectangular table is mainly vertical, that is, the rectangular table is a vertical table.

[0047] Optionally, the second mask image can also be subjected to layout analysis, and the weights of horizontal text, vertical text and lines are calculated respectively, and the table trend is determined according to the calculated weights.

[0048] Optionally, the one or more types of elements in the image of the horizontal table are determined by using existing technologies, and the size of each element is determined. Optionally, the size of the cell is the area, the size of the horizontal line segment is the horizontal length, the size of the connected text is the horizontal length, and the size of the discrete matrix is the area.

[0049] The preset element priority is used to describe the element weight affecting the table trend. The preset element priority is set according to experience. For example, the priority of the cell can be set to be greater than the priority of the connected text, the priority of the connected text can be set to be greater than the priority of the horizontal line segment, and the priority of the discrete matrix can be set to be the smallest.

[0050] In specific implementation, the inclination angle of each type of element can be determined by using existing technologies, which will not be described herein.

[0051] Optionally, the size of the target element and the size of the transverse table satisfy a preset condition, including: the area ratio of the element to the area of the transverse table is greater than or equal to a first proportion threshold; or, the transverse length ratio of the element to the width of the transverse table is greater than or equal to a second proportion threshold. Wherein, the values of the first proportion threshold and the second proportion threshold are determined according to specific needs.

[0052] Taking the preset element priority as an example: cell > horizontal line segment > connected text > discrete matrix, when searching for a target element in the element according to the preset element priority, the elements with obvious features are searched in order according to the preset element priority from high to low, that is, large table, long straight line, long text line and large discrete matrix. That is, first search for a cell whose area ratio to the area of the transverse table is greater than or equal to a first proportion threshold, if found, the inclination angle of the cell is taken as the inclination angle of the transverse table, if not found, then search for a horizontal line segment whose transverse length ratio to the width of the transverse table is greater than or equal to a second proportion threshold. In this way, if the target element is not found at last, the inclination angle of the transverse table is determined according to the inclination angle distribution of the element with the highest priority.

[0053] Optionally, the inclination angle of the transverse table is determined according to the inclination angle distribution of the element with the highest priority, including: obtaining the distribution density of the inclination angle interval of the target type element, wherein the target type element is the element with the highest priority; taking the average value of the inclination angle of the target type element in the inclination angle interval corresponding to the maximum value of the distribution density as the inclination angle of the transverse table. For example, when the priority of the cell is the highest, after obtaining the inclination angle of each cell element, the number of cell elements whose inclination angle falls in each preset inclination angle interval is calculated respectively, and the inclination angle interval corresponding to the maximum value of the number is taken as the target inclination angle interval, and then the average value of the inclination angle of all cell elements whose inclination angle falls in the target inclination angle interval is taken as the inclination angle of the transverse table.

[0054] Then, based on the inclination angle of the transverse table, the image orientation correction method in the prior art is used to correct the image of the transverse table.

[0055] The image of the transverse table is corrected to obtain a corrected image of the table, which facilitates subsequent cell analysis based on the corrected image to improve the accuracy of cell analysis.

[0056] Sub-step S5, performing cell splitting on the corrected image to obtain the structure information of the cells in the table.

[0057] In some optional embodiments, the cell splitting on the corrected image and obtaining the structure information of the cells in the table include: performing morphological processing on the corrected image to obtain a third mask image of the corresponding table; performing line segment detection on the third mask image to obtain coordinates of line segments in the table; performing aggregation on the line segments based on the coordinates to obtain horizontal lines and vertical lines constituting the table, and position coordinates of the horizontal lines and position coordinates of the vertical lines; calculating intersection points of the horizontal lines and the vertical lines based on the position information of the horizontal lines and the position information of the vertical lines, and position coordinates of the intersection points; obtaining logical information of a cell with the intersection point as a vertex based on the horizontal line and the vertical line corresponding to the intersection point; and obtaining position coordinates of the cell with the intersection point as the vertex based on the position coordinates of the intersection point.

[0058] For a specific implementation of performing morphological processing on the corrected image to obtain a third mask image of the corresponding table, refer to the prior art, which will not be described herein again.

[0059] In some optional embodiments, the line segment detection on the third mask image can be performed by using a line segment detector (LSD) algorithm to obtain coordinates of all line segments in the third mask image. The coordinates of all line segments in the third mask image are coordinates of the borders of the cells in the table. Then, according to the coordinates of the line segments, the line segments with horizontally extended lines coinciding are aggregated into a horizontal line, and the line segments with vertically extended lines coinciding are aggregated into a vertical line, to obtain multiple horizontal lines and vertical lines constituting the table, and endpoint coordinates of each horizontal line and each vertical line.

[0060] Next, based on the position information of each horizontal line and the position information of each vertical line obtained by aggregation, the intersection points of each horizontal line and each vertical line are calculated by using a method in the prior art, to obtain intersection point coordinates. For example, the multiple horizontal lines obtained by aggregation are sorted from top to bottom according to the endpoint coordinates, the multiple vertical lines obtained by aggregation are sorted from left to right according to the endpoint coordinates, and the intersection points of each horizontal line and each vertical line are calculated from the top left in sequence.

[0061] For a specific example, taking a table as shown in FIG. 2A as an example, the corrected image of the table is shown in FIG. 2B, and the third mask image of the table is shown in FIG. 2C. Figure 4As shown in the table image, each horizontal line is numbered from 0 in the order from top to bottom, the top horizontal line is numbered 0, and the bottom horizontal line is numbered 5. Each vertical line is numbered from 0 in the order from left to right, the leftmost vertical line is numbered 0, and the rightmost vertical line is numbered 6. Then, the intersection points of each horizontal line and the seven vertical lines from left to right are calculated in turn to obtain the intersection point coordinates, and each two adjacent intersection points are recorded as the coordinates of the top left corner and the top right corner of the cell sandwiched between the two vertical lines. Figure 4 For example, the cell in the top left corner is obtained by the intersection of the horizontal line numbered 0, the horizontal line numbered 1, the vertical line numbered 0, and the vertical line numbered 1. Therefore, the logical information of the cell is represented as {0, 1, 0, 1}. Figure 4 For example, the cell in the bottom right corner is obtained by the intersection of the horizontal line numbered 4, the horizontal line numbered 5, the vertical line numbered 5, and the vertical line numbered 6. Therefore, the logical information of the cell is represented as {5, 6, 4, 5}.

[0062] According to the above table analysis method, the logical information and the vertex position coordinates of each cell in the table can be obtained. The logical information and the vertex position coordinates of all cells included in the table constitute the structure information of the table.

[0063] In step 120, based on the mapping relationship between the preconfigured filling options of the target filling information card and the table structure, the cell position information corresponding to each filling option in the image is obtained.

[0064] In the embodiments of the present application, the mapping relationship between the filling options and the table structure is pre-set for different filling information cards.

[0065] The structure information of the cell includes the logical information and the vertex position coordinates of the cell in the target filling information card. Based on the mapping relationship between the preconfigured filling options of the target filling information card and the table structure, the cell position information corresponding to each filling option in the image is obtained, including: for each cell, determining whether the logical information of the cell conforms to the mapping relationship between the preconfigured filling options of the target filling information card and the table structure; in the case that the logical information of the cell conforms to the mapping relationship between the filling options and the table structure, determining that the cell is a filling option, and taking the vertex position coordinates of the cell as the cell position information of the corresponding filling option; in the case that the logical information of the cell does not conform to the mapping relationship between the filling options and the table structure, determining that the cell is a non-filling option cell.

[0066] Optionally, the logical information of the cell is represented based on the numbering of the horizontal and vertical lines that constitute the cell. Taking the example of numbering the straight lines that enclose the cells in the table from left to right and from top to bottom starting from 0, the logical information of the cell can be recorded in the form of data {colstart, colend, rowstart, rowend}, where colstart represents the number of the line segment above the cell on the horizontal line, colend represents the number of the line segment below the cell on the horizontal line, rowstart represents the number of the line segment on the left of the cell on the vertical line, rowend represents the line segment on the right of the cell on the vertical line, colend-colstart represents the number of horizontally merged cells, and similarly rowend-rowstart represents the number of vertically merged cells.

[0067] by Figure 2 Taking the target filling information card shown as an example, the table in the target filling information card is composed of 6 horizontal lines and 7 vertical lines intersecting each other. The horizontal lines are numbered 0 to 5 from top to bottom, and the vertical lines are numbered 0 to 5 from left to right. The cells in the 2nd row, 2nd column to 5th column, the 3rd row, 2nd column to 5th column, the 4th row, 2nd column to 5th column, and the 5th row, 2nd column to 5th column of the table are filling option cells. Based on the numbering of the horizontal and vertical lines constituting the cells, the logical information of the above filling option cells is expressed as follows: {1,2,1,2}, {2,3, ,4}, {1,2,4,5}, {2,3,4,5}, {3,4,4,5}, {4,5,4,5}, {5,6,4,5}. The mapping relationship between the above filling options and the table structure can be expressed by the formula: {1+n, 2+n, 1+m, 2+m}, where 0≤n≤4, 0≤m≤3, that is, the cells between the second row and second column to the fifth row and sixth column in the table are the filling option cells.

[0068] When analyzing the fill options for the target fill information card image, only cells whose logical information conforms to the above-mentioned mapping formula between fill options and table structure can be selected. These cells are the cells corresponding to the fill options. Furthermore, the vertex position coordinates of these cells are obtained as the cell position information of the corresponding fill options.

[0069] In step 130, the content of the image region corresponding to the fill-in option is analyzed based on the cell position information to obtain the analysis result of each fill-in option in the target fill-in information card.

[0070] In some optional embodiments, the analysis of the content of the image region corresponding to the fill-in option based on the cell position information to obtain the analysis result of each fill-in option in the target fill-in information card comprises: determining the image region where the cell corresponding to each fill-in option in the image is located based on the cell position information; performing gradient analysis on the content of the image region where the cell corresponding to each fill-in option is located to obtain the overall fill-in style information and the fill-in shape gradient information of each fill-in option; and obtaining the analysis result of each fill-in option in the target fill-in information card based on the overall fill-in style information and the fill-in shape gradient information.

[0071] By extracting the mask in the cell where the fill-in option is located and calculating the gradient information, the gradient information of the entire fill-in information card is statistically analyzed, the personal fill-in habit curve can be simulated, the personal fill-in style of the single fill-in information card can be obtained, and the intentional fill-in part in the current fill-in information card can be analyzed and the unintentional fill-in part can be filtered out.

[0072] The specific implementation of determining the image region where the cell corresponding to each fill-in option in the image is located based on the cell position information can be found in the prior art, and will not be described here in the embodiments of the present application.

[0073] In some optional embodiments, the gradient analysis on the content of the image region where the cell corresponding to each fill-in option is located to obtain the overall fill-in style information and the fill-in shape gradient information of each fill-in option comprises: performing image processing on the image region where the cell is located corresponding to each fill-in option to obtain the mask of the fill-in in the cell; obtaining the fill-in shape gradient information of each fill-in option by calculating the edge gradient change of each mask; determining the fill-in option that has not been filled and the fill-in option that has been filled based on the fill-in shape gradient information; and obtaining the overall fill-in style information of the target fill-in information card in the current image based on the fill-in shape gradient information of the fill-in option that has been filled.

[0074] For the image of each cell corresponding to the fill-in option, a polygon approximation algorithm can be used for image processing to obtain the contour shape of the fill-in in the cell, i.e., the mask of the fill-in. Then, by calculating the standard deviation of the gradient change of the mask edge, the standard deviation is used to simply describe the change of the curve inside each cell, and is recorded as the fill-in shape gradient information of each cell, i.e., the fill-in shape gradient information of the fill-in option corresponding to each cell. According to this method, for each table, a set of standard deviations can be obtained, i.e., a set of fill-in shape gradient information. Generally, the fill-in and non-fill-in cases differ greatly, i.e., the result of the non-fill-in area is empty, i.e., there is no irregular shape, so the cells with fill-in parts can be screened out. Optionally, the standard deviation and median of the fill-in shape gradient information (i.e., the standard deviation of the gradient change) of all cells with fill-in shapes are calculated as the overall fill-in style information of the target fill-in information card to represent the fill-in style of the current fill-in information card.

[0075] In some optional embodiments, the overall fill-in style information includes the standard deviation and median of the fill-in shape gradient information of the filled fill-in options, and the obtaining of the analysis result of each fill-in option in the target fill-in information card based on the overall fill-in style information and the fill-in shape gradient information includes: obtaining a difference threshold based on a first difference value of the standard deviation and the median; obtaining a second difference value of the fill-in shape gradient information of each filled fill-in option and the median; obtaining an analysis result that the filled fill-in option is valid fill-in in a case where the second difference value is less than or equal to the difference threshold; obtaining an analysis result that the filled fill-in option is invalid fill-in in a case where the second difference value is greater than the difference threshold; and obtaining the analysis result of each fill-in option in the target fill-in information card according to the analysis result of the filled fill-in option and the unfilled fill-in option.

[0076] After the gradient information of the filling shape of each filling option is obtained in the foregoing steps, and the median and standard deviation of the gradient information of the filling shape as the overall filling style information are obtained, first, the standard deviation and the median of the set of gradient information of the filling shape of the obtained filling options are calculated based on the gradient information of the filling shape (i.e., the standard deviation of the gradient change), and a difference threshold is obtained based on the difference between the calculated standard deviation and median (denoted as "first difference" in the embodiments of the present application). Then, the difference between the gradient information of the filling shape (i.e., the standard deviation of the gradient change) of each filling option and the median (denoted as "second difference" in the embodiments of the present application) is calculated, and the difference between the filling shape in the corresponding filling option and the filling style of the overall filling information card can be obtained. If the second difference is less than or equal to the difference threshold, it is considered that the filling option is an effective filling analysis result because there is no extreme value in the filling option. If the second difference is greater than the difference threshold, it is considered that the filling option is an invalid filling analysis result.

[0077] Finally, the analysis results of the filling options corresponding to all cells in the filling information card are integrated to obtain the analysis results of the filling options in the target filling information card. For example, the analysis results of all filled options (including effective filling and invalid filling) and unfilled filling options are taken as the analysis results of the filling options in the target filling information card.

[0078] In summary, the filling option analysis method disclosed in the embodiments of the present application obtains the structure information of the cells in the image of the target filling information card by performing table structure analysis on the image. Based on the structure information, the mapping relationship between the preconfigured filling options of the target filling information card and the table structure, the cell position information corresponding to each filling option in the image is obtained. Based on the cell position information, the content of the image region corresponding to the filling option is analyzed to obtain the analysis results of each filling option in the target filling information card, which effectively improves the analysis accuracy of the filling options in the filling information card image.

[0079] On the other hand, by using image processing technology to detect the table style, the hardware resources do not need to be excessively dependent, and no specified template is used. Not only is the cost low, but also after the arbitrary relationship between the specified information card and the table structure is replaced, the problem of poor template expandability is solved. Further, the adaptive filling style analysis is performed based on the filling results of the overall filling information card, so as to judge the filling results of the filling options. The single threshold corresponding to the single filling information card effectively avoids the mistakes and omissions caused by the normalized processing of different habits of different people when personalized filling, and further improves the accuracy of the filling option analysis.

[0080] Reference Figure 5The embodiment of the application further discloses a filling option analysis device, and the device comprises:

[0081] The cell structure information acquisition module 510 is configured to perform table structure analysis on an image of a target filling information card, and acquire structure information of a cell in the image.

[0082] The filling option position information acquisition module 520 is configured to acquire cell position information corresponding to each filling option in the image based on the structure information and a mapping relationship between a preconfigured filling option of the target filling information card and a table structure.

[0083] The filling option analysis module 530 is configured to analyze content of an image region corresponding to each filling option based on the cell position information, and obtain an analysis result of each filling option in the target filling information card.

[0084] Optionally, the filling option analysis module 530 is further configured to:

[0085] acquire an image region in which a cell corresponding to each filling option in the image is located based on the cell position information.

[0086] perform gradient analysis on content of the image region in which the cell corresponding to each filling option is located, and obtain overall filling style information and filling shape gradient information of each filling option.

[0087] acquire the analysis result of each filling option in the target filling information card based on the overall filling style information and the filling shape gradient information.

[0088] Optionally, the gradient analysis on the content of the image region in which the cell corresponding to each filling option is located comprises:

[0089] perform image processing on the image region in which the cell corresponding to each filling option is located, and acquire a mask of filling in the cell.

[0090] obtain the filling shape gradient information of each filling option by calculating edge gradient changes of each mask.

[0091] determine the filling option that is not filled and the filling option that is filled based on the filling shape gradient information.

[0092] acquire overall filling style information of the target filling information card in the current image based on the filling shape gradient information of the filling option that is filled.

[0093] Optionally, the overall filling style information comprises a standard deviation and a median of filling shape gradient information of the filled filling option, and the analysis result of each filling option in the target filling information card is obtained based on the overall filling style information and the filling shape gradient information, comprising:

[0094] a first difference value based on the standard deviation and the median, to obtain a difference threshold value;

[0095] for each filled filling option, a second difference value of the filling shape gradient information of the filling option and the median is obtained;

[0096] in the case where the second difference value is less than or equal to the difference threshold value, the analysis result that the filled filling option is valid filling is obtained;

[0097] in the case where the second difference value is greater than the difference threshold value, the analysis result that the filled filling option is invalid filling is obtained;

[0098] According to the analysis result of the filled filling option and the unfilled filling option, the analysis result of each filling option in the target filling information card is obtained.

[0099] Optionally, the cell structure information obtaining module 510 is further used for:

[0100] performing morphological processing on the image of the target filling information card to obtain a first mask image;

[0101] performing polygon approximation processing on the first mask image to obtain a second mask image of each rectangular table through screening;

[0102] based on the second mask image, extracting the image of the rectangular table from the image;

[0103] performing correction processing on the image of the rectangular table to obtain a corrected image of the table;

[0104] performing cell splitting on the corrected image to obtain the structure information of the cells in the table.

[0105] Optionally, the correction processing on the image of the rectangular table to obtain the corrected image of the table comprises:

[0106] based on the second mask image, performing table trend adjustment processing on the image of the rectangular table to obtain an image of a horizontal table;

[0107] determining the size of one or more types of elements in the image of the horizontal table: cells, horizontal line segments, connected text, and discrete matrices;

[0108] According to a preset element priority, a target element is searched for in the elements, wherein a size of the target element and a size of the transverse table meet a preset condition;

[0109] In a case where the target element is searched for, an inclination angle of the transverse table is determined according to an inclination angle of the target element;

[0110] In a case where the target element is not searched for, an inclination angle of the transverse table is determined according to an inclination angle distribution of the element with the highest priority;

[0111] An image of the transverse table is corrected based on the inclination angle of the transverse table.

[0112] The filling option analysis device disclosed in the embodiments of the present application is used to implement the filling option analysis method described in the embodiments of the present application. The specific implementation manners of the modules of the device will not be described again, and can be referred to the specific implementation manners of the corresponding steps of the method embodiments.

[0113] The filling option analysis device disclosed in the embodiments of the present application obtains the structure information of the cells in the image by performing table structure analysis on the image of the target filling information card; obtains the cell position information corresponding to each filling option in the image based on the structure information, and a mapping relationship between the preconfigured filling options of the target filling information card and the table structure; and analyzes the content of the image region corresponding to the filling option based on the cell position information, to obtain the analysis result of each filling option in the target filling information card. This not only effectively improves the analysis accuracy of the filling options in the filling information card image.

[0114] On the other hand, the table style detection is performed by using the image processing technology, without excessive dependence on hardware resources and without using a specified template. This not only has low cost, but also solves the problem of poor template expansibility after replacing the arbitrary relationship between the specified information card and the table structure. Further, the adaptive filling style analysis is performed based on the filling result of the entire filling information card, to determine the filling result of the filling option. The single filling information card corresponds to a single threshold, effectively avoiding errors and omissions caused by the normalized processing of different habits of different people when filling in a personalized manner, and further improving the accuracy of the filling option analysis.

[0115] Each of the embodiments in the specification is described in a progressive manner, and each embodiment focuses on the difference from other embodiments. The same and similar parts of each embodiment can be referred to each other. For the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the part of the method embodiment.

[0116] The above describes in detail the filling and coating option analysis method and device provided by the present application, and the principle and implementation mode of the present application are described by using specific examples. The above example is only used to help understand the method of the present application and its core idea. Meanwhile, for those skilled in the art, according to the idea of the present application, the specific implementation mode and application range can be changed. In summary, the content of the specification should not be understood as a limitation of the present application.

[0117] The device embodiments described above are only schematic, and the units described as separate components can or can not be physically separate, and the components displayed as units can or can not be physical units, that is, they can be located in one place, or distributed on multiple network units. According to actual needs, part or all of the modules can be selected to achieve the purpose of the present embodiment. Those skilled in the art can understand and implement without creative labor.

[0118] The various component embodiments of the present application can be implemented in hardware, or in software modules running on one or more processors, or in a combination thereof. Those skilled in the art should understand that microprocessors or digital signal processors (DSP) can be used in practice to implement some or all of the functions of some or all of the components in the electronic device according to the embodiments of the present application. The present application can also be implemented as a device or apparatus program (for example, a computer program and a computer program product) for executing part or all of the methods described herein. Such program implementing the present application can be stored on a computer readable medium or can have one or more signals in the form. Such signals can be downloaded from an Internet website, or provided on a carrier signal, or provided in any other form.

[0119] For example, Figure 6An electronic device is shown, in which the method according to the application can be implemented. The electronic device can be a PC, a mobile terminal, a personal digital assistant, a tablet computer, etc. The electronic device traditionally comprises a processor 610 and a memory 620 and program code 630 stored on the memory 620 and executable on the processor 610, which, when executed by the processor 610, implements the method described in the above embodiments. The memory 620 can be a computer program product or a computer readable medium. The memory 620 can be an electronic memory such as a flash memory, an EEPROM (electrically erasable programmable read-only memory), an EPROM, a hard disk or a ROM. The memory 620 has a storage space 6201 for the program code 630 of the computer program for executing any of the method steps described above. For example, the storage space 6201 for the program code 630 can comprise individual computer programs for implementing the various steps in the above methods, respectively. The program code 630 is computer readable code. The computer programs can be read from or written to one or more computer program products. The computer program products comprise program code carriers such as a hard disk, a compact disk (CD), a memory card or a floppy disk. The computer programs comprise computer readable code which, when executed on the electronic device, causes the electronic device to perform the method according to the above embodiments.

[0120] The embodiments of the present application further disclose a computer readable storage medium, having stored thereon a computer program, which, when executed by a processor, implements the steps of the fill-in option analysis method according to the embodiments of the present application.

[0121] Such a computer program product can be a computer readable storage medium having stored thereon the computer readable code 630' for performing the method described above when the computer readable code 630' is executed on a processor. Figure 6 The computer readable storage medium can have a similar arrangement of storage sections, storage spaces, etc. as the memory 620 in the electronic device shown. The program code can be stored in the computer readable storage medium, for example, in a compressed form. The computer readable storage medium is typically a portable or stationary storage unit as described with reference to Figure 7 The computer readable storage medium typically comprises computer readable code 630' which is code that is read by a processor, which, when executed by the processor, implements the various steps in the above described method.

[0122] Reference herein to "one embodiment", "an embodiment" or "one or more embodiments" means that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment of the application. The appearances of the phrase "in one embodiment" in various places in this specification are not necessarily all referring to the same embodiment.

[0123] In the description provided herein, numerous specific details are set forth. However, it is understood that embodiments of the application can be practiced without these specific details. In some instances, well-known methods, structures and techniques have not been shown in detail in order not to obscure an understanding of this description.

[0124] In the claims, any reference signs placed between parentheses shall not be construed as limiting the claim. The word 'comprising' does not exclude the presence of elements or steps other than those listed in a claim. The word 'a' or 'an' preceding an element does not exclude the presence of a plurality of such elements. The application can be implemented by means of both hardware and software, and any combination thereof. In a unitary claim, several devices or sub-claims can be joined by means of the word 'or'. The word 'first','second', 'third', etc. do not imply any order. The terms 'first','second', 'third', etc. are to be interpreted according to their ordinary meaning.

[0125] It has to be noted that, while the above describes example embodiments of the application, these are merely examples and do not limit the scope of the application described. Variations of these embodiments can become apparent to those of ordinary skill in the art, once the above teachings are fully appreciated. Those variations are, however, intended to be included within the scope of the application. Accordingly, the application is not to be seen as being limited to the foregoing examples, but is only limited by the scope of the appended patent claims.

Claims

1. A filling option analysis method, characterized in that: The method comprises: Performing table structure analysis on the image of the target filled-in information card to obtain structural information of cells in the image; Based on the structural information, a mapping relationship between pre-configured filling options of the target filling information card and a table structure, obtaining cell position information corresponding to each filling option in the image; The content of the image area corresponding to the filling option is analyzed based on the cell position information to obtain the analysis results of each filling option in the target filling information card.

2. The method according to claim 1, characterized in that The analyzing the content of the image area corresponding to the filling option based on the cell position information to obtain the analysis results of each filling option in the target filling information card includes: Based on the cell position information, determining the image area where the cell corresponding to each of the filling options in the image is located; Performing gradient analysis on the content of the image region where the cells corresponding to the filling options are located to obtain overall filling style information and filling shape gradient information of the filling options; Based on the overall filling style information and the filling shape gradient information, an analysis result of each filling option in the target filling information card is obtained.

3. The method according to claim 2, characterized in that The gradient analysis is performed on the content of the image area where the cells corresponding to the filling options are located to obtain the overall filling style information and the filling shape gradient information of the filling options, including: For each cell corresponding to the filling option, performing image processing on the image region where the cell is located to obtain a mask filled in the cell; By calculating the edge gradient change of each mask, the filling shape gradient information of each filling option is obtained; Determining the unfilled filling options and the filled filling options based on the filling shape gradient information; Based on the filling shape gradient information of the filled filling option, the overall filling style information of the target filling information card in the current image is obtained.

4. The method according to claim 3, characterized in that The overall filling style information includes: the standard deviation and median of the filling shape gradient information of the filled filling options; and obtaining the analysis results of each filling option in the target filling information card based on the overall filling style information and the filling shape gradient information includes: obtaining a difference threshold based on a first difference between the standard deviation and the median; For each filled filling option, obtaining a second difference between the filling shape gradient information of the filling option and the median value; When the second difference is less than or equal to the difference threshold, obtaining an analysis result that the filled option is a valid fill; When the second difference is greater than the difference threshold, obtaining an analysis result that the filled option is invalid; According to the analysis results of the filled-in options and the unfilled-in options, the analysis results of each of the filling options in the target filling information card are obtained.

5. The method according to claim 1, wherein The performing of table structure analysis on the image of the target filled-in information card to obtain structural information of cells in the image includes: Performing morphological processing on the image of the target filled-in information card to obtain a first mask image; Performing polygonal approximation processing on the first mask image to screen and obtain second mask images of each rectangular table; extracting an image of a rectangular table from the image based on the second mask image; performing correction processing on the image of the rectangular table to obtain a corrected image of the table; The corrected image is split into cells to obtain structural information of the cells in the table.

6. The method according to claim 5, characterized in that The step of performing correction processing on the image of the rectangular table to obtain a corrected image of the table includes: Based on the second mask image, performing table trend adjustment processing on the image of the rectangular table to obtain an image of a horizontal table; determining the size of one or more of the following types of elements in the image of the horizontal table: cells, horizontal line segments, connected text, and discrete matrices; Searching for a target element in the elements according to a preset element priority, wherein the size of the target element and the size of the horizontal table meet a preset condition; In the case where the target element is found, determining the inclination angle of the horizontal table according to the inclination angle of the target element; In the case where the target element is not found, determining the tilt angle of the horizontal table according to the tilt angle distribution of the element with the highest priority; The image of the horizontal form is corrected based on the tilt angle of the horizontal form.

7. A filling option analysis device, characterized in that: The device comprises: A cell structure information acquisition module is used to perform table structure analysis on the image of the target filled-in information card to obtain the structural information of the cells in the image; a filling option position information acquisition module, configured to acquire cell position information corresponding to each filling option in the image based on the structural information, a mapping relationship between the pre-configured filling options of the target filling information card and the table structure; The filling option analysis module is used to analyze the content of the image area corresponding to the filling option based on the cell position information to obtain the analysis results of each filling option in the target filling information card.

8. The device according to claim 7, characterized in that The filling option analysis module is further used to: Based on the cell position information, determining the image area where the cell corresponding to each of the filling options in the image is located; Performing gradient analysis on the content of the image region where the cells corresponding to the filling options are located to obtain overall filling style information and filling shape gradient information of the filling options; Based on the overall filling style information and the filling shape gradient information, an analysis result of each filling option in the target filling information card is obtained.

9. An electronic device comprising a memory, a processor, and a program code stored in the memory and executable on the processor, wherein: When the processor executes the program code, the method according to any one of claims 1 to 6 is implemented.

10. A computer-readable storage medium having program code stored thereon, characterized in that: When the program code is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images