Paper image standardization method and device, computer device and storage medium

Abstract

The application relates to the technical field of computer vision, and provides a paper image standardization method, device, computer equipment and storage medium, the method comprising the following steps: extracting a paper content area of a paper image to be processed and extracting pixel point coordinates of a contour of the paper content area to obtain a contour pixel point coordinate list; screening contour vertex coordinates from the contour pixel point coordinate list based on a maximum extreme value feature of vertex distance and a right angle feature of vertex; and performing standardization processing on the paper image to be processed according to the contour vertex coordinates to obtain a standardized paper image. The method can improve the accuracy of text reconstruction.

Description

Technical Field

[0001] This invention belongs to the field of computer vision technology, and in particular relates to a method, apparatus, computer equipment, and storage medium for standardizing paper images. Background Technology

[0002] Paper media is currently the primary medium for transmitting information in people's daily lives, and its convenience and portability make it widely popular. However, due to the perishable nature of paper media and the inconvenience of editing paper information, a series of problems arise, requiring manual editing of paper text content into electronic media such as Word and PDF, which severely impacts work efficiency. Therefore, to solve this tedious task, academia and industry have launched a research project on document reconstruction, the core of which is to convert paper documents into electronic documents based on images of paper.

[0003] However, in practical applications, users take photos of paper documents arbitrarily and without control, which leads to a series of problems such as distortion, occlusion, and deformation of paper images. These images differ significantly in shape from standard paper documents, greatly increasing the difficulty of document reconstruction and reducing the accuracy of document reconstruction. Summary of the Invention

[0004] Therefore, it is necessary to provide a paper image standardization method, apparatus, computer equipment, and storage medium that can improve the accuracy of text reconstruction in response to the above-mentioned technical problems.

[0005] This invention provides a paper image standardization method, comprising:

[0006] Extract the paper content region of the paper image to be processed and extract the pixel coordinates of the outline of the paper content region to obtain a list of outline pixel coordinates.

[0007] The contour vertex coordinates are filtered from the list of contour pixel coordinates based on the maximum extreme value of vertex distance and the right angle of vertex.

[0008] The paper image to be processed is standardized based on the coordinates of the contour vertices to obtain a standardized paper image.

[0009] In one embodiment, the step of filtering contour vertex coordinates from the list of contour pixel coordinates based on the vertex distance maximum extremum feature and vertex right angle feature includes:

[0010] Based on the maximum extreme value feature of vertex distance, the contour pixel coordinates are selected from the list of contour pixel coordinates as candidate contour vertex coordinates;

[0011] The vertex coordinates of the contour are determined from the vertex coordinates of the candidate contour based on the right-angle feature of the vertex.

[0012] In one embodiment, the step of filtering contour pixel coordinates from the list of contour pixel coordinates as candidate contour vertex coordinates based on the vertex distance maximum extremum feature includes:

[0013] The coordinates of the center point of the contour are determined based on the coordinates of each contour pixel in the contour pixel coordinate list.

[0014] The outline pixel coordinate list is regarded as a loop with the beginning and end connected. The outline pixel coordinates of each outline pixel coordinate in the outline pixel coordinate list are traversed. The first preset number of outline pixel coordinates before the current outline pixel coordinate and the first preset number of outline pixel coordinates after the current outline pixel coordinate are combined into an outline pixel coordinate group.

[0015] Calculate the distance between the coordinates of each contour pixel in the contour pixel coordinate group and the coordinates of the contour center point to obtain the pixel distance group;

[0016] Using the distance corresponding to the current contour pixel coordinates as the dividing point, when the distances in the pixel distance group before the dividing point are increasing and the distances after the dividing point are decreasing, the current contour pixel coordinates are used as candidate contour vertex coordinates.

[0017] In one embodiment, determining the contour vertex coordinates from the candidate contour vertex coordinates based on vertex right-angle features includes:

[0018] Traverse the coordinates of each candidate contour vertex, and based on the position of the current candidate contour vertex coordinate in the contour pixel coordinate list, obtain the coordinates of the second preset number of contour pixels before the current candidate contour vertex coordinate and perform line fitting to obtain the first line function parameters; and obtain the coordinates of the second preset number of contour pixels after the current candidate contour vertex coordinate and perform line fitting to obtain the second line function parameters.

[0019] The angle between the lines used to calculate the coordinates of the current candidate contour vertex is obtained by using the first line function parameters and the second line function parameters to obtain a list of the angles between the lines corresponding to the coordinates of each candidate contour vertex.

[0020] Based on the distance between each straight angle and a right angle in the list of straight angles, the coordinates of the four candidate contour vertices with the smallest distance are selected as the contour vertex coordinates.

[0021] In one embodiment, the step of standardizing the paper image to be processed based on the contour vertex coordinates to obtain a standardized paper image includes:

[0022] Based on the industry standard dimensions of the paper contained in the paper image to be processed, determine the standard vertex coordinates of the paper image to be processed;

[0023] Calculate the transformation matrix based on the standard vertex coordinates and the contour vertex coordinates;

[0024] The transformation matrix is ​​used to standardize the paper image to be processed, resulting in a standardized paper image.

[0025] In one embodiment, extracting the paper content region of the paper image to be processed includes:

[0026] The process involves acquiring an image of the paper to be processed and calling a semantic segmentation network trained using knowledge distillation.

[0027] The paper image to be processed is input into the semantic segmentation network, which segments the paper image to obtain the paper content region.

[0028] In one embodiment, the step of extracting the pixel coordinates of the outline of the paper content area to obtain a list of outline pixel coordinates includes:

[0029] After magnifying the paper content area, edge detection is performed to obtain the pixel coordinates of the paper content area outline;

[0030] The coordinates of each pixel point are arranged in clockwise order according to the contour to obtain a list of contour pixel point coordinates.

[0031] A paper image standardization device, comprising:

[0032] The extraction module is used to extract the paper content area of ​​the paper image to be processed and extract the pixel coordinates of the outline of the paper content area to obtain a list of outline pixel coordinates.

[0033] The vertex filtering module is used to filter the vertex coordinates from the list of contour pixel coordinates based on the maximum vertex distance feature and the vertex right angle feature.

[0034] The standardization processing module is used to standardize the paper image to be processed based on the coordinates of the contour vertices to obtain a standardized paper image.

[0035] The present invention also provides a computer device, the computer device including a processor and a memory, the memory storing a computer program, the processor executing the computer program to implement the steps of the paper image standardization method described in any of the above claims.

[0036] The present invention also provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of the paper image standardization method described in any of the preceding claims.

[0037] The aforementioned paper image standardization method, apparatus, computer equipment, and storage medium locate and extract the paper region from a complex image, then find the contour vertices of the paper region based on the inherent invariant features of the paper's contour and right angles, and finally standardize the paper image at any angle based on the contour vertices to obtain a standardized paper image. This provides a standard paper image for subsequent steps in document reconstruction, such as text layout analysis, text detection, and text recognition, greatly reducing the difficulty of subsequent steps and improving the accuracy of text reconstruction tasks. Attached Figure Description

[0038] Figure 1 This is a diagram illustrating the application environment of a paper image standardization method in one embodiment.

[0039] Figure 2 This is a flowchart illustrating a paper image standardization method in one embodiment.

[0040] Figure 3 This is a schematic diagram of the paper image to be processed in one embodiment.

[0041] Figure 4 This is a schematic diagram of the paper content area in one embodiment.

[0042] Figure 5 This is a structural block diagram of a paper image standardization device in one embodiment. Detailed Implementation

[0043] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0044] The paper image standardization method provided in this application can be applied to, for example... Figure 1In the application environment shown, the application environment involves terminal 102 and server 104. Terminal 102 communicates with server 104 via a network. Terminal 102 can be, but is not limited to, various personal computers, laptops, smartphones, tablets, and portable wearable devices. Server 104 can be a standalone server or a server cluster consisting of multiple servers. The paper image standardization method can be implemented by terminal 102 alone or by server 104. Taking server 104 as an example, specifically, server 104 extracts the paper content region of the paper image to be processed and extracts the pixel coordinates of the outline of the paper content region, obtaining a list of outline pixel coordinates; it then filters the outline vertex coordinates from the list of outline pixel coordinates based on the vertex distance maximum extremum feature and vertex right angle feature; finally, it performs standardization processing on the paper image to be processed according to the outline vertex coordinates to obtain a standardized paper image.

[0045] In one embodiment, such as Figure 2 As shown, a paper image standardization method is provided. Taking the application of this method to a server as an example, the method includes the following steps:

[0046] Step S201: Extract the paper content area of ​​the paper image to be processed and extract the pixel coordinates of the outline of the paper content area to obtain a list of outline pixel coordinates.

[0047] The paper image to be processed refers to an image containing paper content. This image can be an image containing only paper content, or it can include both paper content and other object content. For example... Figure 3 As shown, a schematic diagram of a paper image to be processed is provided. Figure 3 The image shown includes other objects besides the paper content, such as a keyboard and napkins. The paper content area refers to the area containing only the paper content, such as... Figure 4 The diagram shown provides a schematic representation of a paper content area. Figure 4 The central gray rectangular area is Figure 3 Includes the paper content area.

[0048] Specifically, when the server receives a paper image standardization task, it first obtains the paper image to be processed, then extracts the paper content area in the paper image to be processed, and then extracts the pixels on the outline of the paper content area to obtain a list of outline pixel coordinates.

[0049] Step S202: Filter the contour vertex coordinates from the contour pixel coordinate list based on the maximum vertex distance feature and the vertex right angle feature.

[0050] Since the vertex of the paper is always the furthest from the center of the paper relative to its surrounding pixels, this principle applies even in scenarios involving image distortion and warping. Furthermore, the vertex angle is a right angle; although the vertex angle will change somewhat under image distortion and warping, it will still be the point closest to a right angle. Therefore, this property can be used to filter out invalid points to obtain the vertices.

[0051] In one embodiment, step S202 includes: filtering contour pixel coordinates from the list of contour pixel coordinates as candidate contour vertex coordinates based on the maximum extreme value feature of vertex distance; and determining contour vertex coordinates from the candidate contour vertex coordinates based on the vertex right angle feature.

[0052] Specifically, since the contour coordinates that can be filtered by the maximum extreme value feature of the distance from the center point may not satisfy the right angle feature, when filtering contour vertex coordinates based on the maximum extreme value feature of vertex distance and vertex right angle feature, the maximum extreme value feature of vertex distance is first used to filter out a portion of the candidate vertex coordinates from the list of contour pixel coordinates, and then the right angle feature of vertex is used to filter out invalid candidate vertices from the candidate vertex coordinates to obtain the determined vertex coordinates.

[0053] Step S203: Standardize the paper image to be processed according to the contour vertex coordinates to obtain a standardized paper image.

[0054] Specifically, after determining the contour vertex coordinates of the paper image to be processed, the paper image to be processed is transformed based on the contour vertex coordinates and the standard vertex coordinates of the paper to complete the standardization process and obtain a standardized paper image.

[0055] The aforementioned paper image standardization method locates and extracts the paper region from a complex image, then identifies the contour vertices of the paper region based on the inherent invariant features of the paper's outline and right angles, and finally standardizes the paper image at any angle based on the contour vertices to obtain a standardized paper image. This standard paper image can provide a series of subsequent steps in the document reconstruction process, such as text layout analysis, text detection, and text recognition, greatly reducing the difficulty of subsequent steps and improving the accuracy of text reconstruction tasks.

[0056] In one embodiment, selecting contour pixel coordinates as candidate contour vertex coordinates from the contour pixel coordinate list based on the maximum extreme value feature of vertex distance includes: determining the contour center point coordinates according to each contour pixel coordinate in the contour pixel coordinate list; treating the contour pixel coordinate list as a loop with its beginning and end connected, traversing each contour pixel coordinate in the contour pixel coordinate list, and forming a contour pixel coordinate group by combining a first preset number of contour pixel coordinates before the current contour pixel coordinate and a first preset number of contour pixel coordinates after the current contour pixel coordinate; calculating the distance between each contour pixel coordinate in the contour pixel coordinate group and the contour center point coordinate to obtain a pixel distance group; using the distance corresponding to the current contour pixel coordinate as the dividing point, when the distances in the pixel distance group before the dividing point are increasing and the distances after the dividing point are decreasing, the current contour pixel coordinate is selected as the candidate contour vertex coordinate.

[0057] Here, the outline center point coordinates refer to the coordinates of the center point of the paper contained in the paper image to be processed. The current outline pixel coordinates refer to the coordinates of the outline pixel point currently accessed when sequentially traversing the outline pixel coordinates list. The first preset quantity is a pre-set value used to determine the number of selected pixel coordinates, and can be set according to actual needs.

[0058] Specifically, in complex and distorted scenes, the distance from the paper vertex to the center point is the farthest relative to other pixels. Therefore, this embodiment utilizes this characteristic to accurately filter candidate contour vertices from the contour pixel coordinate list. First, the coordinates (x, y, z) of the contour center point are obtained from the contour pixel coordinate list IContour. c y y ), coordinates of the center point of the contour (x c y y The calculation formula is as follows:

[0059]

[0060]

[0061] Where n represents the total number of contour pixel coordinates in the list IContour, (x i y i Let represent the coordinates of the i-th contour pixel in the contour pixel coordinate list IContour, where i = {1, 2, 3, ..., n}. Then, consider the contour pixel coordinate list IContour as a loop with its beginning and end connected. For any contour pixel coordinate (x...) within it... i y iTake the k contour pixels located before it and the k contour pixels located after it to form a contour pixel coordinate group, where k is the preset first number. The contour pixel coordinate group is recorded as PC, as follows:

[0062] PC = {(x i-k y i-k ), (x i-k+1 y i-k+1 ), ..., (x i y i ), ..., (x i+k-1 y i+k-1 ), (x i+k y i+k )}

[0063] Secondly, the distance (x) between each contour pixel in the contour pixel coordinate group PC and the coordinates of the contour center point is calculated sequentially. c y y The pixel distance group DisPC is obtained as follows:

[0064] DisPC = {d i-k d i-k+1 , ...,d i , ..., d i+k-1 d i+k}

[0065] Finally, if the first k elements of the pixel distance group DisPC are increasing and the last k elements are decreasing, that is, using the current contour pixel coordinates (x... i y i The corresponding distance d i For each boundary point, determine whether the distance before it is increasing and the distance after it is decreasing. If so, determine the coordinates (x, y) of the contour pixel. i y i If the vertex coordinates of a candidate contour meet the requirements, then they are considered as candidate contour vertex coordinates. By traversing each contour pixel coordinate in the IContour list as described above, all candidate contour vertex coordinates are finally obtained.

[0066] In one embodiment, determining the contour vertex coordinates from the candidate contour vertex coordinates based on the right angle feature includes: traversing each candidate contour vertex coordinate; based on the position of the current candidate contour vertex coordinate in the contour pixel coordinate list, obtaining a first linear function parameter by performing linear fitting on the second preset number of contour pixel coordinates preceding the current candidate contour vertex coordinate; obtaining a second linear function parameter by performing linear fitting on the second preset number of contour pixel coordinates following the current candidate contour vertex coordinate; calculating the angle between the lines of the current candidate contour vertex coordinate using the first and second linear function parameters to obtain a list of angles between the lines of each candidate contour vertex coordinate; and selecting the four candidate contour vertex coordinates with the smallest distance from the right angle in the list of angles between the lines of the current candidate contour vertex coordinates as the contour vertex coordinates.

[0067] Here, the current candidate contour vertex coordinates are the coordinates of the candidate contour vertex currently visited during the traversal of all candidate contour vertex coordinates. The second preset quantity is the same as the first preset quantity and is a pre-set value used to determine the number of selected pixel coordinates, which can be set according to actual needs. The second preset quantity and the first preset quantity can be equal or unequal.

[0068] Specifically, the coordinates of each candidate contour vertex are traversed sequentially. For the coordinates of the current candidate contour vertex (x... j y j To locate its position in the outline pixel coordinate list IContour, similarly, take the outline pixel coordinates t before it and the outline pixel coordinates t after it in the outline pixel coordinate list IContour to form a pixel coordinate group SC.

[0069] SC={(x j-t y j-t ), (x j-t+1 y j-t+1 ), ..., (x j y j ), ..., (x j+t-1 y j+t-1 ), (x j+t y j+t )}

[0070] Then, a straight line fit is performed on the coordinates of the first t contour pixels in SC to obtain the first straight line function parameters. Similarly, a straight line fit is performed on the coordinates of the last t contour pixels in SC to obtain the second straight line function parameters. In this embodiment, any existing method can be used for straight line fitting, such as the least squares method or gradient descent method. This embodiment preferably uses the least squares method for straight line fitting. The first and second straight line functions obtained by fitting are illustrated below:

[0071] y t1 =a1*x t1 +b1

[0072] y t2 =a2*x t2 +b2

[0073] Then, using the parameters a1 and b1 of the first linear function and the parameters a2 and b2 of the second linear function, the coordinates (x, y) of the current candidate contour vertex are calculated. j y j The angle θ between the lines j The above steps calculate the included angles between the lines of all candidate contour vertices, forming a list of included angles IAngle = {θ}. j-1 ,…θ j ,…,θ j+1}. Angle θ between the lines j The calculation formula is as follows:

[0074]

[0075] Finally, calculate the distance between each line angle in the list and the right angle π / 2, and record it as IAngle_Dis={|θ j-1 -π / 2|,…|θ j -π / 2|,…,|θ j+1 -π / 2|}. Select the four smallest elements from IAngle_Dis, and then select four candidate contour vertex coordinates from the candidate contour vertex coordinates according to the corresponding relationship as the final determined contour vertex coordinates. The contour vertex coordinates can be recorded as ISV={(x s1 ,y s1 ), (x s2 ,y s2 ), (x s3 ,y s3 ), (x s4 ,y s4 )}, where (x s1 ,y s1 (x) is the top left vertex, (x) s2 ,y s2 (x) is the top right vertex, (x) s3 ,y s3 (x) is the bottom right vertex. s4 ,y s4The bottom left vertex is defined as follows: First, the top left vertex is determined by selecting the vertex with the smallest y-coordinate among points whose x-coordinate is less than the average of the four vertices' x-coordinates. Then, based on the corresponding positional relationships in the IContour list of contour pixel coordinates, the top right, bottom right, and bottom left vertices are obtained sequentially by sorting them clockwise. In this embodiment, the final contour vertex coordinates are obtained by filtering out candidate points whose features are ineffective due to the paper vertices being right angles and still tending towards right angles even under image distortion and deformation, thus improving the accuracy of vertex acquisition.

[0076] In one embodiment, step S203 includes: determining the standard vertex coordinates of the paper image to be processed based on the industry standard size of the paper contained in the paper image to be processed; calculating a transformation matrix based on the standard vertex coordinates and the contour vertex coordinates; and using the transformation matrix to standardize the paper image to be processed to obtain a standardized paper image.

[0077] Specifically, the standard vertex coordinates of the paper image to be processed are first determined based on the industry standard size of the paper contained within it. For example, if the paper image contains A4 paper, then the standard width w of the A4 paper can be determined. A4 and high h A4 This will give us the standard top-left vertex (0,0) and the standard top-right vertex (w). A4 ,0), standard bottom right vertex (w) A4 ,h A4 ) and the standard bottom left vertex (0, h A4 Then, based on the contour vertex coordinates (x... s1 ,y s1 ), (x s2 ,y s2 ), (x s3 ,y s3 ), (x s4 ,y s4 ) and standard vertex coordinates (0,0), (w A4 ,0), (w A4 ,h A4 ), (0,h A4 Based on the correspondence between the coordinates of the paper and the outline vertex coordinates, the transformation matrix M is calculated using perspective transformation in image processing. Finally, the paper image to be processed is normalized using the transformation matrix M to obtain a normalized paper image. In this embodiment, the paper is normalized based on the standard vertex coordinates and outline vertex coordinates, which can improve the accuracy of the normalization process.

[0078] In one embodiment, extracting the paper content region of a paper image to be processed includes: acquiring the paper image to be processed, and calling a semantic segmentation network trained using knowledge distillation; inputting the paper image to be processed into the semantic segmentation network, which then segments the paper image to be processed to obtain the paper content region.

[0079] Specifically, the server acquires a paper image I to be processed and extracts the paper content regions from the paper image I based on a semantic segmentation network. In this embodiment, the semantic segmentation network can employ any existing neural network architecture; however, the PIDNet network architecture is preferred. A lightweight semantic segmentation network, PID_Small, and a heavyweight network, PID_Large, are built based on the PIDNet network architecture. A batch of paper images containing paper content regions are collected, and annotation tools such as LabelMe are used to label the paper content regions in the paper images to obtain corresponding image labels. These labeled paper images are then used to construct a training dataset S based on an 8:2 ratio. Next, the heavyweight semantic segmentation network PID_Large is trained based on the training dataset S, with the training parameters remaining consistent with the original PIDNet network architecture. The trained heavyweight network PID_Large model is recorded as PID_Large_Frozen. Using PID_Large_Frozen as the Teacher model and the lightweight semantic segmentation network PID_Small as the Student model, the Student model is trained using knowledge distillation on the training dataset S. The trained model is recorded as PID_Small_Frozen, considered as the trained semantic segmentation model. Subsequently, the PID_Small_Frozen model is used to process the paper image I to obtain the paper content regions. That is, Figure 3 The paper image to be processed is input into the PID_Small_Frozen model for processing, and the output is... Figure 4 The image shown, Figure 3 and Figure 4 The images are the same size, the difference is... Figure 4 The pixel values ​​in the content area of ​​the paper are different from the pixel values ​​in other areas, for example... Figure 4 If the pixel value in the content area of ​​the paper is 1, then the pixel value in other areas is 0. The knowledge distillation loss function kd_loss is defined as follows in this embodiment:

[0080] kd_loss = loss so +KLDivLoss(log_softmax(so / T),softmax(to / T))*α*T 2

[0081] Where, loss so is the loss function of the PID_Small network itself. KLDivLoss, log_softmax, and softmax are common functions in deep learning frameworks. so and to represent the output results after merging the PID_Small network and the PID_Large_Frozen model, respectively. a is the weight balancing coefficient, and T is the temperature of knowledge distillation.

[0082] In one embodiment, extracting the pixel coordinates of the outline of the paper content area to obtain a list of outline pixel coordinates includes: performing edge detection after magnifying the paper content area to obtain the pixel coordinates of the outline of the paper content area; and arranging the pixel coordinates in clockwise order according to the outline to obtain a list of outline pixel coordinates.

[0083] Specifically, when the server extracts the outline pixel coordinate list, it first multiplies the paper content area by 225 for magnification. For example, a pixel value of 1 means the pixel value of the paper content area is 225, while the pixel values ​​of other areas remain unchanged. Then, edge detection is performed on the magnified image of the paper content area. In this embodiment, the edge detection operation is preferably implemented using the Canny operator, thereby obtaining the pixel coordinates of each pixel on the outline of the paper content area. Subsequently, the outline vertex coordinates need to be filtered based on the maximum vertex distance feature and the vertex right angle feature. For ease of processing, the pixel coordinates are then arranged clockwise according to the outline, resulting in a list of outline pixel coordinates arranged clockwise. For example, using... Figure 4 Taking the image shown as an example, it is about to Figure 4 After multiplying the pixel values ​​of the paper content area shown in the image by 225, then... Figure 4 Edge detection is performed on the enlarged content area of ​​the paper to obtain the coordinates of each pixel on the outline.

[0084] It should be understood that, although Figure 2 The steps in the flowchart are shown sequentially as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order in which these steps are executed, and they can be performed in other orders. Figure 2 At least some of the steps in the process may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but may be executed at different times. The execution order of these steps or stages is not necessarily sequential, but may be executed in turn or alternately with other steps or at least some of the steps or stages in other steps.

[0085] In one embodiment, such as Figure 5As shown, a paper image standardization device is provided, comprising:

[0086] Extraction module 501 is used to extract the paper content area of ​​the paper image to be processed and extract the pixel coordinates of the outline of the paper content area to obtain a list of outline pixel coordinates.

[0087] Vertex filtering module 502 is used to filter contour vertex coordinates from the contour pixel coordinate list based on the maximum extreme value feature of vertex distance and the right angle feature of vertex.

[0088] The standardization processing module 503 is used to standardize the paper image to be processed according to the coordinates of the contour vertices to obtain a standardized paper image.

[0089] In one embodiment, the vertex filtering module 502 is further configured to filter the contour pixel coordinates from the list of contour pixel coordinates as candidate contour vertex coordinates based on the maximum extreme value feature of vertex distance; and determine the contour vertex coordinates from the candidate contour vertex coordinates based on the vertex right angle feature.

[0090] In one embodiment, the vertex filtering module 502 is further configured to determine the coordinates of the center point of the contour based on the coordinates of each contour pixel in the contour pixel coordinate list; treat the contour pixel coordinate list as a loop with its beginning and end connected, traverse each contour pixel coordinate in the contour pixel coordinate list, and form a contour pixel coordinate group by combining a first preset number of contour pixel coordinates before the current contour pixel coordinate and a first preset number of contour pixel coordinates after the current contour pixel coordinate; calculate the distance between each contour pixel coordinate in the contour pixel coordinate group and the coordinates of the center point of the contour, and obtain a pixel distance group; take the distance corresponding to the current contour pixel coordinate as the dividing point, and when the distances in the pixel distance group before the dividing point are increasing and the distances after the dividing point are decreasing, use the current contour pixel coordinate as the candidate contour vertex coordinate.

[0091] In one embodiment, the vertex filtering module 502 is further configured to traverse the coordinates of each candidate contour vertex, and based on the position of the current candidate contour vertex coordinates in the contour pixel coordinate list, obtain a second preset number of contour pixel coordinates before the current candidate contour vertex coordinates and perform line fitting to obtain a first line function parameter, and obtain a second preset number of contour pixel coordinates after the current candidate contour vertex coordinates and perform line fitting to obtain a second line function parameter; calculate the line angle between the current candidate contour vertex coordinates using the first line function parameter and the second line function parameter to obtain a list of line angles corresponding to each candidate contour vertex coordinate; and filter the four candidate contour vertex coordinates with the smallest distance as the contour vertex coordinates based on the distance between each line angle in the line angle list and the right angle.

[0092] In one embodiment, the standardization processing module 503 is further configured to determine the standard vertex coordinates of the paper image to be processed based on the industry standard size of the paper contained in the paper image to be processed; calculate the transformation matrix based on the standard vertex coordinates and the contour vertex coordinates; and use the transformation matrix to perform standardization processing on the paper image to be processed to obtain a standardized paper image.

[0093] In one embodiment, the extraction module 501 is further configured to acquire the paper image to be processed and to call the semantic segmentation network trained using the knowledge distillation method; input the paper image to be processed into the semantic segmentation network, and have the semantic segmentation network segment the paper image to be processed to obtain the paper content region.

[0094] In one embodiment, the extraction module 501 is further configured to perform edge detection after magnifying the paper content area to obtain the pixel coordinates of the paper content area outline; and arrange the pixel coordinates in a clockwise direction according to the outline to obtain a list of outline pixel coordinates.

[0095] Specific limitations regarding the paper image standardization device can be found in the limitations of the paper image standardization method described above, and will not be repeated here. Each module in the aforementioned paper image standardization device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device in hardware form, or stored in the memory of a computer device in software form, so that the processor can call and execute the operations corresponding to each module. Based on this understanding, all or part of the processes in the methods of the above embodiments of the present invention can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various paper image standardization method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms.

[0096] In one embodiment, a computer device is provided, which may be a server, including a processor, memory, and a network interface. The processor of the computer device provides computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores an operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. The database of the computer device is used to store data. The network interface of the computer device is used to communicate with external terminals via a network connection. When the computer program is executed by the processor, it implements a paper image standardization method. Exemplarily, the computer program may be divided into one or more modules, one or more of which are stored in memory and executed by the processor to complete the invention. The one or more modules may be a series of computer program instruction segments capable of performing specific functions, which describe the execution process of the computer program in the computer device.

[0097] The processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor can be a microprocessor or any conventional processor. The processor is the control center of the computer device, connecting various parts of the computer device via various interfaces and lines.

[0098] The memory can be used to store the computer programs and / or modules. The processor implements various functions of the computer device by running or executing the computer programs and / or modules stored in the memory and by calling data stored in the memory. The memory may mainly include a program storage area and a data storage area. The program storage area may store the operating system, at least one application program required for a function (such as sound playback function, image playback function, etc.), etc.; the data storage area may store data created according to the use of the mobile phone (such as audio data, phonebook, etc.). In addition, the memory may include high-speed random access memory, and may also include non-volatile memory, such as hard disk, RAM, plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, at least one disk storage device, flash memory device, or other volatile solid-state storage device.

[0099] Those skilled in the art will understand that the computer device structure shown in this embodiment is only a partial structure related to the present invention and does not constitute a limitation on the computer device to which the present invention is applied. The specific computer device may include more or fewer components, or combine certain components, or have different component arrangements.

[0100] In one embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to perform the following steps:

[0101] Extract the paper content region of the paper image to be processed and extract the pixel coordinates of the outline of the paper content region to obtain a list of outline pixel coordinates.

[0102] Filter the contour vertex coordinates from the list of contour pixel coordinates based on the maximum extreme value of vertex distance and the right angle of vertex;

[0103] The paper image to be processed is standardized based on the coordinates of the contour vertices to obtain a standardized paper image.

[0104] In one embodiment, when the processor executes the computer program, it further performs the following steps: filtering contour pixel coordinates from the list of contour pixel coordinates as candidate contour vertex coordinates based on the vertex distance maximum extreme value feature; and determining the contour vertex coordinates from the candidate contour vertex coordinates based on the vertex right angle feature.

[0105] In one embodiment, when the processor executes the computer program, it further implements the following steps: determining the coordinates of the center point of the contour based on the coordinates of each contour pixel in the contour pixel coordinate list; treating the contour pixel coordinate list as a loop with its beginning and end connected, traversing each contour pixel coordinate in the contour pixel coordinate list, and forming a contour pixel coordinate group by combining a first preset number of contour pixel coordinates before the current contour pixel coordinate and a first preset number of contour pixel coordinates after the current contour pixel coordinate; calculating the distance between each contour pixel coordinate in the contour pixel coordinate group and the coordinates of the center point of the contour, respectively, to obtain a pixel distance group; taking the distance corresponding to the current contour pixel coordinate as the dividing point, when the distances in the pixel distance group before the dividing point are increasing and the distances after the dividing point are decreasing, the current contour pixel coordinate is used as the candidate contour vertex coordinate.

[0106] In one embodiment, when the processor executes the computer program, it further performs the following steps: traversing the coordinates of each candidate contour vertex, based on the position of the current candidate contour vertex coordinates in the contour pixel coordinate list, obtaining a first linear function parameter by performing linear fitting on the coordinates of a second preset number of contour pixels preceding the current candidate contour vertex coordinates, and obtaining a second linear function parameter by performing linear fitting on the coordinates of a second preset number of contour pixels following the current candidate contour vertex coordinates; calculating the angle between the lines of the current candidate contour vertex coordinates using the first and second linear function parameters, and obtaining a list of angles between the lines of each candidate contour vertex coordinate; and selecting the four candidate contour vertex coordinates with the smallest distance from the right angle in the list of angles between the lines of the current candidate contour vertex coordinates as the contour vertex coordinates.

[0107] In one embodiment, when the processor executes the computer program, it further performs the following steps: determining the standard vertex coordinates of the paper image to be processed based on the industry standard size of the paper contained in the paper image to be processed; calculating the transformation matrix based on the standard vertex coordinates and the contour vertex coordinates; and using the transformation matrix to standardize the paper image to be processed to obtain a standardized paper image.

[0108] In one embodiment, when the processor executes the computer program, it further performs the following steps: acquiring a paper image to be processed, and calling a semantic segmentation network trained using knowledge distillation; inputting the paper image to be processed into the semantic segmentation network, which then segments the paper image to obtain the paper content region.

[0109] In one embodiment, when the processor executes the computer program, it further performs the following steps: after magnifying the paper content area, it performs edge detection to obtain the pixel coordinates of the paper content area outline; and arranges the coordinates of each pixel point in clockwise order according to the outline to obtain a list of outline pixel coordinates.

[0110] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, the computer program performing the following steps when executed by a processor:

[0111] Extract the paper content region of the paper image to be processed and extract the pixel coordinates of the outline of the paper content region to obtain a list of outline pixel coordinates.

[0112] Filter the contour vertex coordinates from the list of contour pixel coordinates based on the maximum extreme value of vertex distance and the right angle of vertex;

[0113] The paper image to be processed is standardized based on the contour vertex coordinates to obtain a standardized paper image. In one embodiment, when the computer program is executed by the processor, it further implements the following steps: filtering contour pixel coordinates from the list of contour pixel coordinates as candidate contour vertex coordinates based on the maximum extreme value feature of vertex distance; and determining the contour vertex coordinates from the candidate contour vertex coordinates based on the vertex right angle feature.

[0114] In one embodiment, when the computer program is executed by the processor, it further implements the following steps: determining the coordinates of the center point of the contour based on the coordinates of each contour pixel in the contour pixel coordinate list; treating the contour pixel coordinate list as a loop with its beginning and end connected, traversing each contour pixel coordinate in the contour pixel coordinate list, and forming a contour pixel coordinate group by combining a first preset number of contour pixel coordinates before the current contour pixel coordinate and a first preset number of contour pixel coordinates after the current contour pixel coordinate; calculating the distance between each contour pixel coordinate in the contour pixel coordinate group and the coordinates of the center point of the contour, respectively, to obtain a pixel distance group; taking the distance corresponding to the current contour pixel coordinate as the dividing point, when the distances in the pixel distance group before the dividing point are increasing and the distances after the dividing point are decreasing, the current contour pixel coordinate is used as the candidate contour vertex coordinate.

[0115] In one embodiment, when the computer program is executed by the processor, it further performs the following steps: traversing the coordinates of each candidate contour vertex; based on the position of the current candidate contour vertex coordinates in the contour pixel coordinate list, obtaining a second preset number of contour pixel coordinates before the current candidate contour vertex coordinates and performing line fitting to obtain a first line function parameter; and obtaining a second preset number of contour pixel coordinates after the current candidate contour vertex coordinates and performing line fitting to obtain a second line function parameter; calculating the line angles between the current candidate contour vertex coordinates using the first and second line function parameters to obtain a list of line angles corresponding to each candidate contour vertex coordinate; and selecting the four candidate contour vertex coordinates with the smallest distance from each line angle in the line angle list as the contour vertex coordinates.

[0116] In one embodiment, when the computer program is executed by the processor, it further performs the following steps: determining the standard vertex coordinates of the paper image to be processed based on the industry standard size of the paper contained in the paper image to be processed; calculating the transformation matrix based on the standard vertex coordinates and the contour vertex coordinates; and using the transformation matrix to standardize the paper image to be processed to obtain a standardized paper image.

[0117] In one embodiment, when the computer program is executed by the processor, it further performs the following steps: acquiring a paper image to be processed, and calling a semantic segmentation network trained using knowledge distillation; inputting the paper image to be processed into the semantic segmentation network, which then segments the paper image to obtain the paper content region.

[0118] In one embodiment, when the computer program is executed by the processor, it further performs the following steps: after magnifying the paper content area, it performs edge detection to obtain the pixel coordinates of the paper content area outline; and arranges the coordinates of each pixel point in clockwise order according to the outline to obtain a list of outline pixel coordinates.

[0119] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the methods described above. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, or optical storage, etc. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc.

[0120] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0121] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this patent application should be determined by the appended claims.

Claims

1. A method for standardizing paper images, characterized in that, include: Extract the paper content region of the paper image to be processed and extract the pixel coordinates of the outline of the paper content region to obtain a list of outline pixel coordinates. The selection of contour vertex coordinates from the contour pixel coordinate list based on the maximum vertex distance feature and the vertex right angle feature includes: selecting contour pixel coordinates as candidate contour vertex coordinates from the contour pixel coordinate list based on the maximum vertex distance feature, specifically including: determining the contour center point coordinates according to each contour pixel coordinate in the contour pixel coordinate list; treating the contour pixel coordinate list as a loop, traversing each contour pixel coordinate in the contour pixel coordinate list, forming a contour pixel coordinate group by combining a first preset number of contour pixel coordinates before the current contour pixel coordinate and a first preset number of contour pixel coordinates after the current contour pixel coordinate; calculating the distance between each contour pixel coordinate in the contour pixel coordinate group and the contour center point coordinate to obtain a pixel distance group; using the distance corresponding to the current contour pixel coordinate as a dividing point, when the distances in the pixel distance group before the dividing point are increasing and the distances after the dividing point are decreasing, the current contour pixel coordinate is selected as a candidate contour vertex coordinate; and determining the contour vertex coordinates from the candidate contour vertex coordinates based on the vertex right angle feature. The paper image to be processed is standardized based on the coordinates of the contour vertices to obtain a standardized paper image.

2. The method according to claim 1, characterized in that, The step of determining the contour vertex coordinates from the candidate contour vertex coordinates based on vertex right-angle features includes: Traverse the coordinates of each candidate contour vertex, and based on the position of the current candidate contour vertex coordinate in the contour pixel coordinate list, obtain the coordinates of the second preset number of contour pixels before the current candidate contour vertex coordinate and perform line fitting to obtain the first line function parameters; and obtain the coordinates of the second preset number of contour pixels after the current candidate contour vertex coordinate and perform line fitting to obtain the second line function parameters. The angle between the lines used to calculate the coordinates of the current candidate contour vertex is obtained by using the first line function parameters and the second line function parameters to obtain a list of the angles between the lines corresponding to the coordinates of each candidate contour vertex. Based on the distance between each straight angle and a right angle in the list of straight angles, the coordinates of the four candidate contour vertices with the smallest distance are selected as the contour vertex coordinates.

3. The method according to claim 1, characterized in that, The step of standardizing the paper image to be processed based on the contour vertex coordinates to obtain a standardized paper image includes: Based on the industry standard dimensions of the paper contained in the paper image to be processed, determine the standard vertex coordinates of the paper image to be processed; Calculate the transformation matrix based on the standard vertex coordinates and the contour vertex coordinates; The transformation matrix is ​​used to standardize the paper image to be processed, resulting in a standardized paper image.

4. The method according to claim 1, characterized in that, The extraction of the paper content region from the paper image to be processed includes: The process involves acquiring an image of the paper to be processed and calling a semantic segmentation network trained using knowledge distillation. The paper image to be processed is input into the semantic segmentation network, which segments the paper image to obtain the paper content region.

5. The method according to claim 1, characterized in that, The step of extracting the pixel coordinates of the outline of the paper content area to obtain a list of outline pixel coordinates includes: After magnifying the paper content area, edge detection is performed to obtain the pixel coordinates of the paper content area outline; The coordinates of each pixel point are arranged in clockwise order according to the contour to obtain a list of contour pixel point coordinates.

6. A paper image standardization device, characterized in that, include: The extraction module is used to extract the paper content area of ​​the paper image to be processed and extract the pixel coordinates of the outline of the paper content area to obtain a list of outline pixel coordinates. The vertex filtering module is used to filter contour vertex coordinates from the contour pixel coordinate list based on the maximum vertex distance feature and the vertex right angle feature. Specifically, it is used to filter contour pixel coordinates from the contour pixel coordinate list as candidate contour vertex coordinates based on the maximum vertex distance feature, including: determining the contour center point coordinates according to each contour pixel coordinate in the contour pixel coordinate list; treating the contour pixel coordinate list as a loop, traversing each contour pixel coordinate in the contour pixel coordinate list, and selecting a first preset number of contour pixel coordinates before the current contour pixel coordinate and the coordinates of the current contour pixel coordinate. The coordinates of the first preset number of contour pixels following the current contour pixel coordinates form a contour pixel coordinate group; the distance between each contour pixel coordinate in the contour pixel coordinate group and the coordinates of the contour center point are calculated to obtain a pixel distance group; using the distance corresponding to the current contour pixel coordinate as a dividing point, when the distances in the pixel distance group before the dividing point are increasing and the distances after the dividing point are decreasing, the current contour pixel coordinates are used as candidate contour vertex coordinates; the vertex filtering module is also used to determine the contour vertex coordinates from the candidate contour vertex coordinates based on the vertex right angle feature. The standardization processing module is used to standardize the paper image to be processed based on the coordinates of the contour vertices to obtain a standardized paper image.

7. A computer device comprising a processor and a memory, wherein the memory stores a computer program, characterized in that, The processor is used to implement the paper image standardization method according to any one of claims 1-5 when executing the computer program.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the paper image standardization method according to any one of claims 1-5.

Images