Document posture determination method, model training method, device, server and medium

Abstract

The embodiments of this specification disclose a document posture determination method, a model training method, a device, a server and a medium. In the document posture determination method, by performing corner point detection on the document image, the corner point position of the document area in the document image is obtained, and based on the corner point position, the multi-dimensional offset parameters of the target document offset in various directions of the spatial coordinate system are determined, thereby determining the spatial posture of the target document. The above scheme provides an effective method for determining the offset of the document posture, and is simple to implement and easy to operate.

Description

Technical Field

[0001] The embodiments of this specification relate to the field of computer technology, and in particular to a document posture determination method, model training method, device, server and medium. Background Art

[0002] With the continuous development of science and technology, electronic technology has also developed rapidly, and the functions that electronic devices can achieve are becoming more and more abundant. In the prior art, document detection is applied to many scenarios, such as verifying the identity of a user by detecting the user's document information, and automatically filling in the user's bank card information by detecting the bank card. When performing document detection, the user is usually required to use an electronic device to scan or take a photo of the document, and then perform document detection on the obtained image. Since the document in the image is usually tilted, the subsequent document detection is inaccurate. Therefore, how to accurately determine the document posture in the document image is crucial for document detection. Summary of the invention

[0003] The embodiments of this specification provide a method for determining a certificate posture, a model training method, a device, a server and a medium.

[0004] In a first aspect, an embodiment of the present specification provides a method for determining a document posture, comprising:

[0005] Performing corner point detection on the document image to determine the corner point position of each corner point of the document area in the document image, wherein the document area is the area where the target document is located;

[0006] Determine a multi-dimensional offset parameter according to the corner point position of each corner point of the document area, wherein the multi-dimensional offset parameter is used to characterize the degree of offset of the target document along the horizontal axis direction, the longitudinal axis direction and the vertical axis direction of the spatial coordinate system;

[0007] Based on the multi-dimensional offset parameters, the spatial posture of the target document is determined.

[0008] In a second aspect, the embodiment of this specification provides a document posture recognition model training method, including:

[0009] Acquire multiple document images;

[0010] According to the document posture determination method provided in the first aspect, determine the document spatial posture corresponding to each document image in the multiple document images;

[0011] Based on the document space posture corresponding to each document image, labeling the document posture of each document image;

[0012] Based on each document image with document posture annotated, the initial document posture recognition model is trained to obtain a trained document posture recognition model, and the trained document posture recognition model is used to perform document posture recognition on the input document image and output the corresponding document spatial posture.

[0013] In a third aspect, an embodiment of the present specification provides a device for determining a document posture, the device comprising:

[0014] A corner point determination module, used to perform corner point detection on the document image, and determine the corner point position of each corner point of the document area in the document image, wherein the document area is the area where the target document is located;

[0015] An offset parameter determination module, used to determine a multi-dimensional offset parameter according to the corner point position of each corner point of the document area, wherein the multi-dimensional offset parameter is used to characterize the degree of offset of the target document along the horizontal axis direction, the longitudinal axis direction and the vertical axis direction of the spatial coordinate system;

[0016] The posture determination module is used to determine the spatial posture of the target document based on the multi-dimensional offset parameters.

[0017] In a fourth aspect, the embodiments of this specification provide a document posture recognition model training method, including:

[0018] An acquisition module, used for acquiring multiple document images;

[0019] An angle determination module, used to determine the angle corresponding to each of the multiple document images according to the document posture determination method provided in the first aspect.

[0020] spatial posture;

[0021] An angle annotation module, used for annotating the document posture of each document image based on the spatial posture corresponding to each document image;

[0022] The model training module is used to train the initial document posture recognition model based on each document image after the document posture is annotated to obtain a trained document posture recognition model. The trained document posture recognition model is used to perform document posture recognition on the input document image and output the corresponding spatial posture.

[0023] In a fifth aspect, an embodiment of the present specification provides a server, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the steps of any of the above methods.

[0024] In a sixth aspect, an embodiment of the present specification provides a computer-readable storage medium having a computer program stored thereon, which implements the steps of any of the above-mentioned methods when executed by a processor.

[0025] The beneficial effects of the embodiments of this specification are as follows:

[0026] In the document posture determination method provided in the embodiment of this specification, the corner point position of each corner point of the document area contained in the document image is obtained, and based on the corner point position of each corner point, a multi-dimensional offset parameter used to characterize the offset degree of the target document along the horizontal axis direction, the longitudinal axis direction and the vertical direction is determined, and then the spatial posture of the target document is determined based on the multi-dimensional offset parameter. It can be seen that the embodiment of this specification can obtain the spatial posture of the target document in the three-dimensional space by processing the two-dimensional image, and provides an effective method for determining the document posture. In addition, since the corner point position of the target document in the document image is easy to obtain, it is easy to operate and easy to implement to determine the document posture by the corner point position. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Various other advantages and benefits will become apparent to those of ordinary skill in the art by reading the detailed description of the preferred embodiments below. The accompanying drawings are only for the purpose of illustrating the preferred embodiments and are not to be considered as limiting the present specification. Also, the same reference symbols are used throughout the accompanying drawings to represent the same components. In the accompanying drawings:

[0028] Figure 1 A flowchart of a method for determining a document posture provided in the first aspect of the embodiment of this specification;

[0029] Figure 2 This is a schematic diagram of corner points of a document area in a document image shown in the embodiment of this specification;

[0030] Figure 3 A schematic diagram of the spatial coordinate system of the target certificate shown in the embodiment of this specification;

[0031] Figure 4 A flowchart of a method for training a document posture recognition model provided in the second aspect of the embodiment of this specification;

[0032] Figure 5 A schematic diagram of a device for determining a certificate posture according to a third aspect of an embodiment of this specification;

[0033] Figure 6 A schematic diagram of a document posture recognition model training device provided in the fourth aspect of the embodiments of this specification;

[0034] Figure 7 A schematic diagram of a server provided in accordance with the fifth aspect of the embodiments of this specification. DETAILED DESCRIPTION

[0035] In order to better understand the above technical scheme, the technical scheme of the embodiments of this specification is described in detail below through the accompanying drawings and specific embodiments. It should be understood that the embodiments of this specification and the specific features in the embodiments are detailed descriptions of the technical scheme of the embodiments of this specification, rather than limitations on the technical scheme of this specification. In the absence of conflict, the embodiments of this specification and the technical features in the embodiments can be combined with each other.

[0036] In a first aspect, the embodiments of this specification provide a method for determining a document posture, such as Figure 1 FIG. 1 is a flowchart of a method for determining a certificate posture provided in an embodiment of the present specification, and the method comprises the following steps:

[0037] Step S11: performing corner point detection on the document image to determine the corner point position of each corner point of the document area in the document image, wherein the document area is the area where the target document is located;

[0038] Step S12: determining a multidimensional offset parameter according to the corner point position of each corner point of the document area, wherein the multidimensional offset parameter is used to characterize the degree of offset of the target document along the horizontal axis direction, the longitudinal axis direction and the vertical axis direction of the spatial coordinate system;

[0039] Step S13: Determine the spatial posture of the target document based on the multi-dimensional offset parameters.

[0040] The method for determining the attitude of a certificate provided in the embodiments of this specification can be applied to an electronic device terminal or a server. For example, it can be applied to a mobile phone, a computer, or an ID acquisition device that can collect ID images. The ID can also be scanned or photographed by an electronic device terminal, and the obtained image can be sent to a corresponding server, so that the server processes the received image to obtain the attitude of the certificate. This is not limited here.

[0041] The target certificate may be a user's ID card, bank card, etc. The target certificate can be scanned or photographed by an electronic device to obtain a certificate image. The certificate image contains a certificate area, which is the area where the image information of the target certificate is located, that is, the certificate area does not contain a background image, and the edge of the certificate area coincides with the edge of the target certificate image information. When scanning or photographing the certificate, in order to ensure that the certificate area in the obtained certificate image is convenient for subsequent posture determination and certificate recognition, the position and size of the certificate area in the certificate image can be standardized by displaying prompt information in the scanning or photographing interface. For example, a rectangular frame can be displayed in the scanning or photographing interface, and prompt information can be displayed to prompt the user to adjust the position of the certificate so that the collected certificate is located in the rectangular frame; or, for certificates such as identity cards, a rectangular frame can be displayed in the scanning or photographing interface, and a head portrait area can be displayed in the rectangular frame to prompt the user to adjust the position of the certificate so that the collected certificate is located in the rectangular frame, and the head portrait on the identity card is located in the head portrait area. Of course, the prompt information may also include other information, such as information for prompting the front or back of the collection certificate, etc., which will not be listed here one by one.

[0042] After obtaining the document image, the corner point position of each corner point of the document area in the document image is determined through step S11. The corner point position can be determined in a variety of ways, such as Harris corner point detection, CNN (Convolutional Neural Network) corner point positioning detection, etc. Figure 2 As shown, the four corner points of the document area are A, B, C, and D. By performing corner point detection on the document image, the corner point information of the four corner points of the document area is obtained. Specifically, the corner point positions can be position information such as corner point coordinates.

[0043] When scanning or photographing the target certificate, ideally, the target certificate is placed on a horizontal plane, and the lens of the camera for scanning or photographing is parallel to the target certificate, so that the certificate area in the obtained certificate image is a rectangle, that is, the upper and lower sides opposite to each other are of the same length, and the left and lower sides opposite to each other are of the same length. However, due to the different positions where the target certificate is placed, or the lens and the target certificate are at a certain angle, the certificate area in the certificate image will be offset or tilted relative to the certificate area photographed under ideal conditions, so that the certificate area in the obtained certificate image will be deformed and not a standard rectangle. In the embodiment of this specification, the offset parameter of the target certificate offset is determined by step S12.

[0044] The offset of the target document is relative to the spatial coordinate system where the target document is located. The spatial coordinate system includes the horizontal axis direction (X axis direction), the longitudinal axis direction (Y axis direction) and the vertical direction (Z axis direction). Since the target document may be tilted or offset in various directions, in the embodiment of this specification, in order to accurately obtain the document posture of the target document, it is necessary to determine the multi-dimensional offset parameters of the target document along various directions. The multi-dimensional offset parameters can be parameters such as offset ratio, offset displacement, etc. used to characterize the offset degree of the target document relative to various directions of the spatial coordinate system.

[0045] After obtaining the multi-dimensional offset parameters, the spatial posture of the target document is further determined. The spatial posture of the target document can be the angle between the target document and the spatial coordinate system in each direction, or the offset angle along the spatial coordinate system in each direction, which is not limited here. The spatial posture of the target document can be determined according to actual needs, for example, by searching from the mapping relationship between the pre-constructed offset parameters and angles, or by determining through simulation calculations. After obtaining the spatial posture of the target document, the document image can be corrected or the document anti-counterfeiting operation can be performed according to the posture angle.

[0046] It can be seen that the document posture determination method provided in the embodiment of this specification can determine the multi-dimensional offset parameters of the target document along various directions through the corner point positions of the target document in the document image, and then determine the spatial posture of the target document, providing an effective way to determine the document posture. In addition, since the corner point positions of the document area in the document image are easy to obtain, the document posture determination method provided in the embodiment of this specification is easy to implement, and the speed of obtaining the document spatial posture is fast and time-saving.

[0047] In the embodiment of this specification, when determining the multidimensional offset parameter, the following method can be used: according to the corner point position of each corner point of the document area, the length of each side of the document area is determined; based on the length of each side of the document area, the length ratio between any two sides of the document area is determined, and based on the length ratio between any two sides, the multidimensional offset parameter is determined.

[0048] In the specific implementation process, Figure 2 As shown in FIG. 1 , after determining the positions of the four corner points A, B, C, and D of the document area, the lengths of the four sides AB, CD, AC, and BD can be calculated based on the positions of the four corner points. Taking the positions of the corner points as the corner point coordinates, the calculation method of the side lengths can be Euclidean distance or other methods, which are not limited here. It should be understood that the lengths of the sides of the document area can reflect the offset state of the target document in space. Figure 3As shown, a spatial coordinate system is established for the target document. If the target document is rotated relative to the X-axis, the lengths of the AB side and the CD side of the document area in the document image are different; if the target document is rotated relative to the Y-axis, the lengths of the AC side and the BD side of the document area in the document image are different; if the target document is rotated relative to the Z-axis, the ratio between the AC side and the BD side of the document area in the document image will also be different from the actual ratio of the two sides of the target document.

[0049] Therefore, in the embodiments of the present specification, when determining the multidimensional offset parameter based on the length of each edge, the following method can be adopted: the document area includes a first edge and a second edge that are opposite to each other, and a third edge and a fourth edge that are opposite to each other; based on a first ratio between a first length of the first edge and a second length of the second edge, a first offset parameter of the target document along the horizontal axis is determined; based on a second ratio between a third length of the third edge and a fourth length of the fourth edge, a second offset parameter of the target document along the longitudinal axis is determined; based on a third ratio between the first length of the first edge and the third length of the third edge, a third offset parameter of the target document along the vertical axis is determined.

[0050] In a specific implementation process, the first side and the second side may correspond to Figure 2 The AB side and the CD side, the third side and the fourth side can correspond to Figure 2 The AC side and the BD side in the equation, or the first side and the second side correspond to Figure 2 The AC side and the BD side, the third side and the fourth side can correspond to Figure 2 The AB side and the CD side in the example are not limited here. For the sake of convenience, it is assumed that the first side corresponds to the AB side, the second side corresponds to the CD side, the third side corresponds to the AC side, and the fourth side corresponds to the BD side. The first length of the first side is L AB , the second length of the second side is L CD , the third length of the third side is L AC , the fourth length of the fourth side is L BD Then, the first ratio is L AB / L CD , the second ratio is L AC / L BD , the third ratio is L AB / L AC .

[0051] In the embodiments of this specification, the calculated first ratio can be directly used as the first offset parameter, the second ratio can be used as the second offset parameter, and the third ratio can be used as the third offset parameter. It is also possible to process each ratio to obtain the corresponding offset parameter. In the specific implementation process, each ratio can be processed in the following manner: in the preset correspondence between the certificate and the side length ratio, the first pair of side length ratio, the second pair of side length ratio and the adjacent two side length ratio corresponding to the target certificate are determined; based on the first pair of side length ratio, the first ratio is normalized to determine the first offset parameter; based on the second pair of side length ratio, the second ratio is normalized to determine the second offset parameter; based on the second pair of side length ratio, the third ratio is normalized to determine the third offset parameter.

[0052] It should be noted that the preset correspondence between the certificate and the side length ratio refers to the correspondence between the certificate and the actual side length ratio of the two sides of the certificate. For example, the preset correspondence between the certificate and the side length ratio can be the correspondence between the certificate name, the certificate type, the side length ratio of the first pair of sides of the certificate, the side length ratio of the second pair of sides of the certificate, and the side length ratio of the adjacent two sides of the certificate. Based on the certificate name or certificate type, the corresponding side length ratios can be determined.

[0053] Generally speaking, the shape of the certificate is a rectangle. Therefore, in the two groups of opposite sides contained in the certificate, the lengths of the two opposite sides contained in each group of opposite sides are equal, that is, the length ratio of the first pair of sides and the length ratio of the second pair of sides of the rectangular certificate are both 1. The length ratio of the adjacent sides of the certificate may vary depending on the size of each type of certificate. For example, the length ratio of the adjacent sides of the identity card is 1.58. Of course, if the shape of the certificate is not a rectangle, the length ratio of the first pair of sides, the length ratio of the second pair of sides, and the length ratio of the adjacent sides of the certificate can be obtained in turn, and the above length ratios can be associated with the certificate. Furthermore, if the shape of the certificate is other polygons, the length ratio of each adjacent side can be determined as needed and associated with the certificate, which is not limited here.

[0054] The preset correspondence between the certificate and the side length ratio can be saved in an electronic device or server. In this way, when the target certificate is scanned or photographed, the first pair of side length ratios, the second pair of side length ratios, and the ratio of the lengths of two adjacent sides of the target certificate can be determined in the above correspondence according to the name or type of the target certificate. Furthermore, the first ratio, the second ratio, and the third ratio are normalized according to the first pair of side length ratios, the second pair of side length ratios, and the ratio of the lengths of two adjacent sides of the target certificate, and the normalized result is used as the corresponding offset parameter. Taking the first pair of side length ratios, the second pair of side length ratios, and the ratio of the lengths of two adjacent sides of the target certificate as 1, 1, and 1.58 as an example, the first offset parameter obtained is L. AB / L CD -1, the second offset parameter is L AC / L BD -1, the third offset is L AB / L AC -1. In the embodiments of the present specification, the ratio of the length of the first pair of sides, the ratio of the length of the second pair of sides, and the ratio of the length of two adjacent sides of the target document can reflect the state of the target document when there is no offset, that is, there is no offset angle between the target document and the X-axis, Y-axis, and Z-axis. Therefore, the offset parameter obtained after normalization can characterize the offset of the target document in the three axes relative to the no-offset state. It should be understood that for documents with equal opposite sides, the value range of the offset parameter obtained after normalization is [-1,1]. If it exceeds this range, the obtained document image is considered unusable, and the user can be prompted to rescan or take a photo.

[0055] Furthermore, after obtaining the offset parameters, the spatial posture of the target document is determined based on the offset parameters. In the embodiment of the present specification, the following method can be used to determine the spatial posture of the target document: based on a preset mapping relationship between the offset parameters and the posture angle, the posture angle corresponding to the target document is determined, and the posture angle includes the pitch angle, yaw angle and roll angle of the target document.

[0056] In the specific implementation process, the preset mapping relationship between the offset parameter and the attitude angle can be set according to actual needs. For example, the mapping relationship in the embodiment of this specification may include a mapping relationship between the first offset parameter and the pitch angle, a mapping relationship between the second offset parameter and the yaw angle, and a mapping relationship between the third offset parameter and the roll angle. Among them, the pitch angle (pitch) is the angle formed by rotating around the X-axis, the yaw angle (yaw) is the angle formed by rotating around the Y-axis, and the roll angle (roll) is the angle formed by rotating around the Z-axis.

[0057] The preset mapping relationship between the offset parameter and the posture angle can be obtained through statistics. For example, the offset angle of the current posture of the target document relative to various directions is known, and then the lengths of each side of the document area in the document image corresponding to the current posture are measured, and the corresponding offset parameter is determined according to the ratio between the two sides. The offset parameter is associated with the posture angle (i.e., the offset angle). In this way, by traversing the offset angles, the preset mapping relationship between the offset parameter and the posture angle is obtained.

[0058] In addition, the preset mapping relationship between the offset parameter and the posture angle is a mapping relationship constructed by performing a three-dimensional affine transformation on the reference image of the target document, wherein the reference image is an image of the target document without offset relative to the spatial coordinate system. In the embodiment of this specification, the reference image can be a document image obtained under an ideal state, that is, the document area in the reference image has no offset relative to all directions. The three-dimensional affine transformation includes spatial translation, rotation, scaling and other transformations of the document image. In the specific implementation process, the reference image can be subjected to an affine transformation to obtain an image after changing any angle, and the angle of transformation along each direction of the spatial coordinate system is recorded, and the side length ratio between the adjacent two sides and the opposite sides of the document area in the transformed image is obtained, and the corresponding offset parameter is calculated, and the offset parameter is associated with the recorded angle to obtain a preset mapping relationship between the offset parameter and the posture angle.

[0059] Of course, in addition to the above methods, other methods can also be used to determine the spatial posture of the target document. For example, after obtaining the corner point position of each corner point of the document area, an affine transformation can be performed according to the corner point position of each corner point to map the document area in the document image to a standard image. The size of the standard image is fixed, and the standard image is an image without offset in all directions relative to the spatial coordinate system. The standard image is then rotated based on the rotation matrix to obtain the same posture as the target document, and the rotation angle is recorded. The rotation angle can be used as the document posture of the target document.

[0060] Furthermore, the solution in the embodiments of this specification can correct the document area in the document image according to the posture angle after obtaining the posture angle, so as to facilitate the subsequent document recognition or document anti-counterfeiting detection. Taking document anti-counterfeiting detection as an example, some documents have anti-counterfeiting marks, and the anti-counterfeiting marks will show different colors at different angles, for example, the first color is shown at the first angle and the second color is shown at the second angle. Therefore, the document anti-counterfeiting can be performed according to the corresponding relationship between the document posture angle and the color of the anti-counterfeiting mark.

[0061] In addition, in the embodiments of this specification, the document image can also be marked by the obtained posture angle to train the document posture recognition model. That is, after determining the posture angle of the target document, the method further includes: annotating the target image with the angle based on the posture angle; constructing a training sample set based on the target image after the angle annotation, and training the initial document posture recognition model based on the training sample set to obtain a trained document posture recognition model, wherein the trained document posture recognition model is used to perform document posture recognition on the input image and output the corresponding document posture angle.

[0062] In the specific implementation process, the posture angle of the target certificate corresponding to the certificate image is obtained by the above method, and the certificate image is annotated with the posture angle, that is, the marking operation is performed, and the certificate image after the angle annotation is used as a training sample. It should be understood that a large number of certificate images can be annotated with angles by this method to obtain a certain number of training sample sets to train the constructed initial certificate posture recognition model. The initial certificate posture recognition model can be selected according to actual needs, for example, using a CNN model, an RNN (Recurrent Neural Network) model, etc. The input of the initial certificate posture recognition model can be the certificate image to be posture recognized, and the output is the posture angle of the certificate corresponding to the input image. After obtaining the trained certificate posture angle model, the trained certificate posture recognition model can be applied to various scenarios, such as certificate posture detection, certificate posture correction, etc.

[0063] In the second aspect, based on the same inventive concept, the embodiment of this specification provides a document posture recognition model training method, please refer to Figure 4 , the method comprising:

[0064] Step S41: Acquire multiple document images;

[0065] Step S42: determining the document spatial posture corresponding to each document image in the plurality of document images according to the document posture determination method provided in the first aspect;

[0066] Step S43: Based on the spatial posture corresponding to each of the document images, labeling the document posture of each of the document images;

[0067] Step S44: Based on each ID image with ID gesture annotated, the initial ID gesture recognition model is trained to obtain a trained ID gesture recognition model, wherein the trained ID gesture recognition model is used to perform ID gesture recognition on the input ID image and output the corresponding spatial gesture.

[0068] In the embodiments of this specification, the multiple ID images may be images obtained by scanning or photographing various IDs, or images from the Internet or various databases. Through the ID posture determination method provided in the first aspect of the embodiments of this specification, each ID image is processed to obtain the ID space posture of each ID image, and the ID space posture is annotated on the corresponding ID image to form a model training sample. Taking the ID space posture as an example of the ID posture angle, the pitch angle, yaw angle, and roll angle of the ID offset along the spatial coordinate system can be annotated in the corresponding ID image.

[0069] The initial document posture recognition model can be selected according to actual needs, such as using a CNN model, an RNN model, etc. The number of layers of the network model and the number of neurons in each layer can be set according to actual needs, and are not limited here. For the initial document posture recognition model, the input is the document image, and the output is the spatial posture corresponding to the document image, such as pitch angle, yaw angle, and roll angle. During the training process, the model parameters can be continuously adjusted according to the spatial posture annotated in the document image until the accuracy of the model output reaches a preset value, or the number of model training times reaches a preset number, and a trained document posture angle model is obtained. The trained document posture recognition model is applied to various scenarios, such as document posture detection, document posture correction, document anti-counterfeiting, and other scenarios.

[0070] Regarding the above method, the specific implementation of each step can refer to the detailed description of the document posture determination method provided in the first aspect of the embodiment of this specification, and will not be elaborated here.

[0071] In the third aspect, based on the same inventive concept, the embodiment of this specification provides a device for determining the posture of a certificate, please refer to Figure 5 , the device comprises:

[0072] A corner point determination module 51 is used to perform corner point detection on the document image, and determine the corner point position of each corner point of the document area in the document image, wherein the document area is the area where the target document is located;

[0073] An offset parameter determination module 52 is used to determine a multi-dimensional offset parameter according to the corner point position of each corner point of the document area, wherein the multi-dimensional offset parameter is used to characterize the offset degree of the target document along the horizontal axis direction, the longitudinal axis direction and the vertical axis direction of the spatial coordinate system;

[0074] The posture determination module 53 is used to determine the spatial posture of the target document based on the multi-dimensional offset parameters.

[0075] In an optional implementation, the offset parameter determination module 52 is configured to:

[0076] Determining the length of each side of the document area according to the corner point position of each corner point of the document area;

[0077] Based on the length of each side of the document area, the length ratio between any two sides of the document area is determined, and based on the length ratio between any two sides, the multi-dimensional offset parameter is determined.

[0078] In an optional implementation, the document area includes a first side and a second side that are opposite to each other, and a third side and a fourth side that are opposite to each other, and the offset parameter determination module 52 is used to:

[0079] Determining a first offset parameter of the target document along the horizontal axis based on a first ratio between a first length of the first side and a second length of the second side;

[0080] Determining a second offset parameter of the target document along the longitudinal axis based on a second ratio between a third length of the third side and a fourth length of the fourth side;

[0081] Based on a third ratio between the first length of the first side and the third length of the third side, a third offset parameter of the target document along the vertical axis direction is determined.

[0082] In an optional implementation, the offset parameter determination module 52 is configured to:

[0083] In the preset correspondence between the certificate and the side length ratio, determine the side length ratio of the first pair of sides, the side length ratio of the second pair of sides, and the side length ratio of two adjacent sides corresponding to the target certificate;

[0084] Normalizing the first ratio based on the first pair of side length ratios to determine the first offset parameter;

[0085] Normalizing the second ratio based on the ratio of the lengths of the second pair of sides to determine the second offset parameter;

[0086] The third ratio is normalized based on the ratio of the lengths of the two adjacent sides to determine the third offset parameter.

[0087] In an optional implementation, the posture determination module 53 is used to:

[0088] Based on a preset mapping relationship between the offset parameter and the attitude angle, the attitude angle corresponding to the target document is determined, and the attitude angle includes the pitch angle, yaw angle and roll angle of the target document.

[0089] In an optional implementation, the preset mapping relationship between the offset parameter and the posture angle is a mapping relationship constructed by performing a three-dimensional affine transformation on a reference image of the target document, wherein the reference image is an image of the target document without offset relative to the spatial coordinate system.

[0090] In an optional implementation, the device further includes:

[0091] An angle marking module, used for marking the angle of the document image based on the posture angle;

[0092] The model training module is used to construct a training sample set based on the ID image after angle annotation, and train a preset ID posture recognition model based on the training sample set to obtain a trained ID posture recognition model, wherein the trained ID posture recognition model is used to perform ID posture recognition on the input image and output the corresponding ID posture angle.

[0093] Regarding the above-mentioned device, the specific functions of each module therein have been described in detail in the embodiment of the document posture determination method provided in the embodiment of this specification, and will not be elaborated here.

[0094] Fourthly, based on the same inventive concept, the embodiment of this specification provides a document posture recognition model training device, please refer to Figure 6 , the device comprises:

[0095] An acquisition module 61 is used to acquire multiple document images;

[0096] An angle determination module 62, configured to determine the document spatial posture corresponding to each of the plurality of document images according to the document posture determination method provided in the first aspect;

[0097] An angle marking module 63, used for marking the document posture of each document image based on the document spatial posture corresponding to each document image;

[0098] The model training module 64 is used to train the initial document posture recognition model based on each document posture annotated document posture to obtain a trained document posture recognition model. The trained document posture recognition model is used to perform document posture recognition on the input document image and output the corresponding document spatial posture.

[0099] Regarding the above-mentioned device, the specific functions of each module therein have been described in detail in the embodiments of the document posture determination method and the document posture recognition model training method provided in the embodiments of this specification, and will not be elaborated here.

[0100] In a fifth aspect, based on the same inventive concept as the document posture determination method and the document posture recognition model training method in the aforementioned embodiment, the embodiment of this specification further provides a server, such as Figure 7 As shown, it includes a memory 404, a processor 402, and a computer program stored in the memory 404 and executable on the processor 402. When the processor 402 executes the program, the steps of any one of the methods for determining the document posture and training the document posture recognition model described above are implemented.

[0101] Among them, Figure 7 In the embodiment of the present invention, a bus architecture (represented by bus 400) is shown, which may include any number of interconnected buses and bridges, and bus 400 links various circuits including one or more processors represented by processor 402 and memory represented by memory 404. Bus 400 may also link various other circuits such as peripherals, voltage regulators, and power management circuits, which are well known in the art and are therefore not further described herein. Bus interface 406 provides an interface between bus 400 and receiver 401 and transmitter 403. Receiver 401 and transmitter 403 may be the same element, namely a transceiver, providing a unit for communicating with various other devices over a transmission medium. Processor 402 is responsible for managing bus 400 and general processing, while memory 404 may be used to store data used by processor 402 when performing operations.

[0102] In the sixth aspect, based on the inventive concept of the document posture determination method and the document posture recognition model training method in the aforementioned embodiments, the embodiments of this specification also provide a computer-readable storage medium on which a computer program is stored. When the program is executed by the processor, the steps of any one of the methods described above in the document posture determination method and the document posture recognition model training method are implemented.

[0103] This specification is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of this specification. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that has the functions specified in one or more boxes.

[0104] These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0105] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0106] Although the preferred embodiments of this specification have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of this specification.

[0107] Obviously, those skilled in the art can make various changes and modifications to this specification without departing from the spirit and scope of this specification. Thus, if these modifications and variations of this specification fall within the scope of the claims of this specification and their equivalents, this specification is also intended to include these modifications and variations.

Claims

1. A method for determining a document posture, the method comprising: Performing corner point detection on the document image to determine the corner point position of each corner point of the document area in the document image, wherein the document area is the area where the target document is located, and the document area includes a first side and a second side that are opposite to each other, and a third side and a fourth side that are opposite to each other; Determining a multidimensional offset parameter according to the corner point position of each corner point of the document area, wherein the multidimensional offset parameter is used to characterize the degree of offset of the target document along the horizontal axis direction, the longitudinal axis direction and the vertical axis direction of the spatial coordinate system, including: determining the length of each side of the document area according to the corner point position of each corner point of the document area; determining the length ratio between any two sides of the document area based on the length of each side of the document area, and determining the multidimensional offset parameter based on the length ratio between any two sides; Determining the spatial posture of the target document based on the multi-dimensional offset parameters; Wherein, determining the length ratio between any two sides of the document area based on the length of each side of the document area, and determining the multi-dimensional offset parameter based on the length ratio between any two sides, includes: Determining a first offset parameter of the target document along the horizontal axis based on a first ratio between a first length of the first side and a second length of the second side; Determining a second offset parameter of the target document along the longitudinal axis based on a second ratio between a third length of the third side and a fourth length of the fourth side; Based on a third ratio between the first length of the first side and the third length of the third side, a third offset parameter of the target document along the vertical axis direction is determined.

2. The method according to claim 1, wherein determining the multi-dimensional offset parameter based on the length ratio between any two edges comprises: In the preset correspondence between the certificate and the side length ratio, determine the side length ratio of the first pair of sides, the side length ratio of the second pair of sides, and the side length ratio of two adjacent sides corresponding to the target certificate; Normalizing the first ratio based on the first pair of side length ratios to determine the first offset parameter; Normalizing the second ratio based on the ratio of the lengths of the second pair of sides to determine the second offset parameter; The third ratio is normalized based on the ratio of the lengths of the two adjacent sides to determine the third offset parameter.

3. The method according to claim 1, wherein determining the spatial posture of the target document based on the multi-dimensional offset parameter comprises: Based on a preset mapping relationship between the offset parameter and the attitude angle, the attitude angle corresponding to the target document is determined, and the attitude angle includes the pitch angle, yaw angle and roll angle of the target document.

4. According to the method of claim 3, the preset mapping relationship between the offset parameter and the posture angle is a mapping relationship constructed by performing a three-dimensional affine transformation on the reference image of the target document, wherein: The reference image is an image of the target document without offset relative to the spatial coordinate system.

5. The method according to claim 3, after determining the spatial posture of the target document, the method further comprises: Annotating the document image with an angle based on the posture angle; Based on the ID image after angle annotation, a training sample set is constructed, and the initial ID posture recognition model is trained based on the training sample set to obtain a trained ID posture recognition model, wherein the trained ID posture recognition model is used to perform ID posture recognition on the input image and output the corresponding ID posture angle.

6. A document posture recognition model training method, the method comprising: Acquire multiple document images; The document posture determination method according to any one of claims 1 to 5, determining the document spatial posture corresponding to each document image in the plurality of document images; Based on the document space posture corresponding to each document image, labeling the document posture of each document image; Based on each document image with document posture annotated, the initial document posture recognition model is trained to obtain a trained document posture recognition model, and the trained document posture recognition model is used to perform document posture recognition on the input document image and output the corresponding document spatial posture.

7. A device for determining a document posture, the device comprising: A corner point determination module, used to perform corner point detection on the document image, and determine the corner point position of each corner point of the document area in the document image, wherein the document area is the area where the target document is located, and the document area includes a first side and a second side that are opposite to each other, and a third side and a fourth side that are opposite to each other; An offset parameter determination module is used to determine a multi-dimensional offset parameter according to the corner point position of each corner point of the document area, and the multi-dimensional offset parameter is used to characterize the offset degree of the target document along the horizontal axis direction, the longitudinal axis direction and the vertical axis direction of the spatial coordinate system, including: determining the length of each side of the document area according to the corner point position of each corner point of the document area; Determine the length ratio between any two sides of the document area based on the length of each side of the document area, and determine the multi-dimensional offset parameter based on the length ratio between any two sides; A posture determination module, used for determining the spatial posture of the target document based on the multi-dimensional offset parameter; The offset parameter determination module is also used for: Determining a first offset parameter of the target document along the horizontal axis based on a first ratio between a first length of the first side and a second length of the second side; Determining a second offset parameter of the target document along the longitudinal axis based on a second ratio between a third length of the third side and a fourth length of the fourth side; Based on a third ratio between the first length of the first side and the third length of the third side, a third offset parameter of the target document along the vertical axis direction is determined.

8. The device according to claim 7, wherein the offset parameter determination module is configured to: In the preset correspondence between the certificate and the side length ratio, determine the side length ratio of the first pair of sides, the side length ratio of the second pair of sides, and the side length ratio of two adjacent sides corresponding to the target certificate; Normalizing the first ratio based on the first pair of side length ratios to determine the first offset parameter; Normalizing the second ratio based on the ratio of the lengths of the second pair of sides to determine the second offset parameter; The third ratio is normalized based on the ratio of the lengths of the two adjacent sides to determine the third offset parameter.

9. The device according to claim 7, wherein the posture determination module is used to: Based on a preset mapping relationship between the offset parameter and the attitude angle, the attitude angle corresponding to the target document is determined, and the attitude angle includes the pitch angle, yaw angle and roll angle of the target document.

10. The device according to claim 9, wherein the preset mapping relationship between the offset parameter and the posture angle is a mapping relationship constructed by performing a three-dimensional affine transformation on the reference image of the target document, wherein: The reference image is an image of the target document without offset relative to the spatial coordinate system.

11. The device according to claim 9, further comprising: An angle marking module, used for marking the angle of the document image based on the posture angle; The model training module is used to construct a training sample set based on the ID image after angle annotation, and train a preset ID posture recognition model based on the training sample set to obtain a trained ID posture recognition model, wherein the trained ID posture recognition model is used to perform ID posture recognition on the input image and output the corresponding ID posture angle.

12. A document posture recognition model training device, the device comprising: An acquisition module, used for acquiring multiple document images; An angle determination module, configured to determine the document spatial posture corresponding to each document image in the plurality of document images according to the document posture determination method according to any one of claims 1 to 5; An angle annotation module, used for annotating the document posture of each document image based on the document spatial posture corresponding to each document image; The model training module is used to train the initial document posture recognition model based on each document image after the document posture is annotated to obtain a trained document posture recognition model. The trained document posture recognition model is used to perform document posture recognition on the input document image and output the corresponding document spatial posture.

13. A server, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the method according to any one of claims 1 to 6 when executing the program.

14. A computer-readable storage medium having a computer program stored thereon, wherein when the program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images