Automatic signature method and device based on deep learning

Abstract

The present invention discloses an automatic signature method and device based on deep learning. The method queries the drawing information and electronic signature matching the target drawing according to the identification information of the target drawing to be signed; generates a file in a preset format according to the drawing information and electronic signature corresponding to the target drawing; after obtaining the file in the preset format, performs a preprocessing operation on the target drawing according to the drawing information corresponding to the target drawing, identifies the signature position of the drawing based on a trained signature position recognition model, realizes accurate positioning of the signature position, and reduces the occurrence of missed signature position detection; performs a restoration operation on the coordinates of the signature position according to the preprocessed target drawing and the drawing information corresponding to the target drawing to obtain the restored signature position, and inserts the electronic signature into the restored signature position, thereby improving the accuracy, standardization, convenience and consistency of the drawing signature, thereby bringing a better signature experience to the user.

Description

Technical Field

[0001] The present invention relates to the field of image recognition technology, and in particular to an automatic signature method and device based on deep learning. Background Art

[0002] Since the signature on a drawing has obvious individual characteristics and identifiability, and can ensure the authenticity and validity of the drawing, it is used in many fields. Therefore, in order to improve the accuracy of the signature on the drawing, it is necessary to accurately identify the signature position first.

[0003] At present, the method of identifying the signature position is generally to query the drawing database according to the signature position to obtain the cell corresponding to the signature position. However, it is found in practice that each query needs to detect whether there is a title bar or frame block from the drawing database. Due to the large number of drawings and the large number of drawings, the detection efficiency is low and it is easy to miss the detection, which affects the signature effect. It can be seen that querying the signature position through the database cannot achieve fast and accurate search of the signature position, which leads to the inability to achieve accurate signature. Therefore, it is necessary to propose a technical solution to improve the recognition accuracy of the signature position to improve the accuracy of the drawing signature. Summary of the invention

[0004] The present invention provides an automatic signature method and device based on deep learning, which can improve the recognition accuracy of the signature position to improve the accuracy of the drawing signature.

[0005] In order to solve the above technical problems, the first aspect of the present invention discloses an automatic signature method based on deep learning, the method comprising:

[0006] According to the identification information of the target drawing to be signed, in the drawing database, the drawing information and electronic signature matching the target drawing are searched;

[0007] According to a preset format, the drawing information and the electronic signature corresponding to the target drawing are converted into a file in the preset format, wherein the file includes the drawing information and the electronic signature corresponding to the target drawing;

[0008] After obtaining the file in the preset format, performing a preprocessing operation on the target drawing according to the drawing information corresponding to the target drawing to obtain the preprocessed target drawing;

[0009] Identify the signature position of the target drawing based on a pre-trained signature position recognition model;

[0010] According to the preprocessed target drawing and the drawing information corresponding to the target drawing, the coordinates of the signature position are restored to obtain the restored signature position, and the electronic signature is inserted into the restored signature position.

[0011] As an optional implementation, in the first aspect of the present invention, the identifying the signature position of the target drawing based on the pre-trained signature position recognition model includes:

[0012] Input the target drawing into a pre-trained first feature extraction model to perform feature extraction, thereby obtaining a high-level feature map of the target drawing;

[0013] Inputting the high-level feature map of the target drawing into a pre-trained candidate region analysis model for analysis to obtain a plurality of candidate signature regions of the target drawing;

[0014] For any of the candidate signature regions, extract local features of the candidate signature region through a pre-trained second feature extraction model, and input the local features of the candidate signature region into a pre-trained parameter analysis model for analysis to obtain the probability of the category to which the candidate signature region belongs and a predicted bounding box;

[0015] Determining the confidence of the predicted bounding box of each candidate signature region according to the predicted bounding box of each candidate signature region;

[0016] Comparing the probability of the category to which each of the candidate signature regions belongs and the confidence of the predicted bounding box with the corresponding preset probability and the first preset confidence to obtain a comparison result;

[0017] According to the comparison result, all the candidate signature areas whose probability of the category to which they belong is greater than or equal to the preset probability or whose confidence of the predicted bounding box is greater than or equal to the first preset confidence are determined as the signature positions of the target drawing.

[0018] As an optional implementation, in the first aspect of the present invention, the inputting the local features of the candidate signature region into a pre-trained parameter analysis model for analysis to obtain the probability and predicted bounding box of the category to which the candidate signature region belongs includes:

[0019] Inputting the local features of the candidate signature area into a pre-trained first parameter analysis model for analysis to obtain the probability of the category to which the candidate signature area belongs;

[0020] The local features of the candidate signature area are input into a pre-trained second parameter analysis model for analysis to obtain a predicted bounding box of the candidate signature area.

[0021] As an optional implementation, in the first aspect of the present invention, before comparing the probability of the category to which each of the candidate signature regions belongs and the confidence of the predicted bounding box with the corresponding preset probability and the first preset confidence, and obtaining the comparison result, the method further includes:

[0022] Determining the overlap of the predicted bounding boxes of all the candidate signature regions according to the predicted bounding box of each candidate signature region;

[0023] According to the confidence and overlap of the predicted bounding boxes of all the candidate signature areas, it is determined whether there is a candidate signature area whose confidence is less than or equal to the second preset confidence or whose overlap is greater than or equal to the preset overlap among all the candidate signature areas, and when it is determined that there is no candidate signature area, triggering the operation of comparing the probability of the category to which each candidate signature area belongs and the confidence of the predicted bounding box among all the candidate signature areas with the corresponding preset probability and the first preset confidence to obtain a comparison result;

[0024] When it is determined that it exists, based on the confidence and overlap of the predicted bounding boxes of all the candidate signature areas, the candidate signature areas whose confidence is less than or equal to the second preset confidence or whose overlap is greater than or equal to the preset overlap are deleted from all the candidate signature areas to obtain all the remaining target candidate signature areas, and trigger the execution of the operation of comparing the probability of the category to which each of the candidate signature areas belongs, the confidence of the predicted bounding box with the corresponding preset probability and the first preset confidence to obtain the comparison result, and all the candidate signature areas are all the remaining target candidate signature areas.

[0025] As an optional embodiment, in the first aspect of the present invention, the method further comprises:

[0026] Obtain a drawing training set and a drawing verification set, wherein the drawing training set consists of a plurality of training drawings and signature positions corresponding to the training drawings, and the drawing verification set consists of a plurality of verification drawings and signature positions corresponding to the verification drawings;

[0027] Training the drawing training set based on a predetermined target recognition model;

[0028] In the process of training the target recognition model, based on a predetermined loss function, the drawing verification set is input into the target recognition model to calculate the loss result of the target recognition model;

[0029] According to the loss result of the target recognition model, the parameters of the target recognition model are optimized to obtain the optimized target recognition model, and the optimized target recognition model is determined as the pre-trained signature position recognition model.

[0030] As an optional implementation, in the first aspect of the present invention, the calculation formula of the loss function is as follows:

[0031] ;

[0032] ;

[0033] ;

[0034] ;

[0035] ;

[0036] ;

[0037] ;

[0038] ;

[0039] In the formula, represents the loss result of the target recognition model, represents the loss deviation term between the actual bounding box of the signature position of the verification drawing in the drawing verification set and the corresponding predicted bounding box, represents the actual bounding box of the signature position of the verified drawing in the drawing verification set, respectively represent the abscissa and ordinate of the center point of the actual bounding box of the signature position of the verification drawing in the drawing verification set, represent the width and height of the actual bounding box of the signature position of the verification drawing in the drawing verification set, respectively, represents the predicted bounding box of the signature position obtained after the verification drawing in the drawing verification set is input into the target recognition model, The horizontal coordinate and the vertical coordinate of the center point of the predicted bounding box of the signature position obtained after the verification drawing in the drawing verification set is input into the target recognition model respectively; are respectively the width and height of the predicted bounding box of the signature position obtained after the verification drawing in the drawing verification set is input into the target recognition model, is the regularization coefficient, The parameters that need to be satisfied by the actual bounding box of the signature position of the drawing to be verified in the drawing verification set; represents the gradient between the width of the predicted bounding box of the signature position obtained after the verification drawing in the drawing verification set is input into the target recognition model and the width of the corresponding actual bounding box; represents the gradient between the height of the predicted bounding box of the signature position obtained after the verification drawing in the drawing verification set is input into the target recognition model and the height of the corresponding actual bounding box; , are the width and height after gradient processing respectively; Indicates taking and The minimum value of Indicates taking and The minimum value of Represents the parameters that need to be satisfied by the actual bounding box of the signature position of the verified drawing in the drawing verification set.

[0040] As an optional implementation, in the first aspect of the present invention, after obtaining the drawing training set and the drawing verification set, the method further includes:

[0041] According to the data of each drawing in the target drawing set, it is judged whether there is a drawing in the target drawing set that does not meet the recognition requirements of the target recognition model, wherein the target drawing set includes the drawing training set and the drawing verification set; when it is judged that there is no drawing, triggering the execution of the operation of training the drawing training set based on the predetermined target recognition model;

[0042] When it is determined that there is, an adjustment operation is performed on the drawings in the target drawing set that do not meet the recognition requirement according to the recognition requirement to obtain the target drawing set that meets the recognition requirement, and the operation of training the drawing training set based on the predetermined target recognition model is triggered;

[0043] Among them, the recognition requirements include one or more of the following: the drawing quality is greater than or equal to the preset drawing quality, there are no typos in the signature in the drawing, the signature in the drawing does not exceed the signature position, and the drawing pixel size is within the preset drawing pixel size range.

[0044] The second aspect of the present invention discloses an automatic signature device based on deep learning, the device comprising:

[0045] A query module, used to query the drawing information and electronic signature matching the target drawing in the drawing database according to the identification information of the target drawing to be signed;

[0046] A generating module, configured to generate a file in the preset format by converting the drawing information and the electronic signature corresponding to the target drawing into the file in the preset format, wherein the file includes the drawing information and the electronic signature corresponding to the target drawing;

[0047] A preprocessing module, configured to, after obtaining the file in the preset format, perform a preprocessing operation on the target drawing according to the drawing information corresponding to the target drawing, so as to obtain the preprocessed target drawing;

[0048] A detection module, used for identifying the signature position of the target drawing based on a pre-trained signature position recognition model;

[0049] A restoration module, used to perform a restoration operation on the coordinates of the signature position according to the preprocessed target drawing and the drawing information corresponding to the target drawing, so as to obtain the restored signature position;

[0050] The signature module is used to insert the electronic signature into the restored signature position.

[0051] As an optional implementation, in the second aspect of the present invention, the specific manner in which the detection module identifies the signature position of the target drawing based on the pre-trained signature position recognition model includes:

[0052] Input the target drawing into a pre-trained first feature extraction model to perform feature extraction, thereby obtaining a high-level feature map of the target drawing;

[0053] Inputting the high-level feature map of the target drawing into a pre-trained candidate region analysis model for analysis to obtain a plurality of candidate signature regions of the target drawing;

[0054] For any of the candidate signature regions, extract local features of the candidate signature region through a pre-trained second feature extraction model, and input the local features of the candidate signature region into a pre-trained parameter analysis model for analysis to obtain the probability of the category to which the candidate signature region belongs and a predicted bounding box;

[0055] Determining the confidence of the predicted bounding box of each candidate signature region according to the predicted bounding box of each candidate signature region;

[0056] Comparing the probability of the category to which each of the candidate signature regions belongs and the confidence of the predicted bounding box with the corresponding preset probability and the first preset confidence to obtain a comparison result;

[0057] According to the comparison result, all the candidate signature areas whose probability of the category to which they belong is greater than or equal to the preset probability or whose confidence of the predicted bounding box is greater than or equal to the first preset confidence are determined as the signature positions of the target drawing.

[0058] As an optional implementation, in the second aspect of the present invention, the detection module inputs the local features of the candidate signature area into a pre-trained parameter analysis model for analysis, and the specific method of obtaining the probability of the category to which the candidate signature area belongs and the predicted bounding box includes:

[0059] Inputting the local features of the candidate signature area into a pre-trained first parameter analysis model for analysis to obtain the probability of the category to which the candidate signature area belongs;

[0060] The local features of the candidate signature area are input into a pre-trained second parameter analysis model for analysis to obtain a predicted bounding box of the candidate signature area.

[0061] As an optional implementation, in the second aspect of the present invention, the device further includes:

[0062] A first determination module is used to compare the probability of the category to which each of the candidate signature regions belongs and the confidence of the predicted bounding box with the corresponding preset probability and the first preset confidence in the detection module, and determine the overlap of the predicted bounding boxes of all the candidate signature regions according to the predicted bounding box of each candidate signature region before obtaining the comparison result;

[0063] a first judgment module, configured to judge, based on the confidences and overlaps of the predicted bounding boxes of all the candidate signature areas, whether there is a candidate signature area whose confidence is less than or equal to a second preset confidence or whose overlap is greater than or equal to a preset overlap among all the candidate signature areas, and, when it is judged that there is no candidate signature area, trigger the detection module to perform the operation of comparing the probability of the category to which each candidate signature area belongs, the confidence of the predicted bounding box, the corresponding preset probability and the first preset confidence, and obtaining a comparison result;

[0064] The first determination module is also used to, when it is determined that there is, delete, from all the candidate signature areas, the candidate signature areas whose confidence is less than or equal to the second preset confidence or whose overlap is greater than or equal to the preset overlap, based on the confidence and overlap of the predicted bounding boxes of all the candidate signature areas, to obtain all the remaining target candidate signature areas, and trigger the detection module to perform the operation of comparing the probability of the category to which each of the candidate signature areas belongs, the confidence of the predicted bounding box with the corresponding preset probability and the first preset confidence, and obtaining the comparison result, wherein all the candidate signature areas are all the remaining target candidate signature areas.

[0065] As an optional implementation, in the second aspect of the present invention, the device further includes:

[0066] A second determination module is used to obtain a drawing training set and a drawing verification set, wherein the drawing training set consists of a plurality of training drawings and signature positions corresponding to the training drawings, and the drawing verification set consists of a plurality of verification drawings and signature positions corresponding to the verification drawings;

[0067] A training module, used for training the drawing training set based on a predetermined target recognition model;

[0068] A calculation module, used for inputting the drawing verification set into the target recognition model based on a predetermined loss function during the training of the target recognition model, and calculating the loss result of the target recognition model;

[0069] The optimization module is used to optimize the parameters of the target recognition model according to the loss result of the target recognition model to obtain the optimized target recognition model, and determine the optimized target recognition model as the pre-trained signature position recognition model.

[0070] As an optional implementation, in the second aspect of the present invention, the calculation formula of the loss function is as follows:

[0071] ;

[0072] ;

[0073] ;

[0074] ;

[0075] ;

[0076] ;

[0077] ;

[0078] ;

[0079] In the formula, represents the loss result of the target recognition model, represents the loss deviation term between the actual bounding box of the signature position of the verification drawing in the drawing verification set and the corresponding predicted bounding box, represents the actual bounding box of the signature position of the verified drawing in the drawing verification set, respectively represent the abscissa and ordinate of the center point of the actual bounding box of the signature position of the verification drawing in the drawing verification set, represent the width and height of the actual bounding box of the signature position of the verification drawing in the drawing verification set, respectively, represents the predicted bounding box of the signature position obtained after the verification drawing in the drawing verification set is input into the target recognition model, The horizontal coordinate and the vertical coordinate of the center point of the predicted bounding box of the signature position obtained after the verification drawing in the drawing verification set is input into the target recognition model respectively; are respectively the width and height of the predicted bounding box of the signature position obtained after the verification drawing in the drawing verification set is input into the target recognition model, is the regularization coefficient, The parameters that need to be satisfied by the actual bounding box of the signature position of the drawing to be verified in the drawing verification set; represents the gradient between the width of the predicted bounding box of the signature position obtained after the verification drawing in the drawing verification set is input into the target recognition model and the width of the corresponding actual bounding box; represents the gradient between the height of the predicted bounding box of the signature position obtained after the verification drawing in the drawing verification set is input into the target recognition model and the height of the corresponding actual bounding box; , are the width and height after gradient processing respectively; Indicates taking and The minimum value of Indicates taking and The minimum value of Represents the parameters that need to be satisfied by the actual bounding box of the signature position of the verified drawing in the drawing verification set.

[0080] As an optional implementation, in the second aspect of the present invention, the device further includes:

[0081] A second judgment module is used to judge whether there are drawings in the target drawing set that do not meet the recognition requirements of the target recognition model after the second determination module obtains the drawing training set and the drawing verification set, based on the data of each drawing in the target drawing set, wherein the target drawing set includes the drawing training set and the drawing verification set; when it is judged that there are no drawings, triggering the training module to perform the operation of training the drawing training set based on the predetermined target recognition model;

[0082] an adjustment module, configured to, when it is determined that there is, perform an adjustment operation on the drawings in the target drawing set that do not meet the recognition requirement according to the recognition requirement, obtain the target drawing set that meets the recognition requirement, and trigger the training module to perform the operation of training the drawing training set based on the predetermined target recognition model;

[0083] Among them, the recognition requirements include one or more of the following: the drawing quality is greater than or equal to the preset drawing quality, there are no typos in the signature in the drawing, the signature in the drawing does not exceed the signature position, and the drawing pixel size is within the preset drawing pixel size range.

[0084] The third aspect of the present invention discloses another automatic signature device based on deep learning, the device comprising:

[0085] A memory storing executable program code;

[0086] a processor coupled to the memory;

[0087] The processor calls the executable program code stored in the memory to execute part or all of the steps in any one of the deep learning-based automatic signature methods disclosed in the first aspect of the present invention.

[0088] The fourth aspect of the present invention discloses a computer storage medium, which stores computer instructions. When the computer instructions are called, they are used to execute part or all of the steps in any deep learning-based automatic signature method disclosed in the first aspect of the present invention.

[0089] Compared with the prior art, the embodiments of the present invention have the following beneficial effects:

[0090] In an embodiment of the present invention, drawing information and electronic signature matching the target drawing are queried in a drawing database according to identification information of a target drawing to be signed; the drawing information and electronic signature corresponding to the target drawing are generated into a file in a preset format according to a preset format, and the file includes the drawing information and electronic signature corresponding to the target drawing; after obtaining the file in the preset format, a preprocessing operation is performed on the target drawing according to the drawing information corresponding to the target drawing to obtain a preprocessed target drawing; the signature position of the target drawing is identified based on a pre-trained signature position recognition model to achieve accurate positioning of the signature position, reduce the occurrence of missed signature position detection, and improve the recognition accuracy and comprehensiveness of the signature position; and according to the preprocessed target drawing and the drawing information corresponding to the target drawing, a restoration operation is performed on the coordinates of the signature position to obtain the restored signature position, and the electronic signature is automatically inserted into the restored signature position to achieve automatic signing, thereby improving the accuracy, standardization, convenience and consistency of drawing signatures, thereby bringing users a better signature experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0091] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0092] Figure 1 It is a flowchart of an automatic signature method based on deep learning disclosed in an embodiment of the present invention;

[0093] Figure 2 It is a flowchart of another automatic signature method based on deep learning disclosed in an embodiment of the present invention;

[0094] Figure 3 This is a flow chart of an automatic signature device based on deep learning disclosed in an embodiment of the present invention.

[0095] Figure 4 It is a structural schematic diagram of another automatic signature device based on deep learning disclosed in an embodiment of the present invention;

[0096] Figure 5 It is a structural schematic diagram of another automatic signature device based on deep learning disclosed in an embodiment of the present invention;

[0097] Figure 6 It is a schematic diagram of loss convergence curves of an improved IOU and a traditional IOU disclosed in an embodiment of the present invention. DETAILED DESCRIPTION

[0098] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0099] The terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish different objects rather than to describe a specific order. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, device, product or end including a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units that are not listed, or may optionally include other steps or units inherent to these processes, methods, products or ends.

[0100] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present invention. The appearance of the phrase in various places in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0101] The present invention discloses an automatic signature method and device based on deep learning. According to the identification information of the target drawing to be signed, the drawing information and electronic signature matching the target drawing are queried in the drawing database; according to the preset format, the drawing information and electronic signature corresponding to the target drawing are generated into a file in a preset format, and the file contains the drawing information and electronic signature corresponding to the target drawing; after obtaining the file in the preset format, according to the drawing information corresponding to the target drawing, the target drawing is preprocessed to obtain the preprocessed target drawing; based on the pre-trained signature position recognition model, the signature position of the target drawing is recognized to achieve accurate positioning of the signature position, reduce the occurrence of signature position leakage, improve the recognition accuracy and comprehensiveness of the signature position, and according to the preprocessed target drawing and the drawing information corresponding to the target drawing, the coordinates of the signature position are restored to obtain the restored signature position, and the electronic signature is automatically inserted into the restored signature position to achieve automatic signing, thereby improving the accuracy, standardization, convenience and consistency of the drawing signature, thereby bringing a better signing experience to the user. The following are detailed descriptions.

[0102] Embodiment 1

[0103] See also Figure 1 , Figure 1 : is a flowchart of an automatic signature method based on deep learning disclosed in an embodiment of the present invention. Figure 1 The described method can be applied to any scenario where signature is required (such as power facility drawing design scenario, highway reconstruction and expansion drawing design scenario). Figure 1 As shown, the automatic signature method based on deep learning may include the following operations:

[0104] 101. According to the identification information of the target drawing to be signed, the drawing information and electronic signature matching the target drawing are searched in the drawing database.

[0105] In the embodiment of the present invention, the target drawing is any drawing that needs to be signed. The drawing database stores drawing information of multiple drawings and corresponding electronic signatures, wherein the drawing information includes but is not limited to drawing size, drawing type, drawing pixels, drawing color, etc.

[0106] 102. According to a preset format, the drawing information and the electronic signature corresponding to the target drawing are converted into a file in a preset format, wherein the file includes the drawing information and the electronic signature corresponding to the target drawing.

[0107] In the embodiment of the present invention, the preset format may be PDF, DPG, DOC, or JPG format, which is not limited in the embodiment of the present invention.

[0108] 103. After obtaining the file in the preset format, a preprocessing operation is performed on the target drawing according to the drawing information corresponding to the target drawing to obtain a preprocessed target drawing.

[0109] In the embodiment of the present invention, optionally, the preprocessing operation includes a filtering operation, a resolution adjustment and a size adjustment operation, and the like.

[0110] 104. Identify the signature position of the target drawing based on a pre-trained signature position recognition model; perform a restoration operation on the coordinates of the signature position according to the pre-processed target drawing and the drawing information corresponding to the target drawing to obtain the restored signature position, and insert the electronic signature into the restored signature position.

[0111] In the embodiment of the present invention, the preprocessed target drawing is input into a pre-trained signature position recognition model for analysis to obtain the signature position of the target drawing.

[0112] In the embodiment of the present invention, the restoration operation is related to the aforementioned preprocessing. If the width of the target drawing is W, the height is H, the origin of the coordinate system is the upper left corner of the target drawing, and the identification coordinate point is (X, Y), if the bottom 1 / 6 of the image of the target drawing is taken, the coordinate Y is restored to 5*H / 6 + Y, and if the middle 1 / 5 of the target drawing is taken, the coordinate X is restored to 2*W / 5 + X. By restoring the coordinates of the target drawing, a signature position that is consistent with the actual drawing size and accurate can be obtained, thereby improving the recognition accuracy of the signature position, and further improving the signature accuracy.

[0113] In the embodiment of the present invention, after the electronic signature is inserted into the signature position, the file is saved and uploaded to FTP.

[0114] It can be seen that the implementation Figure 1The described automatic signature method based on deep learning queries the drawing information and electronic signature matching the target drawing in the drawing database according to the identification information of the target drawing to be signed; generates a file in a preset format according to the drawing information and electronic signature corresponding to the target drawing, which contains the drawing information and electronic signature corresponding to the target drawing; after obtaining the file in the preset format, performs a preprocessing operation on the target drawing according to the drawing information corresponding to the target drawing to obtain the preprocessed target drawing; identifies the signature position of the target drawing based on the pre-trained signature position recognition model to achieve accurate positioning of the signature position, reduce the occurrence of missed signature position detection, improve the recognition accuracy and comprehensiveness of the signature position, and perform a restoration operation on the coordinates of the signature position according to the preprocessed target drawing and the drawing information corresponding to the target drawing to obtain the restored signature position, and automatically insert the electronic signature into the restored signature position to achieve automatic signing, thereby improving the accuracy, standardization, convenience and consistency of drawing signatures, thereby bringing users a better signature experience.

[0115] In an optional embodiment, identifying the signature position of the target drawing based on a pre-trained signature position recognition model includes:

[0116] Input the target drawing into the pre-trained first feature extraction model to extract features, and obtain a high-level feature map of the target drawing;

[0117] Input the high-level feature map of the target drawing into the pre-trained candidate region analysis model for analysis to obtain multiple candidate signature regions of the target drawing;

[0118] For any candidate signature area, extract the local features of the candidate signature area through the pre-trained second feature extraction model, and input the local features of the candidate signature area into the pre-trained parameter analysis model for analysis to obtain the probability of the category to which the candidate signature area belongs and the predicted bounding box;

[0119] Determining the confidence of the predicted bounding box of each candidate signature region according to the predicted bounding box of each candidate signature region;

[0120] Comparing the probability of the category to which each candidate signature region belongs and the confidence of the predicted bounding box among all the candidate signature regions with the corresponding preset probability and the first preset confidence to obtain a comparison result;

[0121] According to the comparison result, all candidate signature areas whose probability of the category to which they belong is greater than or equal to the preset probability or whose confidence of the predicted bounding box is greater than or equal to the first preset confidence are determined as the signature positions of the target drawing.

[0122] In this optional embodiment, the local features of the candidate signature area are input into a pre-trained parameter analysis model for analysis to obtain the probability of the category to which the candidate signature area belongs and the predicted bounding box, including:

[0123] Inputting the local features of the candidate signature area into the pre-trained first parameter analysis model for analysis to obtain the probability of the category to which the candidate signature area belongs;

[0124] The local features of the candidate signature area are input into the pre-trained second parameter analysis model for analysis to obtain a predicted bounding box of the candidate signature area.

[0125] In this optional embodiment, there may be one or more signature positions on the target drawing. However, no matter how many signature positions there are, one predicted bounding box finally corresponds to one signature position.

[0126] In this optional embodiment, the signature position recognition model is composed of a first feature extraction model, a second feature extraction model, a candidate region analysis model, a first parameter analysis model, and a second parameter analysis model. Among them, the signature position recognition model can be Faster R-CNN, and the first feature extraction model can be a VGG convolutional layer model of Faster R-CNN, the second feature extraction model can be a ROI model of Faster R-CNN, the candidate region analysis model can be an RPN model of Faster R-CNN, the first parameter analysis model can be a VGG fully connected layer model of Faster R-CNN, and the second parameter analysis model can be a classification regression fully connected layer model of Faster R-CNN.

[0127] It can be seen that this optional embodiment can perform corresponding analysis on the target drawing or its features based on different models in turn to obtain the final signature position, and can achieve accurate and reliable identification of the signature position of the drawing.

[0128] In another optional embodiment, before comparing the probability of the category to which each candidate signature region belongs and the confidence of the predicted bounding box among all candidate signature regions with the corresponding preset probability and the first preset confidence, and obtaining the comparison result, the method may further include the following steps:

[0129] Determining the overlap of the predicted bounding boxes of all candidate signature regions based on the predicted bounding box of each candidate signature region;

[0130] According to the confidence and overlap of the predicted bounding boxes of all candidate signature areas, it is determined whether there is a candidate signature area whose confidence is less than or equal to the second preset confidence or whose overlap is greater than or equal to the preset overlap among all candidate signature areas, and when it is determined that there is no candidate signature area, triggering the above-mentioned operation of comparing the probability of the category to which each candidate signature area belongs and the confidence of the predicted bounding box among all candidate signature areas with the corresponding preset probability and the first preset confidence to obtain a comparison result;

[0131] When it is determined that it exists, based on the confidence and overlap of the predicted bounding boxes of all candidate signature areas, the candidate signature areas whose confidence is less than or equal to the second preset confidence or whose overlap is greater than or equal to the preset overlap are deleted from all candidate signature areas to obtain all remaining target candidate signature areas, and trigger the execution of the above-mentioned operation of comparing the probability of the category to which each candidate signature area in all candidate signature areas belongs, the confidence of the predicted bounding box with the corresponding preset probability and the first preset confidence to obtain the comparison result, and all candidate signature areas are all remaining target candidate signature areas.

[0132] In this optional embodiment, the first preset reliability is greater than the second preset reliability.

[0133] It can be seen that this optional embodiment removes predicted bounding boxes with low confidence or high overlap, thereby reducing the possibility that the signature position cannot be accurately identified due to the possibility that one predicted bounding box covers multiple signature positions or one signature position is covered by multiple predicted bounding boxes, thereby reducing the occurrence of accurate signing.

[0134] In yet another optional embodiment, the method may further include the following steps:

[0135] Determine the number of determined signature positions, and when the number is equal to 1, determine whether the number of electronic signature types is 1, and when it is determined that the number of electronic signature types is 1, trigger the execution of the above-mentioned operation of inserting the electronic signature into the restored signature position;

[0136] When the number of signature positions is greater than 2 and is equal to the number of electronic signature types, the type of the target drawing is obtained, and the signature position corresponding to each electronic tag is determined according to the type of the target drawing;

[0137] Obtain the signature size of each electronic signature, and based on the signature size of each electronic signature and the corresponding signature position, determine whether all electronic signatures can be inserted into the corresponding signature position with the current signature size. When the judgment result is yes, perform the above-mentioned operation of inserting the electronic signature into the restored signature position.

[0138] Further, when the result of the determination is negative, the signature sizes of all signature positions are calculated, and the sum of the current sizes of all electronic signatures is obtained;

[0139] Calculate the size difference between the sum of the current sizes of all electronic signatures and the signature sizes of all signature positions, and obtain the complexity of each electronic signature. The greater the complexity, the greater the signature size required for the signature position.

[0140] According to the complexity and size difference of all electronic signatures, the sizes of all electronic signatures are adjusted to obtain all adjusted electronic signatures, and the above-mentioned operation of inserting the electronic signatures into the restored signature positions is performed.

[0141] In this optional embodiment, even if it is the same electronic signature, the corresponding signature positions are different for different types of target drawings. If the target drawing is of type M, the electronic signature P corresponds to the A signature position. If the target drawing is of type N, the electronic signature P corresponds to the C signature position.

[0142] It can be seen that after obtaining the signature position, this optional embodiment further determines that there is one signature position and the type of the electronic tag is one type before inserting the electronic signature into the signature position; if there are multiple signature positions and there are multiple electronic signatures, the signature position of each electronic signature is further determined based on the drawing type, and the electronic signature is inserted into the signature position only when the size of all signature positions matches the size of the electronic signature; if they do not match, the size of the electronic signature is further adjusted according to the size difference and the complexity of the electronic signature, and finally the electronic signature is inserted into the signature position. The correspondence between the signature position and the electronic signature is considered from multiple angles, which can further improve the accuracy of inserting the electronic signature into the corresponding signature position.

[0143] Embodiment 2

[0144] See also Figure 2 , Figure 2 : is a flowchart of an automatic signature method based on deep learning disclosed in an embodiment of the present invention. Figure 2 The described method can be applied to any scenario where signature is required (such as power facility drawing design scenario, highway reconstruction and expansion drawing design scenario). Figure 2 As shown, the automatic signature method based on deep learning may include the following operations:

[0145] 201. Obtain a drawing training set and a drawing verification set, wherein the drawing training set consists of a plurality of training drawings and signature positions corresponding to the training drawings, and the drawing verification set consists of a plurality of verification drawings and signature positions corresponding to the verification drawings.

[0146] 202. Train the training set based on a predetermined target recognition model.

[0147] 203. During the training of the target recognition model, based on a predetermined loss function, the drawing verification set is input into the target recognition model, the loss result of the target recognition model is calculated, and according to the loss result of the target recognition model, the parameters of the target recognition model are optimized to obtain an optimized target recognition model, and the optimized target recognition model is determined as the pre-trained signature position recognition model.

[0148] In the embodiment of the present invention, the optimized target recognition model is a model whose loss value is less than or equal to the preset loss value. And the loss function can be a function improved based on the IOU loss function, such as Figure 6 As shown, Figure 6 Schematic diagram of loss convergence curves of an improved IOU and a traditional IOU disclosed in an embodiment of the present invention. Figure 6 It can be seen that the loss effect obtained by improving IOU is better, that is, the accuracy of the trained signature position recognition model is higher.

[0149] In the embodiment of the present invention, the formula of the above loss function is as follows:

[0150] ;

[0151] ;

[0152] ;

[0153] ;

[0154] ;

[0155] ;

[0156] ;

[0157] ;

[0158] ;

[0159] = ;

[0160] In the formula, Represents the loss result of the target recognition model, Represents the loss deviation term between the actual bounding box of the signature position of the verification drawing in the drawing verification set and the corresponding predicted bounding box, Represents the actual bounding box of the signature location on the verified drawing in the drawing verification set, They represent the horizontal and vertical coordinates of the center point of the actual bounding box of the signature position of the verification drawing in the drawing verification set, respectively. They represent the width and height of the actual bounding box of the signature position of the verification drawing in the drawing verification set, represents the predicted bounding box of the signature position obtained after verifying the drawing input target recognition model in the drawing verification set, They are the horizontal and vertical coordinates of the center point of the predicted bounding box of the signature position obtained after the drawing is input into the target recognition model in the drawing verification set; are the width and height of the predicted bounding box of the signature position obtained after the drawing is input into the target recognition model in the drawing verification set, is the regularization coefficient, The parameters that need to be satisfied by the actual bounding box of the signature position of the drawing to be verified in the drawing verification set; Represents the gradient between the width of the predicted bounding box of the signature position obtained after verifying the drawing input target recognition model in the drawing verification set and the width of the corresponding actual bounding box; It represents the gradient between the height of the predicted bounding box of the signature position obtained after verifying the drawing input target recognition model in the drawing verification set and the height of the corresponding actual bounding box; , are the width and height after gradient processing respectively; Indicates taking and The minimum value of Indicates taking and The minimum value of Represents the parameters that need to be satisfied by the actual bounding box of the signature position of the verified drawing in the drawing verification set.

[0161] 204. According to the identification information of the target drawing to be signed, in the drawing database, the drawing information and electronic signature matching the target drawing are searched.

[0162] 205. According to a preset format, the drawing information and the electronic signature corresponding to the target drawing are converted into a file in a preset format, wherein the file includes the drawing information and the electronic signature corresponding to the target drawing.

[0163] 206. After obtaining the file in the preset format, a preprocessing operation is performed on the target drawing according to the drawing information corresponding to the target drawing to obtain a preprocessed target drawing.

[0164] 207. Identify the signature position of the target drawing based on the pre-trained signature position recognition model; perform a restoration operation on the coordinates of the signature position according to the pre-processed target drawing and the drawing information corresponding to the target drawing to obtain the restored signature position, and insert the electronic signature into the restored signature position.

[0165] In the embodiment of the present invention, for other descriptions of step 204 to step 207, please refer to the detailed description of step 101 to step 104 in the first embodiment, which will not be repeated in the embodiment of the present invention.

[0166] It can be seen that the implementation Figure 2 The described automatic signature method based on deep learning queries the drawing information and electronic signature matching the target drawing in the drawing database according to the identification information of the target drawing to be signed; generates a file in a preset format according to the drawing information and electronic signature corresponding to the target drawing, which contains the drawing information and electronic signature corresponding to the target drawing; after obtaining the file in the preset format, performs a preprocessing operation on the target drawing according to the drawing information corresponding to the target drawing to obtain the preprocessed target drawing; identifies the signature position of the target drawing based on the pre-trained signature position recognition model to achieve accurate positioning of the signature position, reduce the occurrence of missed signature position detection, improve the recognition accuracy and comprehensiveness of the signature position, and perform a restoration operation on the coordinates of the signature position according to the preprocessed target drawing and the drawing information corresponding to the target drawing to obtain the restored signature position, and automatically insert the electronic signature into the restored signature position to achieve automatic signing, thereby improving the accuracy, standardization, convenience and consistency of drawing signatures, thereby bringing users a better signature experience. In addition, the target recognition model is pre-trained through the drawing training set, and the target recognition model in training is verified based on the improved loss function and the drawing verification set, and the parameters of the target recognition model are adjusted to obtain the optimal signature position recognition model, thereby improving the determination accuracy, reliability and stability of the signature position recognition model, which is conducive to improving the recognition efficiency and accuracy of the signature position.

[0167] In an optional embodiment, after obtaining the drawing training set and the drawing verification set, the method may further include the following steps:

[0168] According to the data of each drawing in the target drawing set, it is judged whether there is a drawing in the target drawing set that does not meet the recognition requirements of the target recognition model, and the target drawing set includes a drawing training set and a drawing verification set; when it is judged that there is no drawing, triggering the execution of the above-mentioned operation of training the training set based on the predetermined target recognition model;

[0169] When it is determined that there is, an adjustment operation is performed on the drawings in the target drawing set that do not meet the recognition requirements according to the recognition requirements to obtain a target drawing set that meets the recognition requirements, and the above-mentioned operation of training the training set based on the predetermined target recognition model is triggered;

[0170] Among them, the recognition requirements include that the drawing quality is greater than or equal to the preset drawing quality, there are no typos in the signature in the drawing, the signature in the drawing does not exceed the signature position, and the drawing pixel size is within the preset drawing pixel size range.

[0171] It can be seen that this optional embodiment adjusts the drawings in the drawing training set and the drawing verification set that do not meet the model training requirements so that they meet the model recognition requirements, ensures the accuracy of the data, and further ensures the accuracy of the trained signature position recognition model.

[0172] Embodiment 3

[0173] See also Figure 3 , Figure 3 : is a schematic diagram of the structure of an automatic signature device based on deep learning disclosed in an embodiment of the present invention. Figure 3 The described device can be applied to any scenario where signature is required (such as power facility drawing design scenario, highway reconstruction and expansion drawing design scenario), such as Figure 3 As shown, the device comprises:

[0174] The query module 301 is used to query the drawing information and electronic signature matching the target drawing in the drawing database according to the identification information of the target drawing to be signed;

[0175] A generating module 302 is used to generate a file in a preset format according to the preset format, from the drawing information and the electronic signature corresponding to the target drawing, and the file includes the drawing information and the electronic signature corresponding to the target drawing;

[0176] A preprocessing module 303 is used to perform a preprocessing operation on the target drawing according to the drawing information corresponding to the target drawing after obtaining the file in the preset format, so as to obtain a preprocessed target drawing;

[0177] A detection module 304, configured to identify the signature position of the target drawing based on a pre-trained signature position recognition model;

[0178] The restoration module 305 is used to perform a restoration operation on the coordinates of the signature position according to the pre-processed target drawing and the drawing information corresponding to the target drawing to obtain a restored signature position;

[0179] The signature module 306 is used to insert the electronic signature into the restored signature position.

[0180] It can be seen that the implementation Figure 3The described automatic signature device based on deep learning queries the drawing information and electronic signature matching the target drawing in the drawing database according to the identification information of the target drawing to be signed; generates a file in a preset format according to the drawing information and electronic signature corresponding to the target drawing, which contains the drawing information and electronic signature corresponding to the target drawing; after obtaining the file in the preset format, performs a preprocessing operation on the target drawing according to the drawing information corresponding to the target drawing to obtain the preprocessed target drawing; identifies the signature position of the target drawing based on a pre-trained signature position recognition model to achieve accurate positioning of the signature position, reduce the occurrence of missed signature position detection, and improve the recognition accuracy and comprehensiveness of the signature position; and performs a restoration operation on the coordinates of the signature position according to the preprocessed target drawing and the drawing information corresponding to the target drawing to obtain the restored signature position, and automatically inserts the electronic signature into the restored signature position to achieve automatic signing, thereby improving the accuracy, standardization, convenience and consistency of drawing signatures, thereby bringing users a better signature experience.

[0181] In an optional embodiment, the specific manner in which the detection module 304 identifies the signature position of the target drawing based on the pre-trained signature position recognition model includes:

[0182] Input the target drawing into the pre-trained first feature extraction model to extract features, and obtain a high-level feature map of the target drawing;

[0183] Input the high-level feature map of the target drawing into the pre-trained candidate region analysis model for analysis to obtain multiple candidate signature regions of the target drawing;

[0184] For any candidate signature area, extract the local features of the candidate signature area through the pre-trained second feature extraction model, and input the local features of the candidate signature area into the pre-trained parameter analysis model for analysis to obtain the probability of the category to which the candidate signature area belongs and the predicted bounding box;

[0185] Determining the confidence of the predicted bounding box of each candidate signature region according to the predicted bounding box of each candidate signature region;

[0186] Comparing the probability of the category to which each candidate signature region belongs and the confidence of the predicted bounding box among all the candidate signature regions with the corresponding preset probability and the first preset confidence to obtain a comparison result;

[0187] According to the comparison result, all candidate signature areas whose probability of the category to which they belong is greater than or equal to the preset probability or whose confidence of the predicted bounding box is greater than or equal to the first preset confidence are determined as the signature positions of the target drawing.

[0188] In this optional embodiment, the detection module 304 inputs the local features of the candidate signature area into the pre-trained parameter analysis model for analysis, and obtains the probability of the category to which the candidate signature area belongs and the specific method of predicting the bounding box includes:

[0189] Inputting the local features of the candidate signature area into the pre-trained first parameter analysis model for analysis to obtain the probability of the category to which the candidate signature area belongs;

[0190] The local features of the candidate signature area are input into the pre-trained second parameter analysis model for analysis to obtain a predicted bounding box of the candidate signature area.

[0191] In this optional embodiment, there may be one or more signature positions on the target drawing. However, no matter how many signature positions there are, one predicted bounding box finally corresponds to one signature position.

[0192] It can be seen that the implementation Figure 3 The described deep learning-based automatic signature device can also perform corresponding analysis on the target drawing or its features based on different models in turn to obtain the final signature position, thereby enabling accurate and reliable identification of the signature position of the drawing.

[0193] In another optional embodiment, Figure 4 As shown, Figure 4 Another automatic signature device based on deep learning disclosed in an embodiment of the present invention is as follows: Figure 4 As shown, the device may also include:

[0194] A first determination module 307 is used to compare the probability of the category to which each candidate signature region belongs and the confidence of the predicted bounding box among all the candidate signature regions with the corresponding preset probability and the first preset confidence in the detection module 304, and determine the overlap of the predicted bounding boxes of all the candidate signature regions according to the predicted bounding box of each candidate signature region before obtaining the comparison result;

[0195] The first judgment module 308 is used to judge whether there is a candidate signature region whose confidence is less than or equal to the second preset confidence or whose overlap is greater than or equal to the preset overlap among all the candidate signature regions according to the confidence and overlap of the predicted bounding boxes of all the candidate signature regions, and when it is judged that there is no candidate signature region, the detection module 304 is triggered to perform the above-mentioned operation of comparing the probability of the category to which each candidate signature region belongs and the confidence of the predicted bounding box among all the candidate signature regions with the corresponding preset probability and the first preset confidence to obtain the comparison result;

[0196] The first determination module 307 is further used to, when it is determined that there is, delete, from all candidate signature areas, the candidate signature areas whose confidence is less than or equal to the second preset confidence or whose overlap is greater than or equal to the preset overlap, according to the confidence and overlap of the predicted bounding boxes of all candidate signature areas, to obtain all remaining target candidate signature areas, and trigger the detection module 304 to perform the above-mentioned operation of comparing the probability of the category to which each candidate signature area in all candidate signature areas belongs, the confidence of the predicted bounding box with the corresponding preset probability and the first preset confidence to obtain the comparison result, and the all candidate signature areas are all remaining target candidate signature areas.

[0197] It can be seen that the implementation Figure 4 The described deep learning-based automatic signature device can also remove predicted bounding boxes with low confidence or high overlap, thereby reducing the possibility that a predicted bounding box covers multiple signature positions, or a signature position is covered by multiple predicted bounding boxes, resulting in the inability to accurately identify the signature position, thereby preventing accurate signing.

[0198] In yet another optional embodiment, Figure 4 As shown, the device may also include:

[0199] The second determination module 309 is used to obtain a drawing training set and a drawing verification set, wherein the drawing training set consists of a plurality of training drawings and signature positions corresponding to the training drawings, and the drawing verification set consists of a plurality of verification drawings and signature positions corresponding to the verification drawings;

[0200] A training module 310, configured to train a training set based on a predetermined target recognition model;

[0201] The calculation module 311 is used to input the drawing verification set into the target recognition model based on a predetermined loss function during the training process of the target recognition model, and calculate the loss result of the target recognition model;

[0202] The optimization module 312 is used to optimize the parameters of the target recognition model according to the loss result of the target recognition model to obtain the optimized target recognition model, and to determine the optimized target recognition model as a pre-trained signature position recognition model. The drawing verification set is input into the target recognition model to calculate the loss result of the target recognition model.

[0203] In the embodiment of the present invention, the optimized target recognition model is a model whose loss value is less than or equal to a preset loss value, and the loss function can be a function improved based on the IOU loss function.

[0204] In the embodiment of the present invention, the formula of the above loss function is as follows:

[0205] ;

[0206] ;

[0207] ;

[0208] ;

[0209] ;

[0210] ;

[0211] ;

[0212] ;

[0213] ;

[0214] = ;

[0215] In the formula, Represents the loss result of the target recognition model, Represents the loss deviation term between the actual bounding box of the signature position of the verification drawing in the drawing verification set and the corresponding predicted bounding box, Represents the actual bounding box of the signature location on the verified drawing in the drawing verification set, They represent the horizontal and vertical coordinates of the center point of the actual bounding box of the signature position of the verification drawing in the drawing verification set, respectively. They represent the width and height of the actual bounding box of the signature position of the verification drawing in the drawing verification set, represents the predicted bounding box of the signature position obtained after verifying the drawing input target recognition model in the drawing verification set, They are the horizontal and vertical coordinates of the center point of the predicted bounding box of the signature position obtained after the drawing is input into the target recognition model in the drawing verification set; are the width and height of the predicted bounding box of the signature position obtained after the drawing is input into the target recognition model in the drawing verification set, is the regularization coefficient, The parameters that need to be satisfied by the actual bounding box of the signature position of the drawing to be verified in the drawing verification set; Represents the gradient between the width of the predicted bounding box of the signature position obtained after verifying the drawing input target recognition model in the drawing verification set and the width of the corresponding actual bounding box; It represents the gradient between the height of the predicted bounding box of the signature position obtained after verifying the drawing input target recognition model in the drawing verification set and the height of the corresponding actual bounding box; , are the width and height after gradient processing respectively; Indicates taking and The minimum value of Indicates taking and The minimum value of Represents the parameters that need to be satisfied by the actual bounding box of the signature position of the verified drawing in the drawing verification set.

[0216] It can be seen that the implementation Figure 4 The described deep learning-based automatic signature device can also pre-train the target recognition model through a drawing training set, and verify the target recognition model in training based on an improved loss function and a drawing verification set, and adjust the parameters of the target recognition model to obtain the optimal signature position recognition model, thereby improving the determination accuracy, reliability and stability of the signature position recognition model, thereby helping to improve the recognition efficiency and accuracy of the signature position.

[0217] In yet another optional embodiment, Figure 4 As shown, the device also includes:

[0218] The second judging module 313 is used to judge whether there are drawings in the target drawing set that do not meet the recognition requirements of the target recognition model after the second determining module 309 obtains the drawing training set and the drawing verification set, based on the data of each drawing in the target drawing set, and the target drawing set includes the drawing training set and the drawing verification set; when it is judged that there are no drawings, the training module 310 is triggered to perform the above-mentioned operation of training the training set based on the predetermined target recognition model;

[0219] The adjustment module 314 is used for, when it is determined that there is, performing an adjustment operation on the drawings in the target drawing set that do not meet the recognition requirement according to the recognition requirement, obtaining a target drawing set that meets the recognition requirement, and triggering the training module 310 to perform an operation of training the training set based on a predetermined target recognition model;

[0220] Among them, the recognition requirements include that the drawing quality is greater than or equal to the preset drawing quality, there are no typos in the signature in the drawing, the signature in the drawing does not exceed the signature position, and the drawing pixel size is within the preset drawing pixel size range.

[0221] It can be seen that the implementation Figure 4 The described deep learning-based automatic signature device can also adjust the drawings in the drawing training set and the drawing verification set that do not meet the model training requirements so that they meet the model recognition requirements, ensure the accuracy of the data, and further ensure the accuracy of the trained signature position recognition model.

[0222] Embodiment 4

[0223] See also Figure 5 , Figure 5 is a schematic diagram of the structure of another automatic signature device based on deep learning disclosed in an embodiment of the present invention. Figure 5 The described apparatus may include:

[0224] A memory 401 storing executable program codes;

[0225] a processor 402 coupled to the memory 401;

[0226] Furthermore, it may also include an input interface 403 and an output interface 404 coupled to the processor 402;

[0227] The processor 402 calls the executable program code stored in the memory 401 to execute part or all of the steps in the automatic signature method based on deep learning disclosed in Embodiment 1 or Embodiment 2 of the present invention.

[0228] Embodiment 5

[0229] An embodiment of the present invention discloses a computer storage medium, which stores computer instructions. When the computer instructions are called, they are used to execute some or all of the steps in the deep learning-based automatic signature method disclosed in Embodiment 1 or Embodiment 2 of the present invention.

[0230] The device embodiments described above are only illustrative, wherein the modules described as separate components may or may not be physically separated, and the components displayed as modules may or may not be physical modules, i.e., they may be located in one place, or they may be distributed on multiple network modules. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Those of ordinary skill in the art may understand and implement it without creative work.

[0231] Through the specific description of the above embodiments, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on such an understanding, the above technical solution can be essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, and the storage medium includes a read-only memory (ROM), a random access memory (RAM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), a one-time programmable read-only memory (OTPROM), an electronically erasable rewritable read-only memory (EEPROM), a compact disc (CD-ROM) or other optical disc storage, magnetic disk storage, magnetic tape storage, or any other computer-readable medium that can be used to carry or store data.

[0232] Finally, it should be noted that the deep learning-based automatic signature method and device disclosed in the embodiments of the present invention only disclose the preferred embodiments of the present invention, which are only used to illustrate the technical solution of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments can still be modified, or some of the technical features therein can be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. An automatic signature method based on deep learning, characterized in that: The method comprises: According to the identification information of the target drawing to be signed, in the drawing database, the drawing information and electronic signature matching the target drawing are searched; According to a preset format, the drawing information and the electronic signature corresponding to the target drawing are converted into a file in the preset format, wherein the file includes the drawing information and the electronic signature corresponding to the target drawing; After obtaining the file in the preset format, performing a preprocessing operation on the target drawing according to the drawing information corresponding to the target drawing to obtain the preprocessed target drawing; Identify the signature position of the target drawing based on a pre-trained signature position recognition model; According to the preprocessed target drawing and the drawing information corresponding to the target drawing, the coordinates of the signature position are restored to obtain the restored signature position, and the electronic signature is inserted into the restored signature position; The step of identifying the signature position of the target drawing based on a pre-trained signature position recognition model includes: Input the target drawing into a pre-trained first feature extraction model to perform feature extraction, thereby obtaining a high-level feature map of the target drawing; Inputting the high-level feature map of the target drawing into a pre-trained candidate region analysis model for analysis to obtain a plurality of candidate signature regions of the target drawing; For any of the candidate signature regions, extract local features of the candidate signature region through a pre-trained second feature extraction model, and input the local features of the candidate signature region into a pre-trained parameter analysis model for analysis to obtain the probability of the category to which the candidate signature region belongs and a predicted bounding box; Determining the confidence of the predicted bounding box of each candidate signature region according to the predicted bounding box of each candidate signature region; Comparing the probability of the category to which each of the candidate signature regions belongs and the confidence of the predicted bounding box with the corresponding preset probability and the first preset confidence to obtain a comparison result; According to the comparison result, all the candidate signature areas whose probability of the category to which they belong is greater than or equal to the preset probability or whose confidence of the predicted bounding box is greater than or equal to the first preset confidence are determined as the signature positions of the target drawing.

2. The automatic signature method based on deep learning according to claim 1, characterized in that: The inputting the local features of the candidate signature area into a pre-trained parameter analysis model for analysis to obtain the probability and predicted bounding box of the category to which the candidate signature area belongs, includes: Inputting the local features of the candidate signature area into a pre-trained first parameter analysis model for analysis to obtain the probability of the category to which the candidate signature area belongs; The local features of the candidate signature area are input into a pre-trained second parameter analysis model for analysis to obtain a predicted bounding box of the candidate signature area.

3. The automatic signature method based on deep learning according to claim 1, characterized in that: Before comparing the probability of the category to which each of the candidate signature regions belongs and the confidence of the predicted bounding box with the corresponding preset probability and the first preset confidence to obtain the comparison result, the method further includes: Determining the overlap of the predicted bounding boxes of all the candidate signature regions according to the predicted bounding box of each candidate signature region; According to the confidence and overlap of the predicted bounding boxes of all the candidate signature areas, it is determined whether there is a candidate signature area whose confidence is less than or equal to the second preset confidence or whose overlap is greater than or equal to the preset overlap among all the candidate signature areas, and when it is determined that there is no candidate signature area, triggering the operation of comparing the probability of the category to which each candidate signature area belongs and the confidence of the predicted bounding box among all the candidate signature areas with the corresponding preset probability and the first preset confidence to obtain a comparison result; When it is determined that it exists, based on the confidence and overlap of the predicted bounding boxes of all the candidate signature areas, the candidate signature areas whose confidence is less than or equal to the second preset confidence or whose overlap is greater than or equal to the preset overlap are deleted from all the candidate signature areas to obtain all the remaining target candidate signature areas, and trigger the execution of the operation of comparing the probability of the category to which each of the candidate signature areas belongs, the confidence of the predicted bounding box with the corresponding preset probability and the first preset confidence to obtain the comparison result, and all the candidate signature areas are all the remaining target candidate signature areas.

4. The automatic signature method based on deep learning according to any one of claims 1 to 3, characterized in that: The method further comprises: Obtain a drawing training set and a drawing verification set, wherein the drawing training set consists of a plurality of training drawings and signature positions corresponding to the training drawings, and the drawing verification set consists of a plurality of verification drawings and signature positions corresponding to the verification drawings; Training the drawing training set based on a predetermined target recognition model; In the process of training the target recognition model, based on a predetermined loss function, the drawing verification set is input into the target recognition model to calculate the loss result of the target recognition model; According to the loss result of the target recognition model, the parameters of the target recognition model are optimized to obtain the optimized target recognition model, and the optimized target recognition model is determined as the pre-trained signature position recognition model.

5. The automatic signature method based on deep learning according to claim 4 is characterized in that: The formula of the loss function is as follows: In the formula, represents the loss result of the target recognition model, represents the loss deviation term between the actual bounding box of the signature position of the verification drawing in the drawing verification set and the corresponding predicted bounding box, represents the actual bounding box of the signature position of the verified drawing in the drawing verification set, respectively represent the abscissa and ordinate of the center point of the actual bounding box of the signature position of the verification drawing in the drawing verification set, represent the width and height of the actual bounding box of the signature position of the verification drawing in the drawing verification set, respectively, represents the predicted bounding box of the signature position obtained after the verification drawing in the drawing verification set is input into the target recognition model, The horizontal coordinate and the vertical coordinate of the center point of the predicted bounding box of the signature position obtained after the verification drawing in the drawing verification set is input into the target recognition model respectively; are respectively the width and height of the predicted bounding box of the signature position obtained after the verification drawing in the drawing verification set is input into the target recognition model, is the regularization coefficient, The parameters that need to be satisfied by the actual bounding box of the signature position of the drawing to be verified in the drawing verification set; represents the gradient between the width of the predicted bounding box of the signature position obtained after the verification drawing in the drawing verification set is input into the target recognition model and the width of the corresponding actual bounding box; represents the gradient between the height of the predicted bounding box of the signature position obtained after the verification drawing in the drawing verification set is input into the target recognition model and the height of the corresponding actual bounding box; Indicates taking and The minimum value of Indicates taking and The minimum value of Represents the parameters that need to be satisfied by the actual bounding box of the signature position of the verified drawing in the drawing verification set.

6. The automatic signature method based on deep learning according to claim 4 is characterized in that: After obtaining the drawing training set and the drawing verification set, the method further includes: According to the data of each drawing in the target drawing set, it is judged whether there is a drawing in the target drawing set that does not meet the recognition requirements of the target recognition model, wherein the target drawing set includes the drawing training set and the drawing verification set; when it is judged that there is no drawing, triggering the execution of the operation of training the drawing training set based on the predetermined target recognition model; When it is determined that there is, an adjustment operation is performed on the drawings in the target drawing set that do not meet the recognition requirement according to the recognition requirement to obtain the target drawing set that meets the recognition requirement, and the operation of training the drawing training set based on the predetermined target recognition model is triggered; Among them, the recognition requirements include one or more of the following: the drawing quality is greater than or equal to the preset drawing quality, there are no typos in the signature in the drawing, the signature in the drawing does not exceed the signature position, and the drawing pixel size is within the preset drawing pixel size range.

7. An automatic signature device based on deep learning, characterized in that: The device is applied to the automatic signature method based on deep learning according to any one of claims 1 to 6, and the device includes: A query module, used to query the drawing information and electronic signature matching the target drawing in the drawing database according to the identification information of the target drawing to be signed; A generating module, configured to generate a file in the preset format by converting the drawing information and the electronic signature corresponding to the target drawing into the file in the preset format, wherein the file includes the drawing information and the electronic signature corresponding to the target drawing; A preprocessing module, configured to, after obtaining the file in the preset format, perform a preprocessing operation on the target drawing according to the drawing information corresponding to the target drawing, so as to obtain the preprocessed target drawing; A detection module, used for identifying the signature position of the target drawing based on a pre-trained signature position recognition model; A restoration module, used to perform a restoration operation on the coordinates of the signature position according to the preprocessed target drawing and the drawing information corresponding to the target drawing, so as to obtain the restored signature position; The signature module is used to insert the electronic signature into the restored signature position.

8. An automatic signature device based on deep learning, characterized in that: The device comprises: A memory storing executable program code; a processor coupled to the memory; The processor calls the executable program code stored in the memory to execute the deep learning-based automatic signature method as described in any one of claims 1-6.

9. A computer storage medium, characterized in that The computer storage medium stores computer instructions, which, when called, are used to execute the deep learning-based automatic signature method as described in any one of claims 1-6.

Images