Identity information extraction method and device, electronic equipment and storage medium
By acquiring sample identity images and annotations, the identity information extraction model is optimized, which solves the problem of low accuracy in identity information extraction when the image quality is poor, and achieves higher extraction accuracy and model generalization ability.
Patent Information
- Application Number
- CN202411586963.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-11-07
- Publication Date
- 2025-02-28
AI Technical Summary
In the prior art, poor image quality leads to low extraction accuracy when extracting identity information, which makes it difficult to effectively solve this problem.
By obtaining sample identity images and labels, using the preset original identity information extraction model for information extraction, and optimizing the model based on the annotation and prediction information, the target identity information extraction model is obtained, thereby improving the extraction accuracy.
Through model optimization, the accuracy and generalization ability of identity information extraction are improved, and identity information can still be accurately extracted when the image quality is poor.
Smart Images
Figure CN119540957A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of artificial intelligence technology, is applicable to the field of financial technology, and particularly relates to a method and device for extracting identity information, an electronic device, and a storage medium. Background Art
[0002] Identity information extraction is an artificial intelligence technology that can automatically identify and extract key information about personal identity from identity image data in a structured manner. It can be applied to a variety of scenarios. For example, in the financial field, it can be applied to real-name authentication, payment verification, anti-fraud and risk management.
[0003] At present, optical character recognition technology is mainly used to extract identity information from identity images. This technology has high requirements for identity images. When the image quality is poor, the accuracy of identity information extraction will be poor.
[0004] Therefore, how to improve the accuracy of identity information extraction has become a technical problem that needs to be solved urgently. Summary of the invention
[0005] The main purpose of the embodiments of the present application is to provide a method and device for extracting identity information, an electronic device and a storage medium, aiming to improve the accuracy of extracting identity information.
[0006] To achieve the above purpose, a first aspect of an embodiment of the present application proposes a method for extracting identity information, the method comprising:
[0007] Acquire sample identity data; wherein the sample identity data includes a sample identity image and a sample identity annotation, and the sample identity annotation is used to characterize the identity information contained in the sample identity image;
[0008] Extracting information from the sample identity image using a preset original identity information extraction model to obtain sample identity prediction information;
[0009] Optimizing the original identity information extraction model based on the sample identity annotation and the sample identity prediction information to obtain a target identity information extraction model;
[0010] The target identity information extraction model is used to extract information from the pre-acquired target identity image to obtain the target identity information.
[0011] In some embodiments, extracting information from the sample identity image using a preset original identity information extraction model to obtain sample identity prediction information includes:
[0012] Extracting image features of the sample identity image based on the sample identity annotation to obtain sample image feature data;
[0013] The sample image feature data is predicted using the original identity information extraction model to obtain the sample identity prediction information.
[0014] In some embodiments, performing information prediction on the sample image feature data by using the original identity information extraction model to obtain the sample identity prediction information includes:
[0015] Performing feature segmentation on the sample image feature data based on preset identity information distribution parameters to obtain sample identity segmentation features;
[0016] Performing information prediction on the sample identity division feature through the original identity information extraction model to obtain sample identity prediction sub-information;
[0017] The sample identity prediction sub-information is combined based on a preset identity information template to obtain the sample identity prediction information.
[0018] In some embodiments, the step of extracting image features from the sample identity image based on the sample identity annotation to obtain sample image feature data includes:
[0019] Performing image preprocessing on the sample identity image to obtain sample image data;
[0020] Performing convolution processing on the sample image data to obtain sample convolution feature data;
[0021] Performing pooling processing on the sample convolution feature data to obtain sample pooling feature data;
[0022] Performing feature mapping on the sample pooled feature data to obtain original sample feature data;
[0023] Extract features from the sample identity annotations to obtain original sample text embedding features;
[0024] Attention calculation is performed on the original sample text embedding features and the original sample feature data to obtain the sample image feature data.
[0025] In some embodiments, performing attention calculation on the original sample text embedding feature and the original sample feature data to obtain the sample image feature data includes:
[0026] Mapping and transforming the original sample text embedding features to obtain a sample text mapping matrix; wherein the sample text mapping matrix includes a sample key mapping matrix and a sample value mapping matrix;
[0027] Determining a sample scaling factor according to the data dimension of the sample key mapping matrix;
[0028] Performing aggregation calculation on the original sample feature data and the sample key mapping matrix based on the sample scaling factor to obtain a sample attention score;
[0029] Normalizing the sample attention score to obtain the sample attention weight;
[0030] The sample attention weight and the sample value mapping matrix are aggregated and calculated to obtain the sample image feature data.
[0031] In some embodiments, performing image preprocessing on the sample identity image to obtain sample image data includes:
[0032] Performing image denoising on the sample identity image to obtain a denoised sample identity image;
[0033] Performing image enhancement processing on the denoised sample identity image to obtain an enhanced sample identity image;
[0034] The enhanced sample identity image is resized to obtain the sample image data.
[0035] In some embodiments, the optimizing the original identity information extraction model based on the sample identity annotation and the sample identity prediction information to obtain a target identity information extraction model includes:
[0036] Performing loss calculation on the sample identity label and the sample identity prediction information to obtain identity prediction loss data;
[0037] The parameters of the original identity information extraction model are adjusted based on the identity prediction loss data to obtain the target identity information extraction model.
[0038] To achieve the above-mentioned purpose, a second aspect of an embodiment of the present application provides an identity information extraction device, the device comprising:
[0039] A sample data acquisition module, used to acquire sample identity data; wherein the sample identity data includes a sample identity image and a sample identity annotation, and the sample identity annotation is used to characterize the identity information contained in the sample identity image;
[0040] A sample identity information extraction module is used to extract information from the sample identity image through a preset original identity information extraction model to obtain sample identity prediction information;
[0041] A model optimization module, used to optimize the original identity information extraction model based on the sample identity annotation and the sample identity prediction information to obtain a target identity information extraction model;
[0042] The target identity information extraction module is used to extract information from the pre-acquired target identity image through the target identity information extraction model to obtain the target identity information.
[0043] To achieve the above-mentioned purpose, the third aspect of an embodiment of the present application proposes an electronic device, which includes a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the method described in the first aspect is implemented.
[0044] To achieve the above-mentioned purpose, the fourth aspect of an embodiment of the present application proposes a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method described in the first aspect is implemented.
[0045] The identity information extraction method and device, electronic device and storage medium proposed in the present application obtain sample identity images and sample identity annotations used to characterize the identity information contained in the sample identity images, extract information from the sample identity images through a preset original identity information extraction model, and obtain sample identity prediction information; based on the sample identity annotations and the sample identity prediction information, the original identity information extraction model is optimized, the prediction deviation of the model is continuously corrected, and a target identity information extraction model is obtained, thereby improving the generalization ability of the model, so that when the target identity information extraction model is used to extract information from the pre-acquired target identity image in the subsequent process, the target identity information can be extracted more accurately, thereby improving the accuracy of identity information extraction. BRIEF DESCRIPTION OF THE DRAWINGS
[0046] Figure 1 is a flow chart of the identity information extraction method provided in an embodiment of the present application;
[0047] Figure 2 yes Figure 1 Flow chart of step S102 in FIG.
[0048] Figure 3 yes Figure 2 Flow chart of step S201 in FIG.
[0049] Figure 4 yes Figure 3 Flow chart of step S301 in FIG.
[0050] Figure 5 yes Figure 3 Flow chart of step S306 in FIG.
[0051] Figure 6 yes Figure 2 Flow chart of step S202 in FIG.
[0052] Figure 7 yes Figure 1 Flow chart of step S103 in FIG.
[0053] Figure 8 is a schematic diagram of the structure of an identity information extraction device provided in an embodiment of the present application;
[0054] Fig. 9 It is a schematic diagram of the hardware structure of the electronic device provided in the embodiment of the present application. DETAILED DESCRIPTION
[0055] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.
[0056] It should be noted that, although the functional modules are divided in the device schematic diagram and the logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than the module division in the device or the order in the flowchart. The terms "first", "second", etc. in the specification, claims and the above drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.
[0057] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein are only for the purpose of describing the embodiments of this application and are not intended to limit this application.
[0058] First, some nouns involved in this application are analyzed:
[0059] Artificial intelligence (AI) is a new technical science that studies and develops theories, methods, technologies and application systems for simulating, extending and expanding human intelligence. AI is a branch of computer science that attempts to understand the essence of intelligence and produce a new type of intelligent machine that can respond in a similar way to human intelligence. Research in this field includes robotics, speech recognition, image recognition, natural language processing and expert systems. AI can simulate the information process of human consciousness and thinking. AI is also a theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results.
[0060] Information Extraction: A text processing technology that extracts specified types of entity, relationship, event and other factual information from natural language text and forms structured data output. Information extraction is a technology that extracts specific information from text data. Text data is composed of some specific units, such as sentences, paragraphs, and chapters. Text information is composed of some small specific units, such as characters, words, phrases, sentences, paragraphs, or a combination of these specific units. Extracting noun phrases, names, place names, etc. from text data are all text information extraction. Of course, the information extracted by text information extraction technology can be various types of information.
[0061] Identity information extraction, identity information extraction is an artificial intelligence technology that uses optical character recognition (OCR) and image processing algorithms, or natural language processing technology, to automatically identify and extract key information about personal identity from identity documents or related texts, such as name, gender, date of birth, address, ID number, etc., and convert it into structured data output.
[0062] Identity information extraction technology can be widely used in many fields such as finance, telecommunications, and the Internet to achieve fast and accurate identity authentication and information entry.
[0063] For example, in the financial field, it can be applied to scenarios such as real-name authentication, payment verification, anti-fraud and risk management.
[0064] At present, optical character recognition technology is mainly used to extract identity information from identity images. During the processing, multiple processes need to be executed, such as image correction through perspective transformation, text area detection, text recognition using optical character recognition technology, and finally classification of fields to finally obtain the results of each field. The processing flow is lengthy, and each stage is sequential and serial. The running time and model errors will continue to accumulate, which will eventually affect the efficiency and accuracy of identity information extraction.
[0065] In addition, optical character recognition technology has high requirements for identity images. When the image quality is poor, the accuracy of identity information extraction will be poor.
[0066] Based on this, the embodiments of the present application provide a method and device for extracting identity information, an electronic device and a storage medium, aiming to improve the accuracy of extracting identity information.
[0067] The identity information extraction method and device, electronic device and storage medium provided in the embodiments of the present application are specifically described through the following embodiments. First, the identity information extraction method in the embodiments of the present application is described.
[0068] The embodiments of the present application can acquire and process relevant data based on artificial intelligence technology. Among them, artificial intelligence (AI) is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results.
[0069] AI basic technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing technology, operation / interaction systems, mechatronics, etc. AI software technologies mainly include computer vision technology, robotics technology, biometrics technology, speech processing technology, natural language processing technology, and machine learning / deep learning.
[0070] The identity information extraction method provided in the embodiment of the present application relates to the field of artificial intelligence technology and is applicable to the field of financial technology. The identity information extraction method provided in the embodiment of the present application can be applied to a terminal, can be applied to a server side, or can be software running in a terminal or a server side. In some embodiments, the terminal can be a smart phone, a tablet computer, a laptop computer, a desktop computer, etc.; the server side can be configured as an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms; the software can be an application that implements the identity information extraction method, etc., but is not limited to the above forms.
[0071] The present application can be used in many general or special computer system environments or configurations. For example: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, etc. The present application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present application can also be practiced in distributed computing environments, in which tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media including storage devices.
[0072] It should be noted that in each specific implementation of the present application, when it comes to the need to perform relevant processing based on data related to user identity or characteristics such as user information, user behavior data, user historical data, and user location information, the user's permission or consent will be obtained first, and the collection, use, and processing of these data will comply with relevant laws, regulations, and standards. In addition, when the embodiment of the present application needs to obtain the user's sensitive personal information, the user's separate permission or consent will be obtained through a pop-up window or by jumping to a confirmation page. After clearly obtaining the user's separate permission or consent, the necessary user-related data for the normal operation of the embodiment of the present application will be obtained.
[0073] Figure 1 is an optional flowchart of the identity information extraction method provided in the embodiment of the present application. Figure 1 The method may include but is not limited to steps S101 to S104.
[0074] Step S101, obtaining sample identity data; wherein the sample identity data includes a sample identity image and a sample identity annotation, and the sample identity annotation is used to characterize the identity information contained in the sample identity image;
[0075] Step S102, extracting information from the sample identity image using a preset original identity information extraction model to obtain sample identity prediction information;
[0076] Step S103, optimizing the original identity information extraction model based on the sample identity labeling and the sample identity prediction information to obtain a target identity information extraction model;
[0077] Step S104: extract information from the pre-acquired target identity image using a target identity information extraction model to obtain target identity information.
[0078] In steps S101 to S104 shown in the embodiment of the present application, a sample identity image and a sample identity annotation for characterizing the identity information contained in the sample identity image are obtained, and information is extracted from the sample identity image through a preset original identity information extraction model to obtain sample identity prediction information; the original identity information extraction model is optimized based on the sample identity annotation and the sample identity prediction information, and the prediction deviation of the model is continuously corrected to obtain a target identity information extraction model, thereby improving the generalization ability of the model, so that when the target identity information extraction model is used to extract information from the pre-acquired target identity image in the subsequent process, the identity information can be extracted more accurately, thereby improving the accuracy of identity information extraction.
[0079] In step S101 of some embodiments, the sample identity data is used to perform model training on the original identity information extraction model to improve the generalization ability of the model and the accuracy of information extraction, thereby providing a solid foundation for subsequent identity information extraction.
[0080] The sample identity data includes a sample identity image and a sample identity annotation. The sample identity image may be an identity card image or an image of a related document, and the sample identity annotation is used to characterize the identity information contained in the sample identity image.
[0081] In some embodiments, if the sample identity annotation is a structure separated by multiple fields, the multiple fields need to be combined in a preset order to form a sample identity annotation in the form of a text.
[0082] For example:
[0083] Name: Zhang San,
[0084] Gender: Male,
[0085] Nationality: Han,
[0086] Born: January 1, 1999
[0087] Address: No. 110, xx Town, xx City, xx Province,
[0088] Citizen ID number: 123******56,
[0089] After combining them in order, we get a sample identity in the form of a text: Name: Zhang San Gender: Male Ethnicity: Han Birth Date: January 1, 1999 Address: No. xx, xx Town, xx City, xx Province Citizen ID Number: 123******6.
[0090] See also Figure 2 In some embodiments, step S102 may include but is not limited to steps S201 to S202:
[0091] Step S201, extracting image features from the sample identity image based on the sample identity annotation to obtain sample image feature data;
[0092] Step S202: perform information prediction on the sample image feature data through the original identity information extraction model to obtain sample identity prediction information.
[0093] In steps S201 to S202 shown in the embodiment of the present application, feature extraction is performed on the sample identity image based on the sample identity annotation to obtain sample image feature data, and information prediction is performed on the sample image feature data through the original identity information extraction model to obtain sample identity prediction information. This can efficiently and accurately extract the identity information of the sample from the sample identity image, thereby improving the accuracy and efficiency of identity recognition.
[0094] See also Figure 3 In some embodiments, step S201 may include but is not limited to steps S301 to S306:
[0095] Step S301, performing image preprocessing on the sample identity image to obtain sample image data;
[0096] Step S302, performing convolution processing on the sample image data to obtain sample convolution feature data;
[0097] Step S303, performing pooling processing on the sample convolution feature data to obtain sample pooling feature data;
[0098] Step S304, performing feature mapping on the sample pooling feature data to obtain original sample feature data;
[0099] Step S305, extracting features from the sample identity tags to obtain original sample text embedding features;
[0100] Step S306, performing attention calculation on the original sample text embedding features and the original sample feature data to obtain sample image feature data.
[0101] In steps S301 to S306 shown in the embodiment of the present application, by performing image preprocessing, convolution processing, pooling processing, feature mapping, etc. on the sample identity image, the original sample feature data is obtained, and feature extraction is performed on the sample identity annotation to obtain the original sample text embedding feature, and attention calculation is performed on the original sample text embedding feature and the original sample feature data. This can effectively extract richer and more relevant feature information from the sample identity image and the identity annotation, which helps to enhance the original identity information extraction model's recognition ability and reasoning accuracy for the sample identity.
[0102] In step S201 of some embodiments, image features may be extracted from the sample identity image based on the sample identity annotation by using a preset original image feature extraction model to obtain sample image feature data;
[0103] Specifically, the preset original image feature extraction model can be a ResNet model (such as ResNet-18), a DenseNet model, a ResNeXt model, etc., and the specific selection needs to be combined with the actual application scenario, but is not limited to this.
[0104] See also Figure 4 In some embodiments, step S301 may include but is not limited to steps S401 to S403:
[0105] Step S401, performing image denoising on the sample identity image to obtain a denoised sample identity image;
[0106] Step S402, performing image enhancement processing on the denoised sample identity image to obtain an enhanced sample identity image;
[0107] Step S403, resizing the enhanced sample identity image to obtain sample image data.
[0108] Steps S401 to S403 shown in the embodiment of the present application can effectively improve image quality, reduce noise interference, enhance image features, and ensure the consistency of image data by denoising, enhancing and resizing the sample identity image in sequence, thereby providing more reliable and clearer sample image data for subsequent identity information extraction, thereby improving the accuracy of identity information extraction processing.
[0109] In step S401 of some embodiments, the sample identity image may be denoised by smoothing filtering (such as mean filtering, Gaussian filtering), median filtering, or other techniques, which can reduce noise interference in the identity image and improve image quality.
[0110] In step S402 of some embodiments, the denoised sample identity image may be subjected to contrast enhancement, sharpening, and grayscale adjustment to improve the visual effect of the image and enhance the features of key information areas in the image.
[0111] In step S403 of some embodiments, it is understandable that the size of each sample identity image is not exactly the same. In order to improve the uniformity of the data input into the original identity information extraction model, it is necessary to resize the true strong sample identity image after image denoising and image enhancement based on a preset image size to obtain sample image data.
[0112] It should be noted that the preset image size needs to be determined according to the data requirements of the original image feature extraction model, but is not limited thereto.
[0113] In addition, it is understandable that the sample identity image may also have an angular deviation. In the identity information extraction method provided in the embodiment of the present application, there is no need to perform a correction operation on the sample identity image, which can shorten the image processing time to a certain extent, improve the efficiency of data processing, and thus improve the efficiency of identity information extraction.
[0114] In some embodiments, convolution processing, pooling processing, and feature mapping operations are performed on the sample image data through a preset original image feature extraction model, that is, the above steps S302 to S304 are executed through a preset original image feature extraction model.
[0115] In one embodiment, the preset original image feature extraction model is a ResNet-18 model, which may include multiple image convolution processing layers, average pooling layers, and fully connected layers, and each image convolution processing layer includes an image convolution layer and a residual calculation layer;
[0116] Specifically, the sample image data is convolved and residuals are calculated through multiple image convolution processing layers to obtain sample convolution feature data;
[0117] The sample convolution feature data is pooled through the average pooling layer to reduce the size of the feature map and extract global features to obtain sample pooled feature data;
[0118] Finally, the sample pooling feature data is feature mapped through the fully connected layer to obtain the original sample feature data.
[0119] In step S305 of some embodiments, a preset word segmenter may be used to perform feature extraction on the sample identity annotation to obtain an original sample text embedding feature, wherein the data dimension of the original sample text embedding feature is the same as the data dimension of the original sample feature data.
[0120] Specifically, the preset word segmenter can be a model based on the BPE (Byte Pair Encoding) method, a model based on the SentencePiece Tokenizer method, or other pre-trained word segmentation models, but is not limited thereto.
[0121] See also Figure 5 In some embodiments, step S306 may also include but is not limited to steps S501 to S505:
[0122] Step S501, mapping and transforming the original sample text embedding features to obtain a sample text mapping matrix; wherein the sample text mapping matrix includes a sample key mapping matrix and a sample value mapping matrix;
[0123] Step S502, determining a sample scaling factor according to the data dimension of the sample key mapping matrix;
[0124] Step S503, performing aggregation calculation on the original sample feature data and the sample key mapping matrix based on the sample scaling factor to obtain a sample attention score;
[0125] Step S504, normalize the sample attention score to obtain the sample attention weight;
[0126] Step S505, performing aggregation calculation on the sample attention weight and the sample value mapping matrix to obtain sample image feature data.
[0127] In steps S501 to S505 shown in the embodiment of the present application, a sample key mapping matrix and a sample value mapping matrix are obtained by mapping and transforming the original sample text embedding features, and a sample scaling factor is determined according to the data dimension of the sample key mapping matrix; thereby, the original sample feature data and the sample key mapping matrix are aggregated based on the sample scaling factor to obtain a sample attention score, the sample attention score is normalized to obtain a sample attention weight, the sample attention weight is aggregated with the sample value mapping matrix to obtain sample image feature data, and a cross-attention calculation process is implemented. This can more accurately capture the relationship between text and image features and improve the accuracy and efficiency of feature extraction.
[0128] In step S501 of some embodiments, a sample text mapping matrix is obtained by performing a linear transformation on the original sample text embedding features; wherein the sample text mapping matrix includes a sample key mapping matrix and a sample value mapping matrix;
[0129] In step S502 of some embodiments, the sample scaling factor is used to scale the sample attention score to avoid affecting the effect of attention calculation due to excessively large values.
[0130] Specifically, the sample scaling factor can be obtained by calculating the square root of the data dimension of the sample key mapping matrix and then taking the inverse.
[0131] In some embodiments, step S503 may include but is not limited to:
[0132] Perform linear transformation on the original sample feature data to obtain the original sample query mapping matrix;
[0133] Perform dot product calculation on the sample scaling factor, the original sample query mapping matrix, and the sample key mapping matrix to obtain the sample attention score;
[0134] Specifically, the sample attention score is used to measure the degree of association between the elements in the original sample query mapping matrix and the elements in the sample key mapping matrix.
[0135] In step S504 of some embodiments, the sample attention score can be normalized and calculated by applying methods such as softmax function, Z-score normalization, maximum normalization, etc., so as to obtain the sample attention weight, where the sample attention weight is used to represent the degree of attention of each element in the original sample query mapping matrix and each element in the sample key mapping matrix.
[0136] In step S505 of some embodiments, a weighted sum is calculated on the sample attention weight and the sample value mapping matrix to obtain sample image feature data, which incorporates information from the original sample text embedding features and is dynamically generated based on elements in the original sample feature data.
[0137] See also Figure 6 In some embodiments, step S202 includes but is not limited to steps S601 to S603:
[0138] Step S601, performing feature segmentation on sample image feature data based on preset identity information distribution parameters to obtain sample identity segmentation features;
[0139] Step S602, performing information prediction on the sample identity classification features through the original identity information extraction model to obtain sample identity prediction sub-information;
[0140] Step S603: combining the sample identity prediction sub-information based on a preset identity information template to obtain sample identity prediction information.
[0141] In steps S601 to S603 shown in the embodiment of the present application, sample identity division features are obtained by performing feature division on the sample image feature data based on preset identity information distribution parameters, and then the original identity information extraction model is used to perform information prediction on the sample identity division features to obtain sample identity prediction sub-information. Finally, the sample identity prediction sub-information is combined with the preset identity information template to obtain sample identity prediction information, which can efficiently integrate the identity information extracted from the sample image feature data and improve the integrity and accuracy of identity information recognition.
[0142] It is understandable that the information fields contained in each identity image are fixed, and the field distribution positions are also fixed. Therefore, the sample image feature data can be feature divided based on the preset identity information distribution parameters to obtain multiple fields each corresponding to the sample identity division features.
[0143] In step S602 of some embodiments, the original identity information extraction model can extract text information from the feature data. Specifically, the original identity information extraction model can be a large language model, a model based on a convolutional neural network, or a model based on a machine learning algorithm, which needs to be selected in combination with the actual application scenario, but is not limited thereto.
[0144] In one embodiment, the original identity information extraction model is a GPT2 model, which can predict information on sample identity division features in sequence to obtain multiple sample identity prediction sub-information.
[0145] Finally, the identified multiple sample identity prediction sub-information is filled into the preset identity information template to obtain the sample identity prediction information.
[0146] For example:
[0147] The preset identity information template can be: {name, gender, ethnicity, birthplace, address, citizenship number};
[0148] The identity prediction sub-information of multiple samples are: Li Si, male, Manchu, January 1, 1997, No. 120, xx Town, xx City, xx Province, 231******37;
[0149] Fill multiple sample identity prediction sub-information into the preset identity information template, and obtain the sample identity prediction information: {Name: Li Si, Gender: Male, Nationality: Manchu, Birth: January 1, 1997, Address: No. 120, xx Town, xx City, xx Province, Citizen ID Number: 231******37}.
[0150] See also Figure 7 In some embodiments, step S103 may include but is not limited to steps S701 to S702:
[0151] Step S701, performing loss calculation on sample identity labeling and sample identity prediction information to obtain identity prediction loss data;
[0152] Step S702: adjusting parameters of the original identity information extraction model based on the identity prediction loss data to obtain a target identity information extraction model.
[0153] In steps S701 to S702 shown in the embodiment of the present application, iterative optimization of the model is achieved by calculating the loss of sample identity labeling and sample identity prediction information and adjusting the parameters of the original identity information extraction model accordingly, which can more accurately extract identity information, thereby improving the accuracy and reliability of identity information extraction.
[0154] In step S701 of some embodiments, the loss calculation can be performed using a mean square error loss function, a cross entropy loss function, a likelihood loss function, or other loss functions. The specific selection needs to be made in combination with the actual application scenario, but is not limited thereto.
[0155] In step S702 of some embodiments, a back propagation method, a gradient descent method, or the like may be used to adjust parameters of the original identity information extraction model based on the identity prediction loss data.
[0156] In some embodiments, after step S702, the identity information extraction method further includes but is not limited to the following steps:
[0157] The parameters of the original image feature extraction model are adjusted based on the identity prediction loss data to obtain the target image feature extraction model.
[0158] Specifically, the parameter adjustment method may adopt a back propagation method, a gradient descent method, or the like.
[0159] In step S104 of some embodiments, the target identity image is an ID card image or an image of a related document of the target object. For example, in the financial field, the target object applies for real-name authentication and uploads the target identity image. Therefore, it is necessary to extract the target identity information from the target identity image.
[0160] In some embodiments, step S104 may include but is not limited to the following steps:
[0161] Performing image denoising on the target identity image to obtain a denoised target identity image;
[0162] Performing image enhancement processing on the denoised target identity image to obtain an enhanced target identity image;
[0163] resizing the enhanced target identity image to obtain target image data;
[0164] Performing convolution processing on the target image data through the target image feature extraction model to obtain target convolution feature data, performing pooling processing on the target convolution feature data to obtain target pooling feature data, and performing feature mapping on the target pooling feature data to obtain target image feature data;
[0165] Performing feature segmentation on target image feature data based on preset identity information distribution parameters to obtain target identity segmentation features;
[0166] The target identity information extraction model is used to perform information prediction on the target identity division features to obtain target identity prediction sub-information;
[0167] The target identity prediction sub-information is combined based on a preset identity information template to obtain the target identity prediction information.
[0168] It should be noted that the specific implementation of the above step S104 is basically the same as the specific implementation of the above steps S301 to S304, steps S401 to S403, and steps S601 to S603, and will not be repeated here.
[0169] It can be understood that the identity information extraction method provided in the embodiment of the present application, the identity information prediction model belongs to the OCR-FREE model, that is, a model that is not implemented based on optical character recognition technology, which can be shortened in the process, effectively improve the efficiency of identity information extraction and reduce dependence on hardware resources, and there is no error accumulation in the processing process, which can improve the accuracy of identity information extraction.
[0170] See also Figure 8 The embodiment of the present application also provides an identity information extraction device, which can implement the above identity information extraction method, and the device includes:
[0171] The sample data acquisition module 801 is used to acquire sample identity data; wherein the sample identity data includes a sample identity image and a sample identity annotation, and the sample identity annotation is used to characterize the identity information contained in the sample identity image;
[0172] The sample identity information extraction module 802 is used to extract information from the sample identity image through a preset original identity information extraction model to obtain sample identity prediction information;
[0173] A model optimization module 803 is used to optimize the original identity information extraction model based on the sample identity labeling and the sample identity prediction information to obtain a target identity information extraction model;
[0174] The target identity information extraction module 804 is used to extract information from the pre-acquired target identity image using a target identity information extraction model to obtain target identity information.
[0175] The specific implementation of the identity information extraction device is basically the same as the specific embodiment of the above-mentioned identity information extraction method, and will not be repeated here.
[0176] The embodiment of the present application also provides an electronic device, the electronic device includes a memory and a processor, the memory stores a computer program, and the processor implements the above-mentioned identity information extraction method when executing the computer program. The electronic device can be any intelligent terminal including a tablet computer, a car computer, etc.
[0177] See also Fig. 9 , Fig. 9 The hardware structure of an electronic device of another embodiment is illustrated, and the electronic device includes:
[0178] The processor 901 may be implemented by a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (Application Specific Integrated Circuit, ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of the present application;
[0179] The memory 902 can be implemented in the form of a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 902 can store an operating system and other applications. When the technical solution provided in the embodiments of this specification is implemented by software or firmware, the relevant program code is stored in the memory 902, and the processor 901 calls and executes the identity information extraction method of the embodiment of the present application;
[0180] Input / output interface 903, used to implement information input and output;
[0181] Communication interface 904, used to realize communication interaction between the device and other devices, which can be realized by wired mode (such as USB, network cable, etc.) or wireless mode (such as mobile network, WIFI, Bluetooth, etc.);
[0182] A bus 905 that transmits information between various components of the device (e.g., the processor 901, the memory 902, the input / output interface 903, and the communication interface 904);
[0183] The processor 901 , the memory 902 , the input / output interface 903 and the communication interface 904 are connected to each other in communication within the device via a bus 905 .
[0184] An embodiment of the present application also provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the above-mentioned identity information extraction method is implemented.
[0185] The memory, as a non-transient computer-readable storage medium, can be used to store non-transient software programs and non-transient computer executable programs. In addition, the memory may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory may optionally include a memory remotely disposed relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.
[0186] The identity information extraction method and device, electronic device and storage medium provided in the embodiments of the present application obtain sample identity images and sample identity annotations for characterizing the identity information contained in the sample identity images, extract information from the sample identity images through a preset original identity information extraction model, and obtain sample identity prediction information; optimize the original identity information extraction model based on the sample identity annotations and the sample identity prediction information, continuously correct the prediction deviation of the model, obtain a target identity information extraction model, and improve the generalization ability of the model, so that when the target identity information extraction model is used to extract information from the pre-acquired target identity image in the subsequent process, the target identity information can be extracted more accurately, thereby improving the accuracy of identity information extraction.
[0187] The embodiments described in the embodiments of the present application are intended to more clearly illustrate the technical solutions of the embodiments of the present application and do not constitute a limitation on the technical solutions provided in the embodiments of the present application. Those skilled in the art will appreciate that with the evolution of technology and the emergence of new application scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.
[0188] Those skilled in the art will appreciate that the technical solutions shown in the figures do not constitute a limitation on the embodiments of the present application, and may include more or fewer steps than shown in the figures, or a combination of certain steps, or different steps.
[0189] The device embodiments described above are merely illustrative, and the units described as separate components may or may not be physically separated, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
[0190] Those skilled in the art will appreciate that all or some of the steps in the methods disclosed above, and the functional modules / units in the systems and devices may be implemented as software, firmware, hardware, or a suitable combination thereof.
[0191] The terms "first", "second", "third", "fourth", etc. (if any) in the specification of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.
[0192] It should be understood that in the present application, "at least one (item)" means one or more, and "plurality" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the objects associated before and after are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.
[0193] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the above units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.
[0194] The units described above as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
[0195] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.
[0196] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including multiple instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, referred to as ROM), random access memory (Random Access Memory, referred to as RAM), disk or optical disk and other media that can store programs.
[0197] The preferred embodiments of the present invention are described above with reference to the accompanying drawings, but the scope of the rights of the present invention is not limited thereto. Any modification, equivalent substitution and improvement made by a person skilled in the art without departing from the scope and essence of the present invention should be within the scope of the rights of the present invention.
Claims
1. A method for extracting identity information, characterized in that: The method comprises: Acquire sample identity data; wherein the sample identity data includes a sample identity image and a sample identity annotation, and the sample identity annotation is used to characterize the identity information contained in the sample identity image; Extracting information from the sample identity image using a preset original identity information extraction model to obtain sample identity prediction information; Optimizing the original identity information extraction model based on the sample identity annotation and the sample identity prediction information to obtain a target identity information extraction model; The target identity information extraction model is used to extract information from the pre-acquired target identity image to obtain the target identity information.
2. The method according to claim 1, characterized in that: The extracting information from the sample identity image by using a preset original identity information extraction model to obtain sample identity prediction information includes: Extracting image features of the sample identity image based on the sample identity annotation to obtain sample image feature data; The sample image feature data is predicted using the original identity information extraction model to obtain the sample identity prediction information.
3. The method according to claim 2, characterized in that The performing information prediction on the sample image feature data by using the original identity information extraction model to obtain the sample identity prediction information includes: Performing feature segmentation on the sample image feature data based on preset identity information distribution parameters to obtain sample identity segmentation features; Performing information prediction on the sample identity division feature through the original identity information extraction model to obtain sample identity prediction sub-information; The sample identity prediction sub-information is combined based on a preset identity information template to obtain the sample identity prediction information.
4. The method according to claim 2, characterized in that: The extracting image features of the sample identity image based on the sample identity annotation to obtain sample image feature data includes: Performing image preprocessing on the sample identity image to obtain sample image data; Performing convolution processing on the sample image data to obtain sample convolution feature data; Performing pooling processing on the sample convolution feature data to obtain sample pooling feature data; Performing feature mapping on the sample pooled feature data to obtain original sample feature data; Extract features from the sample identity annotations to obtain original sample text embedding features; Attention calculation is performed on the original sample text embedding features and the original sample feature data to obtain the sample image feature data.
5. The method according to claim 4, characterized in that The performing attention calculation on the original sample text embedding feature and the original sample feature data to obtain the sample image feature data includes: Mapping and transforming the original sample text embedding features to obtain a sample text mapping matrix; wherein the sample text mapping matrix includes a sample key mapping matrix and a sample value mapping matrix; Determining a sample scaling factor according to the data dimension of the sample key mapping matrix; Performing aggregation calculation on the original sample feature data and the sample key mapping matrix based on the sample scaling factor to obtain a sample attention score; Normalizing the sample attention score to obtain the sample attention weight; The sample attention weight and the sample value mapping matrix are aggregated and calculated to obtain the sample image feature data.
6. The method according to claim 4, characterized in that The performing image preprocessing on the sample identity image to obtain sample image data includes: Performing image denoising on the sample identity image to obtain a denoised sample identity image; Performing image enhancement processing on the denoised sample identity image to obtain an enhanced sample identity image; The enhanced sample identity image is resized to obtain the sample image data.
7. The method according to any one of claims 1 to 6, characterized in that: The optimizing the original identity information extraction model based on the sample identity annotation and the sample identity prediction information to obtain a target identity information extraction model includes: Performing loss calculation on the sample identity label and the sample identity prediction information to obtain identity prediction loss data; The parameters of the original identity information extraction model are adjusted based on the identity prediction loss data to obtain the target identity information extraction model.
8. An identity information extraction device, characterized in that: The device comprises: A sample data acquisition module, used to acquire sample identity data; wherein the sample identity data includes a sample identity image and a sample identity annotation, and the sample identity annotation is used to characterize the identity information contained in the sample identity image; A sample identity information extraction module is used to extract information from the sample identity image through a preset original identity information extraction model to obtain sample identity prediction information; A model optimization module, used to optimize the original identity information extraction model based on the sample identity annotation and the sample identity prediction information to obtain a target identity information extraction model; The target identity information extraction module is used to extract information from the pre-acquired target identity image through the target identity information extraction model to obtain the target identity information.
9. An electronic device, characterized in that: The electronic device comprises a memory and a processor, the memory stores a computer program, and the processor implements the method according to any one of claims 1 to 7 when executing the computer program.
10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.