A method, device, computer equipment and storage medium for processing passport images
By detecting and establishing relationships among character regions in passport images, multiple text content regions are extracted and recognized using a text recognition model. This solves the problem of low accuracy in passport image text content recognition in traditional methods and achieves higher recognition accuracy.
Patent Information
- Application Number
- CN202111583465.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-12-22
- Publication Date
- 2024-11-26
- Estimated Expiration
- 2041-12-22
AI Technical Summary
Traditional methods often overlook or misidentify characters with various forms when recognizing text in passport images, resulting in low recognition accuracy.
A trained character detection model is used to detect character regions in passport images and establish multiple associations to extract multiple text content regions. A trained text recognition model is then used to identify the text content of these regions.
It improves the accuracy of text recognition in passport images, avoiding the omission or misidentification of characters with multiple forms as other forms of characters.
Smart Images

Figure CN114445823B_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of computer technology, and in particular to a method, apparatus, computer equipment and storage medium for processing passport images. Background Art
[0002] With the continuous development of technology, more and more devices can detect the characters contained in images and recognize the text content contained in images.
[0003] Traditional methods for detecting characters contained in images and methods for identifying text content contained in images can only detect or identify characters in the same form. However, in some scenarios, such as passport images, an image contains characters in multiple forms. Under related technologies, some of the characters in multiple forms are easily ignored or mistakenly identified as characters in other forms, resulting in incorrect recognition of text content in passport images.
[0004] It can be seen that under the related art, for passport images, the accuracy of the process of identifying the text content contained in the image is low. Summary of the invention
[0005] The embodiments of the present application provide a method, apparatus, computer device and storage medium for processing a passport image, which are used to solve the problem of low accuracy in the process of identifying text content contained in a passport image.
[0006] In a first aspect, a method for processing a passport image is provided, comprising:
[0007] Obtaining passport images to be processed;
[0008] Using a trained character detection model, based on the passport image to be processed, detect the character regions of the respective characters contained in the passport image to be processed;
[0009] Establishing multiple association relationships for each character region, and extracting multiple text content regions from the passport image to be processed corresponding to the multiple association relationships, wherein each association relationship is associated with multiple character regions;
[0010] The trained text recognition model is used to identify the text content contained in the passport image to be processed based on the multiple text content areas.
[0011] Optionally, multiple association relationships are established for each character region, including:
[0012] Based on the area sizes of the respective character areas, determining the character height difference between every two adjacent character areas and the character spacing between every two adjacent character areas;
[0013] Based on the height difference of each character and the distance between each character, a plurality of groups of character areas are selected from each character area, and a plurality of association relationships are established respectively.
[0014] Optionally, based on the height difference of each character and the spacing between each character, multiple groups of character areas are selected from each character area, and multiple association relationships are established respectively, including:
[0015] Based on the height difference of each character and the spacing between each character, clustering the character regions to obtain multiple groups of character regions, wherein the height difference of the characters among the multiple character regions included in each group of character regions is less than a height threshold, and the difference between the character spacings is less than a spacing threshold;
[0016] For the multiple character regions included in each group of character regions, an association relationship is established to obtain the multiple association relationships.
[0017] Optionally, corresponding to the multiple association relationships, extracting multiple text content areas from the passport image to be processed respectively includes:
[0018] Mapping each character area to a coordinate system established based on the passport image to be processed;
[0019] For the various association relationships, perform the following operations respectively:
[0020] Performing linear regression processing on the center coordinates of each of the multiple character regions corresponding to an association relationship to obtain a linear equation;
[0021] Determining the region coordinates of the corresponding text content region based on the coordinates of the intersections between the plurality of character regions and the straight line represented by the linear equation, the slope of the linear equation, and the region sizes of the plurality of character regions;
[0022] Based on the region coordinates, the text content region is extracted from the passport image to be processed.
[0023] Optionally, the trained character detection model is trained using the following method:
[0024] Acquire each initial sample data, wherein each initial sample data includes an initial sample image and a sample content area contained in the initial sample image;
[0025] Extracting the background image of each initial sample image;
[0026] Based on the pre-stored passport feature information, a plurality of random text contents are randomly generated;
[0027] Respectively synthesizing the multiple random text contents and the respective background images to generate multiple newly added sample data;
[0028] Based on the initial sample data and the multiple newly added sample data, the character detection model to be trained is subjected to multiple rounds of iterative training until the training loss meets the training target, and the character detection model to be trained is output as the trained character detection model.
[0029] Optionally, based on the respective initial sample data and the plurality of newly added sample data, the character detection model to be trained is subjected to multiple rounds of iterative training, including:
[0030] Using a plurality of pre-stored pre-processing strategies, respectively adjusting the clarity of each of the initial sample images, and respectively obtaining each of the processed sample images;
[0031] Generate each processed sample data based on each processed sample image and a sample content area included in an initial sample image corresponding to each processed sample image;
[0032] Based on the processed sample data and the multiple newly added sample data, multiple rounds of iterative training are performed on the character detection model to be trained.
[0033] In a second aspect, a device for processing a passport image is provided, comprising:
[0034] Acquisition module: used to obtain the passport image to be processed;
[0035] Processing module: used for detecting the character regions of the respective characters contained in the passport image to be processed by using the trained character detection model based on the passport image to be processed;
[0036] The processing module is further used to: establish multiple association relationships for each character region, and extract multiple text content regions from the passport image to be processed corresponding to the multiple association relationships, wherein each association relationship is associated with multiple character regions;
[0037] The processing module is further used to: use a trained text recognition model to identify text content contained in the passport image to be processed based on the multiple text content areas respectively.
[0038] Optionally, the processing module is specifically used for:
[0039] Based on the area sizes of the respective character areas, determining the character height difference between every two adjacent character areas and the character spacing between every two adjacent character areas;
[0040] Based on the height difference of each character and the distance between each character, a plurality of groups of character areas are selected from each character area, and a plurality of association relationships are established respectively.
[0041] Optionally, the processing module is specifically used for:
[0042] Based on the height difference of each character and the spacing between each character, clustering the character regions to obtain multiple groups of character regions, wherein the height difference of the characters among the multiple character regions included in each group of character regions is less than a height threshold, and the difference between the character spacings is less than a spacing threshold;
[0043] For the multiple character regions included in each group of character regions, an association relationship is established to obtain the multiple association relationships.
[0044] Optionally, the processing module is specifically used for:
[0045] Mapping each character area to a coordinate system established based on the passport image to be processed;
[0046] For the various association relationships, perform the following operations respectively:
[0047] Performing linear regression processing on the center coordinates of each of the multiple character regions corresponding to an association relationship to obtain a linear equation;
[0048] Determining the region coordinates of the corresponding text content region based on the coordinates of the intersections between the plurality of character regions and the straight line represented by the linear equation, the slope of the linear equation, and the region sizes of the plurality of character regions;
[0049] Based on the region coordinates, the text content region is extracted from the passport image to be processed.
[0050] Optionally, the processing module is further used to train the trained character detection model using the following method:
[0051] Acquire each initial sample data, wherein each initial sample data includes an initial sample image and a sample content area contained in the initial sample image;
[0052] Extracting the background image of each initial sample image;
[0053] Based on the pre-stored passport feature information, a plurality of random text contents are randomly generated;
[0054] Respectively synthesizing the plurality of random text contents and the respective background images to generate a plurality of newly added sample data;
[0055] Based on the initial sample data and the multiple newly added sample data, the character detection model to be trained is subjected to multiple rounds of iterative training until the training loss meets the training target, and the character detection model to be trained is output as the trained character detection model.
[0056] Optionally, the processing module is specifically used for:
[0057] Using a plurality of pre-stored pre-processing strategies, respectively adjusting the clarity of each of the initial sample images, and respectively obtaining each of the processed sample images;
[0058] Generate each processed sample data based on each processed sample image and a sample content area included in an initial sample image corresponding to each processed sample image;
[0059] Based on the processed sample data and the multiple newly added sample data, multiple rounds of iterative training are performed on the character detection model to be trained.
[0060] According to a third aspect, a computer program product is provided, comprising a computer program, which implements the method according to the first aspect when executed by a processor.
[0061] According to a fourth aspect, a computer device is provided, comprising:
[0062] A memory for storing program instructions;
[0063] The processor is used to call the program instructions stored in the memory and execute the method as described in the first aspect according to the obtained program instructions.
[0064] According to a fifth aspect, a computer-readable storage medium is provided, wherein the storage medium stores computer-executable instructions, and the computer-executable instructions are used to enable a computer to execute the method described in the first aspect.
[0065] In the embodiment of the present application, after the character regions of the characters contained in the passport image to be processed are detected by using the trained character detection model, multiple association relationships are established for each character region, and then multiple text content regions can be extracted from the passport image to be processed corresponding to the multiple association relationships. Therefore, the characters contained in the text content region have the same expression form.
[0066] Therefore, when using the trained text recognition model to identify the text content contained in the passport image to be processed, each text content area can be identified separately, avoiding the situation where some characters in multiple forms of expression in the passport image are ignored or mistakenly identified as characters in other forms of expression, thereby improving the accuracy of text content recognition in the passport image. BRIEF DESCRIPTION OF THE DRAWINGS
[0067] Figure 1 An application scenario of the method for processing a passport image provided in an embodiment of the present application;
[0068] Figure 2A flow chart of a method for processing a passport image provided in an embodiment of the present application Figure 1 ;
[0069] Figure 3 A flow chart of a method for processing a passport image provided in an embodiment of the present application Figure 2 ;
[0070] Figure 4 A schematic diagram of a structure of a device for processing a passport image provided in an embodiment of the present application Figure 1 ;
[0071] Figure 5 A schematic diagram of a structure of a device for processing a passport image provided in an embodiment of the present application Figure 2 . DETAILED DESCRIPTION
[0072] In order to make the purpose, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application.
[0073] It should be noted that in the embodiments of the present application, when the user data involved is applied to specific products or technologies, user permission or consent is required, and the collection, use and processing of the relevant data must comply with relevant laws, regulations and standards of relevant countries and regions.
[0074] The following is a brief introduction to the application fields of the method for processing passport images provided in the embodiments of the present application.
[0075] With the continuous development of technology, more and more devices can detect the characters contained in images and recognize the text content contained in images.
[0076] Traditional methods for detecting characters contained in images and identifying text content contained in images can only detect or identify characters in the same form of expression. For example, optical character recognition (OCR) technology first digitizes and stores the passport image to be processed to obtain a digitized passport image. After obtaining the digitized passport image, the characters in the passport image to be processed are identified based on the digitized passport image. Using OCR technology can avoid manually comparing the passport image to be processed and inputting text content, which greatly saves manpower and improves the efficiency of inputting text content.
[0077] Detecting the characters contained in an image is to capture the area containing the characters in the image. You can use models such as Yolo, CTPN, DBnet, PseNet, etc. You can also use the OpenCV algorithm library to use methods such as contour detection and connected domain analysis to extract image features, and then use the extracted features to detect the area containing the characters in the image. You can also automatically extract image features through a convolutional neural network, and determine the area containing the characters in the image based on the image features.
[0078] Identifying the text content contained in an image means identifying the text content in an area based on the area of characters contained in the image. The area of characters contained in the image can be segmented first. After segmentation, each area contains only one character. The characters obtained after segmentation are classified one by one using a machine learning algorithm to obtain the recognition result of a single character. Alternatively, the text content can be directly obtained based on the area of characters contained in the image through a text recognition model such as CRNN+CTC, CNN+Attention, SRN, and other models.
[0079] However, in some scenarios, such as passport images, an image contains characters in multiple forms, such as characters in multiple sizes, characters in multiple fonts, characters in multiple languages, characters in multiple text directions, etc. At the same time, the background image of the passport image is complex and can easily interfere with the character recognition process.
[0080] Under the related art, some characters in multiple forms of expression are easily ignored or mistakenly recognized as characters in other forms of expression, resulting in incorrect recognition of text content in passport images. At the same time, the amount of sample image data of passport images is small, so the accuracy of the model trained based on fewer sample images fluctuates.
[0081] It can be seen that under the related art, for passport images, the accuracy of the process of identifying the text content contained in the image is low.
[0082] In order to solve the problem of low accuracy in the process of identifying the text content contained in a passport image, the present application proposes a method for processing a passport image. In the method, after obtaining the passport image to be processed, a trained character detection model is used to detect the character regions of each character contained in the passport image to be processed based on the passport image to be processed. For each character region, multiple association relationships are established, and corresponding to the multiple association relationships, multiple text content regions are extracted from the passport image to be processed respectively. Each association relationship is associated with multiple character regions. The trained text recognition model is used to identify the text content contained in the passport image to be processed based on the multiple text content regions.
[0083] In the embodiment of the present application, after the character regions of the characters contained in the passport image to be processed are detected by using the trained character detection model, multiple association relationships are established for each character region, and then multiple text content regions can be extracted from the passport image to be processed corresponding to the multiple association relationships. Therefore, the characters contained in the text content region have the same expression form.
[0084] Therefore, when using the trained text recognition model to identify the text content contained in the passport image to be processed, each text content area can be identified separately, avoiding the situation where some characters in multiple forms of expression in the passport image are ignored or mistakenly identified as characters in other forms of expression, thereby improving the accuracy of text content recognition in the passport image.
[0085] The following describes the application scenarios of the method for processing passport images provided by this application.
[0086] Please refer to Figure 1 , is a schematic diagram of an application scenario of the method for processing passport images provided by the present application. The application scenario includes a client 101 and a server 102. The client 101 and the server 102 can communicate with each other. The communication method can be to communicate using wired communication technology, such as communicating by connecting a network cable or a serial port cable; or to communicate using wireless communication technology, such as communicating through Bluetooth or wireless fidelity (WIFI) and other technologies, without specific limitation.
[0087] The client 101 generally refers to a device that can provide a passport image to be processed, such as a terminal device, a third-party application program that can be accessed by the terminal device, or a webpage that can be accessed by the terminal device. The terminal device includes but is not limited to a mobile phone, a computer, an intelligent transportation device, an intelligent appliance, etc. The server 102 generally refers to a device that can process the passport image to be processed and obtain the text content contained in the passport image to be processed, such as a terminal device or a server. The server includes but is not limited to a cloud server, a local server, or an associated third-party server. Both the client 101 and the server 102 can use cloud computing to reduce the occupation of local computing resources; cloud storage can also be used to reduce the occupation of local storage resources.
[0088] As an embodiment, the client 101 and the server 102 may be the same device, and there is no specific limitation. In the embodiment of the present application, the client 101 and the server 102 are different devices as an example for introduction.
[0089] The following is based on Figure 1 , the method for processing passport images provided in the embodiment of the present application is specifically introduced. Please refer to Figure 2, is a flow chart of a method for processing a passport image provided in an embodiment of the present application.
[0090] S201, obtaining a passport image to be processed.
[0091] The client can take a photo of the passport through a camera to obtain the passport image to be processed. The client can send the passport image to be processed to the server, and the server can receive the passport image to be processed sent by the client. The server can also obtain multiple passport images to be processed from each client at a preset time period, and process each passport image to be processed in turn, etc., without specific restrictions.
[0092] S202, using the trained character detection model, based on the passport image to be processed, detecting the character regions of the respective characters contained in the passport image to be processed.
[0093] After obtaining the passport image to be processed, the server can use the trained character detection model to detect the character regions of each character contained in the passport image to be processed based on the passport image to be processed. The characters can be Chinese characters, Chinese words, foreign words or foreign letters, etc., without specific restrictions. The trained character detection model can obtain the sentence region of each sentence contained in the passport image to be processed in units of punctuation. After obtaining each sentence region, the server can use the trained character detection model to perform semantic segmentation on each sentence region to obtain a character heat map. Based on the character heat map, the character region of each character contained in the passport image to be processed can be obtained.
[0094] As an embodiment, the trained character detection model can be obtained by performing multiple rounds of iterative training on the character detection model to be trained based on each sample data, and when the training loss meets the training target, the character detection model to be trained is output as the trained character detection model. Each sample data includes a sample passport image and each sample character region included in the sample passport image.
[0095] The model structure of the character detection model can adopt the model structure of the PseNet model, etc., without specific limitation. The model parameters of the character detection model can adopt the model parameters obtained from network resources as the initial model parameters, and then perform model training based on the initial model parameters.
[0096] S203, establishing a plurality of association relationships for each character region, and extracting a plurality of text content regions from the passport image to be processed respectively corresponding to the plurality of association relationships.
[0097] After obtaining each character region included in the passport image to be processed, the server can establish multiple association relationships for each character region, and extract multiple text content regions from the passport image to be processed corresponding to the multiple association relationships, wherein each association relationship is associated with multiple character regions.
[0098] As an example, please refer to Figure 3 , an example is given to introduce the method for processing passport images provided in this application.
[0099] S301, the server may determine the character height difference between each two adjacent character regions and the character spacing between each two adjacent character regions based on the region size of each character region. For example, the difference in region height between each two adjacent character regions is used as the character height difference, and the distance between the center points of each two adjacent character regions is used as the character spacing.
[0100] After obtaining the height differences of each character and the spacings between each character, the server may execute S302 to select multiple groups of character areas from each character area based on the height differences of each character and the spacings between each character, and establish multiple association relationships respectively.
[0101] For example, the server may cluster each character region based on the height difference of each character and the spacing between each character to obtain multiple groups of character regions, wherein the height difference of the characters among the multiple character regions included in each group of character regions is less than the height threshold, and the difference between the character spacings is less than the spacing threshold. For the multiple character regions included in each group of character regions, an association relationship is established to obtain multiple association relationships.
[0102] For another example, the server can use each character area as a reference in turn, determine the best character area from other character areas, and establish an association relationship between the two character areas. The best character area is adjacent to the reference character area, and is located after the reference character area in word order, and the height difference is less than the height threshold. The next best character area is then used as a reference to continue to determine the next best character area. In addition to meeting the above requirements, the difference between the character spacing between the next best character area and the best character area, and the character spacing between the best character area and the reference character area, is less than the spacing threshold.
[0103] Based on the character areas determined by the above method, a variety of association relationships can be established. The characters in the character areas associated with each association relationship have the same font size, the same language, the same font, the same text direction, etc. Therefore, based on the association relationship, the characters associated with each association relationship can be identified more accurately, thereby improving the recognition accuracy.
[0104] As an embodiment, after establishing multiple associations, the server can continue to execute S303 to map each character area to a coordinate system established based on the passport image to be processed. For multiple associations, operations are performed separately. The operations performed by the server for multiple associations are similar. The following is an introduction taking one association as an example.
[0105] The server may continue to execute S304, perform linear regression processing on the center coordinates of each of the multiple character regions corresponding to an association relationship to obtain a linear equation. The server may continue to execute S305, determine the region coordinates of the corresponding text content region based on the intersection coordinates between the multiple character regions and the straight line represented by the linear equation, the slope of the linear equation, and the region sizes of the multiple character regions.
[0106] For example, the server may use the two coordinates with the farthest spacing among the intersection coordinates between multiple character areas and the straight line represented by the linear equation as the area width of the corresponding text content area; or the server may use the two coordinates with the farthest spacing among the intersection coordinates between the area center points of each of the multiple character areas and the straight line represented by the linear equation as the area width of the corresponding text content area, and so on, without specific limitation.
[0107] The server may use the slope of the linear equation as the area slope of the corresponding text content area, and use the average of the area heights of the multiple character areas as the area height of the corresponding text content area, thereby determining the area coordinates of the corresponding text content area.
[0108] After obtaining the region coordinates of the text content region, the server may proceed to execute S306 to extract the text content region from the passport image to be processed based on the region coordinates.
[0109] As an embodiment, since the amount of passport image data is small, the server can expand the amount of passport image data, and then train the character detection model to be trained to improve the detection accuracy of the trained character detection model. The expansion can use PIL, Opencv library and a large number of font ttf files, etc., without specific restrictions.
[0110] As an embodiment, when the real sample data is sufficient, the passport sample may not be expanded. When the real sample data is insufficient, the number of expanded samples is consistent with the number of real samples, and the sum of the two numbers reaches the number required for model training.
[0111] As an embodiment, the server may obtain each initial sample data, wherein each initial sample data includes an initial sample image and a sample content area contained in the initial sample image. The initial sample image contained in the initial sample data may be a real passport image, and image expansion is performed based on the real passport image.
[0112] The server can extract the background image of each initial sample image and randomly generate multiple random text contents based on the pre-stored passport feature information. After obtaining the background image, the server can randomly generate characters of the corresponding format according to the specified format of each character in the passport to obtain multiple random text contents matching the passport feature information.
[0113] The server can synthesize multiple random text contents and each background image to generate multiple new sample data. Based on each initial sample data and multiple new sample data, the character detection model to be trained is trained for multiple rounds until the training loss meets the training target, and the character detection model to be trained is output as the trained character detection model.
[0114] As an embodiment, after obtaining the initial sample images, the server can use a plurality of pre-stored pre-processing strategies to adjust the clarity of each initial sample image and obtain each processed sample image. Thus, it is possible to simulate passport images obtained under different shooting environments and further improve the accuracy of the trained character detection model.
[0115] The various preprocessing strategies include using a bilinear interpolation algorithm to blur the initial sample image; using a median filter to blur the initial sample image; using Gaussian noise or salt and pepper noise to add noise to the initial sample image; changing the color of characters contained in the initial sample image; modifying the contrast, saturation, brightness, sharpness, etc. of the initial sample image; randomly cropping the initial sample image; and performing a perspective transformation on the initial sample image, wherein the parameter matrix of the perspective transformation is randomly generated within a reasonable range to achieve the purpose of simulating the perspective difference in reality.
[0116] After obtaining each processed sample image, the server can generate each processed sample data based on each processed sample image and the sample content area contained in the initial sample image corresponding to each processed sample image. Based on each processed sample data and multiple newly added sample data, the character detection model to be trained is trained for multiple rounds of iterations.
[0117] S204, using the trained text recognition model, respectively based on a plurality of text content regions, identifying text content contained in the passport image to be processed.
[0118] After obtaining the multiple text content regions, the server may use the trained text recognition model to recognize the text content contained in the passport image to be processed based on the multiple text content regions respectively.
[0119] As an embodiment, the model training deployment can apply the TensorFlow framework and the Pytorch framework, and the client can apply the Tornado framework to obtain the passport image to be processed.
[0120] The model structure of the text recognition model can adopt the model structure of the SRN model, etc., without specific limitation. The model parameters of the text recognition model can adopt the model parameters obtained from network resources as initial model parameters, and then perform model training based on the initial model parameters.
[0121] By expanding passport images and annotations, the problem of insufficient samples is solved on the one hand, and the generated samples do not need to be manually annotated with text positions and content, which greatly saves manpower and time. Using character detection models, such as the PseNet model, to detect character areas, the detection accuracy and recall rate of dense characters are higher than models that directly detect text content areas, such as Yolo and CTPN. Through text recognition models, such as SRN, the direct recognition of text line images avoids the defect of traditional character recognition methods that require character cutting, and both efficiency and accuracy are improved.
[0122] Based on the same inventive concept, the embodiment of the present application provides a device for processing a passport image, which can realize the functions corresponding to the aforementioned method for processing a passport image. Figure 4 , the device includes an acquisition module 401 and a processing module 402, wherein:
[0123] Acquisition module 401: used to obtain the passport image to be processed;
[0124] Processing module 402: used to detect the character regions of the characters contained in the passport image to be processed by using the trained character detection model based on the passport image to be processed;
[0125] The processing module 402 is further used to: establish multiple association relationships for each character region, and extract multiple text content regions from the passport image to be processed corresponding to the multiple association relationships, wherein each association relationship is associated with multiple character regions;
[0126] The processing module 402 is further configured to: use a trained text recognition model to identify text content contained in the passport image to be processed based on a plurality of text content regions.
[0127] In a possible embodiment, the processing module 402 is specifically configured to:
[0128] Determine the character height difference between every two adjacent character areas and the character spacing between every two adjacent character areas based on the area size of each character area;
[0129] Based on the height difference of each character and the distance between each character, multiple groups of character areas are selected from each character area, and multiple association relationships are established respectively.
[0130] In a possible embodiment, the processing module 402 is specifically configured to:
[0131] Based on the height difference of each character and the spacing between each character, clustering processing is performed on each character region to obtain multiple groups of character regions, wherein the height difference of the characters among the multiple character regions included in each group of character regions is less than the height threshold, and the difference between the character spacings is less than the spacing threshold;
[0132] For multiple character regions included in each group of character regions, an association relationship is established to obtain multiple association relationships.
[0133] In a possible embodiment, the processing module 402 is specifically configured to:
[0134] Mapping each character area to a coordinate system established based on the passport image to be processed;
[0135] For various relationships, perform the following operations:
[0136] Performing linear regression processing on the center coordinates of each of the multiple character regions corresponding to an association relationship to obtain a linear equation;
[0137] Determining the region coordinates of the corresponding text content region based on the coordinates of the intersections between the plurality of character regions and the straight line represented by the linear equation, the slope of the linear equation, and the region sizes of the plurality of character regions;
[0138] Based on the region coordinates, the text content region is extracted from the passport image to be processed.
[0139] In a possible embodiment, the processing module 402 is further configured to train the trained character detection model using the following method:
[0140] Acquire each initial sample data, wherein each initial sample data includes an initial sample image and a sample content area contained in the initial sample image;
[0141] Extracting the background image of each initial sample image;
[0142] Based on the pre-stored passport feature information, a plurality of random text contents are randomly generated;
[0143] Synthesize multiple random text contents and each background image respectively to generate multiple new sample data;
[0144] Based on each initial sample data and multiple newly added sample data, the character detection model to be trained is trained for multiple rounds of iterative training until the training loss meets the training target, and the character detection model to be trained is output as the trained character detection model.
[0145] In a possible embodiment, the processing module 402 is specifically configured to:
[0146] Using a plurality of pre-stored pre-processing strategies, the clarity of each initial sample image is adjusted respectively, and each processed sample image is obtained respectively;
[0147] Generate each processed sample data based on each processed sample image and a sample content area included in the initial sample image corresponding to each processed sample image;
[0148] Based on each processed sample data and multiple newly added sample data, multiple rounds of iterative training are performed on the character detection model to be trained.
[0149] Please refer to Figure 5 The above-mentioned passport image processing apparatus can be run on a computer device 500, and the current version and historical version of the data storage program and the application software corresponding to the data storage program can be installed on the computer device 500, and the computer device 500 includes a processor 580 and a memory 520. In some embodiments, the computer device 500 may include a display unit 540, and the display unit 540 includes a display panel 541 for displaying an interactive operation interface by a user.
[0150] In a possible embodiment, the display panel 541 may be configured in the form of a liquid crystal display (LCD) or an organic light-emitting diode (OLED).
[0151] The processor 580 is used to read the computer program and then execute the method defined by the computer program. For example, the processor 580 reads the data storage program or file, so as to run the data storage program on the computer device 500 and display the corresponding interface on the display unit 540. The processor 580 may include one or more general-purpose processors and may also include one or more DSPs (Digital Signal Processors) to perform related operations to implement the technical solutions provided in the embodiments of the present application.
[0152] The memory 520 generally includes internal memory and external memory, and the internal memory can be a random access memory (RAM), a read-only memory (ROM), and a cache (CACHE), etc. The external memory can be a hard disk, an optical disk, a USB disk, a floppy disk, or a tape drive, etc. The memory 520 is used to store computer programs and other data, and the computer program includes applications corresponding to each client, etc. Other data may include data generated after the operating system or application is run, and the data includes system data (such as configuration parameters of the operating system) and user data. In the embodiment of the present application, program instructions are stored in the memory 520, and the processor 580 executes the program instructions in the memory 520 to implement any of the methods discussed in the previous figure.
[0153] The display unit 540 is used to receive input digital information, character information or contact touch operation / contactless gesture, and generate signal input related to user settings and function control of the computer device 500. Specifically, in the embodiment of the present application, the display unit 540 may include a display panel 541. The display panel 541 is, for example, a touch screen, which can collect user touch operations on or near it (such as operations performed by the user using fingers, stylus, or any other suitable object or accessory on or on the display panel 541), and drive corresponding connection devices according to a pre-set program.
[0154] In a possible embodiment, the display panel 541 may include two parts: a touch detection device and a touch controller. The touch detection device detects the touch position of the player, detects the signal brought by the touch operation, and transmits the signal to the touch controller; the touch controller receives the touch information from the touch detection device, converts it into the touch point coordinates, and then sends it to the processor 580, and can receive and execute the command sent by the processor 580.
[0155] The display panel 541 may be implemented in various types such as resistive, capacitive, infrared, and surface acoustic wave. In addition to the display unit 540, in some embodiments, the computer device 500 may further include an input unit 530, which may include an image input device 531 and other input devices 532, wherein the other input devices may include, but are not limited to, one or more of a physical keyboard, a function key (such as a volume control key, a switch key, etc.), a trackball, a mouse, a joystick, etc.
[0156] In addition to the above, the computer device 500 may also include a power supply 590 for supplying power to other modules, an audio circuit 560, a near field communication module 570, and an RF circuit 510. The computer device 500 may also include one or more sensors 550, such as an acceleration sensor, a light sensor, a pressure sensor, etc. The audio circuit 560 specifically includes a speaker 561 and a microphone 562, etc. For example, the computer device 500 can collect the user's voice through the microphone 562 and perform corresponding operations, etc.
[0157] As an embodiment, the number of processors 580 may be one or more, and the processor 580 and the memory 520 may be coupled or relatively independently configured.
[0158] As an example, Figure 5 The processor 580 in the embodiment may be used to implement the following Figure 4 The functions of the acquisition module 401 and the processing module 402 in.
[0159] As an example, Figure 5 The processor 580 in can be used to implement the corresponding functions of the server or terminal device discussed above.
[0160] A person skilled in the art can understand that: all or part of the steps of implementing the above method embodiment can be completed by hardware related to program instructions, and the aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it executes the steps of the above method embodiment; and the aforementioned storage medium includes: mobile storage devices, read-only memories (ROM, Read-Only Memory), random access memories (RAM, Random Access Memory), disks or optical disks, etc. Various media that can store program codes.
[0161] Alternatively, if the above-mentioned integrated unit of the present invention is implemented in the form of a software function module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiment of the present invention can be essentially or partly reflected in the form of a software product, for example, through a computer program product, which is stored in a storage medium and includes a number of instructions for a computer device to execute all or part of the methods described in each embodiment of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as mobile storage devices, ROM, RAM, magnetic disks or optical disks.
[0162] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.
Claims
1. A method for processing a passport image, characterized in that: include: Obtaining passport images to be processed; Using a trained character detection model, based on the passport image to be processed, detect the character regions of the respective characters contained in the passport image to be processed; Based on the area sizes of the respective character areas, determining the character height difference between every two adjacent character areas and the character spacing between every two adjacent character areas; Based on the height difference of each character and the spacing between each character, clustering the character regions to obtain multiple groups of character regions, wherein the height difference of the characters among the multiple character regions included in each group of character regions is less than a height threshold, and the difference between the character spacings is less than a spacing threshold; For multiple character regions included in each group of character regions, an association relationship is established to obtain multiple association relationships; Corresponding to the multiple association relationships, extracting multiple text content regions from the passport image to be processed respectively, wherein each association relationship is associated with multiple character regions; The trained text recognition model is used to identify the text content contained in the passport image to be processed based on the multiple text content areas.
2. The method according to claim 1, characterized in that Corresponding to the multiple association relationships, a plurality of text content regions are extracted from the passport image to be processed respectively, including: Mapping each character area to a coordinate system established based on the passport image to be processed; For the various association relationships, perform the following operations respectively: Performing linear regression processing on the center coordinates of each of the multiple character regions corresponding to an association relationship to obtain a linear equation; Determining the region coordinates of the corresponding text content region based on the coordinates of the intersections between the plurality of character regions and the straight line represented by the linear equation, the slope of the linear equation, and the region sizes of the plurality of character regions; Based on the region coordinates, the text content region is extracted from the passport image to be processed.
3. The method according to claim 1 or 2, characterized in that: The trained character detection model is trained using the following method: Acquire each initial sample data, wherein each initial sample data includes an initial sample image and a sample content area contained in the initial sample image; Extracting the background image of each initial sample image; Based on the pre-stored passport feature information, a plurality of random text contents are randomly generated; Respectively synthesizing the plurality of random text contents and the respective background images to generate a plurality of newly added sample data; Based on the initial sample data and the multiple newly added sample data, the character detection model to be trained is subjected to multiple rounds of iterative training until the training loss meets the training target, and the character detection model to be trained is output as the trained character detection model.
4. The method according to claim 3, characterized in that Based on the initial sample data and the multiple newly added sample data, the character detection model to be trained is subjected to multiple rounds of iterative training, including: Using a plurality of pre-stored pre-processing strategies, respectively adjusting the clarity of each of the initial sample images, and respectively obtaining each of the processed sample images; Generate each processed sample data based on each processed sample image and a sample content area included in an initial sample image corresponding to each processed sample image; Based on the processed sample data and the multiple newly added sample data, multiple rounds of iterative training are performed on the character detection model to be trained.
5. A device for processing a passport image, characterized in that: include: Acquisition module: used to obtain the passport image to be processed; Processing module: used for detecting the character regions of the respective characters contained in the passport image to be processed by using the trained character detection model based on the passport image to be processed; The processing module is further used for: determining the character height difference between each two adjacent character regions and the character spacing between each two adjacent character regions based on the respective region sizes of the respective character regions; performing clustering processing on the respective character regions based on the respective character height differences and the respective character spacings to obtain multiple groups of character regions; establishing an association relationship for multiple character regions included in each group of character regions to obtain multiple association relationships; corresponding to the multiple association relationships, extracting multiple text content regions from the passport image to be processed respectively, wherein each association relationship is associated with multiple character regions; wherein, between the multiple character regions included in each group of character regions, the character height difference is less than a height threshold, and the difference in character spacing is less than a spacing threshold; The processing module is further used to: use a trained text recognition model to identify text content contained in the passport image to be processed based on the multiple text content areas respectively.
6. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to claims 1 to 4 are implemented.
7. A computer device, characterized in that: include: A memory for storing program instructions; The processor is used to call the program instructions stored in the memory, and execute the method according to any one of claims 1 to 4 according to the obtained program instructions.
8. A computer-readable storage medium, characterized in that: The storage medium stores computer-executable instructions, and the computer-executable instructions are used to enable a computer to execute the method according to any one of claims 1 to 4.
