Text recognition method, device, equipment and storage medium

Abstract

The present application discloses a text recognition method, device, equipment and storage medium, belonging to the field of artificial intelligence technology. The method comprises: annotating a target type of text image to be tested to obtain a candidate text image, wherein the candidate text image comprises a plurality of detection text boxes, each detection text box comprises a plurality of texts in the text image to be tested whose distances from each other are less than a preset distance threshold; obtaining a template image corresponding to the target type, wherein the template image comprises a plurality of standard text boxes, and different standard text boxes are used to indicate the area where texts containing different types of information are located in the sample text image of the target type; matching the detection text boxes in the candidate text image and the standard text boxes in the template image; extracting the target type of information from the candidate text image according to the result of the matching process. The technical solution provided in the embodiment of the present application can improve the speed of extracting key information of the text.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to a text recognition method, device, equipment and storage medium. Background Art

[0002] Text recognition technology has great applications. Currently, we can obtain different types of information contained in the text by recognizing the text.

[0003] In the related technology, text recognition can be performed through the template matching method. Specifically, a keyword image can be designed first, and the keyword image can be used to slide continuously on the image to be tested. After each slide, the similarities of the pixels at all corresponding positions on the image to be tested and the template image are summed. When the sum of the pixel similarities is the largest, the match is successful, and finally text recognition is performed on the result of the successful match.

[0004] However, this method performs pixel-level comparison and matching, which reduces the speed of text recognition. Summary of the invention

[0005] Based on this, the embodiments of the present application provide a method, apparatus, device and storage medium for text recognition, which can improve the speed of extracting key information from text.

[0006] In a first aspect, a method for text recognition is provided, the method comprising:

[0007] The target type of text image to be tested is annotated to obtain a candidate text image, wherein the candidate text image includes a plurality of detection text boxes, each detection text box includes a plurality of characters in the text image to be tested whose distances from each other are less than a preset distance threshold; a template image corresponding to the target type is obtained, wherein the template image includes a plurality of standard text boxes, and different standard text boxes are used to indicate the area where text containing different types of information is located in the sample text image of the target type; matching processing is performed on the detection text boxes in the candidate text image and the standard text boxes in the template image; and according to the result of the matching processing, information of the target type is extracted from the candidate text image.

[0008] In one embodiment, matching the detected text box in the candidate text image with the standard text box in the template image includes:

[0009] The size of the template image is adjusted based on the size of the candidate text image to obtain an adjusted template image, wherein the adjusted template image includes a plurality of adjustment standard text boxes; and the detection text box in the candidate text image and the adjustment standard text box in the adjustment template image are matched.

[0010] In one embodiment, adjusting the size of the template image based on the size of the candidate text image includes:

[0011] Calculate the sum of the lengths of each detected text box in the candidate text image to obtain a first length sum value; calculate the sum of the widths of each detected text box in the candidate text image to obtain a first width sum value; calculate the sum of the lengths of each standard text box in the template image to obtain a second length sum value; calculate the sum of the widths of each standard text box in the template image to obtain a second width sum value; adjust the length of the template image based on the ratio of the first length sum value to the second length sum value, and adjust the width of the template image based on the ratio of the first width sum value to the second width sum value.

[0012] In one embodiment, matching the detected text box in the candidate text image with the standard text box in the template image includes:

[0013] The template text image is superimposed on the candidate image and slid; after each slide, the sum of the intersection and union ratios of each detection text box in the candidate text image and each standard text box in the template image is calculated; the sliding position of the candidate text image corresponding to the calculated maximum sum of intersection and union ratios relative to the template image is taken as the final relative position; based on the final relative position, the detection text box in the candidate text image and the standard text box in the template image are matched.

[0014] In one embodiment, calculating the sum of intersection-and-union ratios of each detected text box in the candidate text image and each standard text box in the template image includes:

[0015] For each detected text box in the candidate text image, the intersection-and-union ratio between the detected text box and each standard text box in the template image is calculated, and the maximum intersection-and-union ratio calculated is used as the candidate intersection-and-union ratio corresponding to the detected text box; the sum of the candidate intersection-and-union ratios corresponding to each detected text box in the candidate text image is calculated.

[0016] In one embodiment, matching the detected text box in the candidate text image with the standard text box in the template image based on the final relative position includes:

[0017] For each detected text box in the candidate text image located at the final relative position, the intersection-and-union ratio of the detected text box and each standard text box in the template image located at the final relative position is calculated, and the standard text box corresponding to the calculated maximum intersection-and-union ratio is used as the standard text box matching the detected text box.

[0018] In one embodiment, matching the detected text box in the candidate text image with the standard text box in the template image includes:

[0019] Based on the position of each detection text box in the candidate text image, the extreme points of the detection text box in the candidate text image are determined, based on the extreme points of the detection text box in the candidate text image, a first coordinate system is established in the candidate text image, and the coordinates of each detection text box in the first coordinate system are obtained; based on the position of each standard text box in the template image, the extreme points of the standard text box in the template image are determined, based on the extreme points of the standard text box in the template image, a second coordinate system is established in the template image, and the coordinates of each standard text box in the second coordinate system are obtained; based on the coordinates of each detection text box in the first coordinate system and the coordinates of each standard text box in the second coordinate system, the detection text boxes in the candidate text image and the standard text boxes in the template image are matched.

[0020] In one embodiment, matching the detected text boxes in the candidate text image with the standard text boxes in the template image based on the coordinates of each detected text box in the first coordinate system and the coordinates of each standard text box in the second coordinate system includes:

[0021] For each detected text box in the candidate text image, the projected position of the detected text box in the second coordinate system is determined based on the coordinates of the detected text box in the first coordinate system, and the intersection and union ratios of the detected text box and each standard text box are calculated when the detected text box is located at the projected position. The standard text box corresponding to the calculated maximum intersection and union ratio is used as the standard text box that matches the detected text box.

[0022] In one embodiment, after matching the detection text box in the candidate text image with the standard text box in the template image, the method further includes:

[0023] Detect whether there is an empty standard text box in the template image, and the empty standard text box does not match any detection text box in the candidate text image; if there is an empty standard text box in the template image, intercept the missed detection area in the candidate text image based on the position of the empty standard text box; annotate the missed detection area, and obtain the detection text box corresponding to the empty standard text box through the annotation process.

[0024] In one embodiment, before obtaining the template image corresponding to the target type, the method further includes:

[0025] Acquire multiple sample text images of the target type; perform annotation processing on each sample text image to obtain a text box set corresponding to each sample text image, wherein the text box set includes multiple candidate text boxes, and each candidate text box includes multiple characters in the corresponding sample text image whose distances from each other are less than a preset distance threshold; for each information type contained in the sample text image, obtain the candidate text box corresponding to the information type from each text box set, and calculate the size mean and position mean of the obtained candidate text box; generate a standard text box corresponding to each information type based on the size mean and position mean of the candidate text box corresponding to each information type; and obtain a template image based on each generated standard text box.

[0026] In one embodiment, according to the result of the matching process, information of the target type is extracted from the candidate text image, including:

[0027] A standard text box is obtained from a template image to indicate an area where text containing information of a target type is located; a detection text box corresponding to the standard text box is determined, and text is extracted from the determined detection text box; and recognition processing is performed on the extracted text to obtain information of the target type.

[0028] In a second aspect, a device for text recognition is provided, the device for text recognition comprising:

[0029] A labeling module is used to label the target type of the to-be-tested text image to obtain a candidate text image, wherein the candidate text image includes a plurality of detection text frames, each detection text frame includes a plurality of characters in the to-be-tested text image whose distances from each other are less than a preset distance threshold;

[0030] An acquisition module, used to acquire a template image corresponding to a target type, the template image including a plurality of standard text boxes, each of which is used to indicate an area in a sample text image of the target type where text containing different types of information is located;

[0031] A matching module is used to match the detection text box in the candidate text image with the standard text box in the template image;

[0032] The extraction module is used to extract target type information from the text image to be tested according to the result of the matching process.

[0033] In one of the embodiments, the device further includes a first adjustment processing unit and a first matching unit;

[0034] The first adjustment processing unit is used to adjust the size of the template image based on the size of the candidate text image to obtain an adjusted template image, wherein the adjusted template image includes a plurality of adjustment standard text boxes;

[0035] The first matching unit is used to match the detection text box in the candidate text image with the adjustment standard text box in the adjustment template image.

[0036] In one of the embodiments, the device further includes a first calculation unit and a second adjustment processing unit;

[0037] The first calculation unit is used to calculate the sum of the lengths of each detection text box in the candidate text picture to obtain a first length sum value; calculate the sum of the widths of each detection text box in the candidate text picture to obtain a first width sum value; calculate the sum of the lengths of each standard text box in the template picture to obtain a second length sum value; calculate the sum of the widths of each standard text box in the template picture to obtain a second width sum value;

[0038] The second adjustment processing unit is used to adjust the length of the template image based on the ratio of the first length sum value to the second length sum value, and to adjust the width of the template image based on the ratio of the first width sum value to the second width sum value.

[0039] In one of the embodiments, the device further includes a sliding unit, a second calculating unit, and a third matching unit;

[0040] The sliding unit is used to slide the template text image over the candidate image;

[0041] The second calculation unit is used to calculate the sum of intersection-and-union ratios of each detection text box in the candidate text picture and each standard text box in the template picture after each sliding; and use the sliding position of the candidate text picture corresponding to the calculated maximum sum of intersection-and-union ratios relative to the template picture as the final relative position;

[0042] The third matching unit is used to match the detection text box in the candidate text image with the standard text box in the template image based on the final relative position.

[0043] In one of the embodiments, the apparatus further comprises a third computing unit and a fourth computing unit;

[0044] The third calculation unit is used to calculate, for each detection text box in the candidate text picture, an intersection-and-union ratio between the detection text box and each standard text box in the template picture, and use the calculated maximum intersection-and-union ratio as a candidate intersection-and-union ratio corresponding to the detection text box;

[0045] The fourth calculation unit is used to calculate the sum of candidate intersection-and-union ratios corresponding to each detected text box in the candidate text image.

[0046] In one of the embodiments, the apparatus further comprises a fifth computing unit;

[0047] The fifth calculation unit is used to calculate the intersection and union ratio of each detection text box in the candidate text image located at the final relative position and each standard text box in the template image located at the final relative position, and use the standard text box corresponding to the calculated maximum intersection and union ratio as the standard text box that matches the detection text box.

[0048] In one of the embodiments, the device further includes a second acquisition unit, an establishment unit, and a fourth matching unit;

[0049] The second acquisition unit is used to determine the extreme points of the detection text boxes in the candidate text picture based on the positions of the detection text boxes in the candidate text picture, establish a first coordinate system in the candidate text picture based on the extreme points of the detection text boxes in the candidate text picture, and acquire the coordinates of each detection text box in the first coordinate system;

[0050] The establishing unit is used to determine the extreme points of the standard text boxes in the template image based on the positions of the standard text boxes in the template image, establish a second coordinate system in the template image based on the extreme points of the standard text boxes in the template image, and obtain the coordinates of each standard text box in the second coordinate system;

[0051] The fourth matching unit is used to match the detection text box in the candidate text image with the standard text box in the template image based on the coordinates of each detection text box in the first coordinate system and the coordinates of each standard text box in the second coordinate system.

[0052] In one of the embodiments, the apparatus further comprises a sixth computing unit;

[0053] The sixth calculation unit is used to determine, for each detection text box in the candidate text image, a projected position of the detection text box in the second coordinate system based on the coordinates of the detection text box in the first coordinate system, and calculate the intersection and union ratio of the detection text box with each standard text box when the detection text box is located at the projected position, and use the standard text box corresponding to the calculated maximum intersection and union ratio as the standard text box that matches the detection text box.

[0054] In one of the embodiments, the device further includes a detection unit, a capture unit, and a first labeling unit;

[0055] The detection unit is used to detect whether there is an empty standard text box in the template image, and the empty standard text box does not match any detection text box in the candidate text image;

[0056] The interception unit is used to intercept the missed detection area in the candidate text image based on the position of the empty standard text box if there is an empty standard text box in the template image;

[0057] The first labeling unit is used to label the missed detection area, and obtain a detection text box corresponding to the empty standard text box through the labeling process.

[0058] In one of the embodiments, the apparatus further includes a third acquisition unit, a second labeling unit, a seventh calculation unit, and a generation unit;

[0059] The third acquisition unit is used to acquire multiple sample text images of the target type;

[0060] The second annotation unit is used to perform annotation processing on each sample text image to obtain a text box set corresponding to each sample text image, the text box set includes multiple candidate text boxes, and each candidate text box includes multiple characters in the corresponding sample text image whose distances from each other are less than a preset distance threshold;

[0061] The seventh calculation unit is used to obtain, for each information type contained in the sample text image, candidate text boxes corresponding to the information type from each text box set, and calculate the size average and position average of the obtained candidate text boxes;

[0062] The generating unit is used to generate a standard text box corresponding to each information type based on the size average and position average of the candidate text boxes corresponding to each information type; and obtain a template image based on the generated standard text boxes.

[0063] In one of the embodiments, the apparatus further includes a fourth acquisition unit, a determination unit, and an identification unit;

[0064] The fourth acquisition unit is used to acquire, from the template image, a standard text box for indicating an area where text containing information of the target type is located;

[0065] The determination unit is used to determine a detection text box corresponding to the standard text box, and extract text from the determined detection text box;

[0066] The recognition unit is used to perform recognition processing on the extracted text to obtain information of the target type.

[0067] In a third aspect, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, any text recognition method as described in the first aspect is implemented.

[0068] In a fourth aspect, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, any text recognition method as described in the first aspect is implemented.

[0069] The beneficial effects brought by the technical solution provided by the embodiment of the present application include at least:

[0070] First, a target type of text image to be tested is annotated to obtain a candidate text image, wherein the candidate text image includes multiple detection text boxes, each detection text box includes multiple characters in the text image to be tested whose distances from each other are less than a preset distance threshold. Then, a template image corresponding to the target type is obtained, wherein the template image includes multiple standard text boxes, and different standard text boxes are used to indicate areas in the sample text image of the target type where texts containing different types of information are located. Then, the detection text boxes in the candidate text image and the standard text boxes in the template image are matched. Finally, according to the result of the matching, the information of the target type is extracted from the candidate text image. In the technical solution provided in the embodiment of the present application, since the candidate text image contains multiple detection text boxes and the template image contains multiple standard text boxes, the matching process is not pixel-level matching, but matching of two text boxes. This can save time in extracting key text information and improve the speed of extracting key text information. BRIEF DESCRIPTION OF THE DRAWINGS

[0071] Figure 1 A block diagram of a server involved in an embodiment of the present application;

[0072] Figure 2 A flowchart of a text recognition method provided in an embodiment of the present application;

[0073] Figure 3 A schematic diagram of an original image of a driving license provided in an embodiment of the present application;

[0074] Figure 4 A schematic diagram of the positions of various detection text boxes in a candidate text image of a driving license provided in an embodiment of the present application;

[0075] Figure 5 A schematic diagram of a driving license template image provided in an embodiment of the present application;

[0076] Figure 6 A schematic diagram of a driving license template matching result provided in an embodiment of the present application;

[0077] Figure 7 A flowchart of a sliding matching process of a detection text box in a candidate text picture and a standard text box in a template picture provided in an embodiment of the present application;

[0078] Figure 8 A schematic diagram of calculating the intersection-and-union ratio of a detection text box and each standard text box in a template image provided in an embodiment of the present application;

[0079] Fig. 9A flowchart of a coordinate normalization matching process for a detection text box in a candidate text image and a standard text box in a template image provided in an embodiment of the present application;

[0080] Fig.10 A schematic diagram of an original ID card image provided in an embodiment of the present application;

[0081] Fig.11 A schematic diagram of the positions of various detection text boxes in an ID card candidate text image provided in an embodiment of the present application;

[0082] Fig.12 A schematic diagram of an ID card template image provided in an embodiment of the present application;

[0083] Fig.13 A schematic diagram of an identity card template matching result provided in an embodiment of the present application;

[0084] Fig.14 A flowchart of a method for extracting target type information provided in an embodiment of the present application;

[0085] Fig.15 A flowchart of a text recognition device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0086] In order to make the objectives, technical solutions and advantages of the present application clearer, the implementation methods of the present application will be further described in detail below with reference to the accompanying drawings.

[0087] Currently, text recognition technology is widely used in fields such as license plate recognition, ID card recognition, and express delivery order recognition, and can obtain different types of information contained in the text by recognizing the text.

[0088] In the related art, text recognition can be performed by a template matching method. Specifically, a keyword image can be designed first, and the keyword image can be slid from a specified starting position. After each slide, the similarity of the pixels at all corresponding positions on the image to be tested and the template image is summed. When the sum of the similarity of the pixels is the largest, the match is successful. Finally, text recognition is performed on the result after the successful match. For example, for a given image to be tested that is an ID card image, and the designed keyword image is a name image, then the name image needs to be slid continuously on the ID card image. The upper left corner position can be specified as the starting position, and the image to be tested can be slid downward or rightward according to a certain sliding step length. After each slide, the similarity of the pixels at all corresponding positions on the ID card image and the name image is summed. When the sum of the similarity of the pixels is the largest, it means that the match is successful, and the area where the name is located on the ID card image can be determined, and the name information can be finally extracted.

[0089] However, this method is to compare and match at the pixel level, that is, it is necessary to calculate the similarity of pixels at all corresponding positions on the image to be tested and the keyword image, and then sum the similarities of the pixels. When the similarity of the pixels is the largest, the match is successful. Since the similarity needs to be calculated for all pixels, the number of pixels on the image is very large, which reduces the speed of text recognition.

[0090] In view of this, an embodiment of the present application provides a text recognition method, in which a target type of text image to be tested is first annotated to obtain a candidate text image, the candidate text image includes multiple detection text boxes, each detection text box includes multiple characters in the text image to be tested whose distances from each other are less than a preset distance threshold, then a template image corresponding to the target type is obtained, the template image includes multiple standard text boxes, and different standard text boxes are used to indicate the area where text containing different types of information is located in the sample text image of the target type, then the detection text boxes in the candidate text image and the standard text boxes in the template image are matched, and finally, according to the result of the matching process, the information of the target type is extracted from the candidate text image. Since in the technical solution provided in the embodiment of the present application, the candidate text image contains multiple detection text boxes and the template image contains multiple standard text boxes, the matching process is not pixel-level matching, but matching of two text boxes, which can save time for extracting key text information and improve the speed of extracting key text information.

[0091] The text recognition method provided in the embodiment of the present application can be applied to a server, wherein the server can be a single server or a server cluster composed of multiple servers, and the embodiment of the present application does not make any specific limitation on this.

[0092] Please refer to Figure 1 , which shows a block diagram of a server provided by an embodiment of the present application, such as Figure 1 As shown, the server may include a processor and a memory connected via a system bus. The processor of the server is used to provide computing and control capabilities. The memory of the server includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. When the computer program is executed by the processor, a text recognition method is implemented.

[0093] Those skilled in the art will understand that Figure 1 The structure shown in the figure is only a block diagram of a partial structure related to the solution of the present application, and does not constitute a limitation on the server to which the solution of the present application is applied. Specifically, the server may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0094] Please refer to Figure 2 , which shows a flowchart of a text recognition method provided by an embodiment of the present application, and the text recognition method can be applied to a server. Figure 2 As shown, the text recognition method may include the following steps:

[0095] Step 201: The server performs labeling processing on the target type of the text image to be tested to obtain a candidate text image.

[0096] The text image to be tested is an image containing multiple texts, and the target type is a type with a specific format, which may include an image type or a form type, for example, a common ID card form type, a driver's license form type, and the like.

[0097] The process of annotating the target type of text image to be tested can be to input the acquired text image to be tested into a model. After calculation by the trained model, the model outputs the coordinates and size of each detection text box, and annotates the text image to be tested based on the coordinates and size, and finally obtains the candidate text image.

[0098] Below, the present application embodiment will briefly describe the training process of the text detection model based on deep learning described above:

[0099] Specifically, when obtaining a text detection model based on deep learning, a large number of sample images need to be obtained. For example, the sample image can be a driving license image. Then, text boxes are used to mark all the text in all the sample images, and multiple texts whose distances are less than a preset distance threshold are marked in a text box. For example, the preset distance threshold can be the length of a character. Then, when multiple texts whose distances are less than the length of a character are marked in the same text box, the annotated sample images are used as input to train the text detection model based on deep learning. After each training, the coordinates and sizes of each detection text box are output. Finally, the text image to be tested can be annotated based on the coordinates and sizes to obtain candidate text images.

[0100] After the deep learning-based text detection model is trained, the text image to be tested is taken as input, and the text image to be tested can be annotated to obtain a candidate text image, and the candidate text image includes multiple detection text boxes, such as Figure 3 As shown, Figure 3 The original image of the driving license, such as Figure 4 As shown, Figure 4 Schematic diagram of the positions of the detected text boxes in the candidate text image of the driving license.

[0101] Step 202: The server obtains a template image corresponding to the target type, where the template image includes a plurality of standard text boxes.

[0102] Among them, each standard text box is used to indicate the area where text containing different types of information is located in the sample text image of the target type.

[0103] In this embodiment, the target type is the target text type form in step 201 above, and the template image is an image containing multiple standard text boxes, each of which indicates the area where the text of different types of information in the sample text image is located, such as Figure 5 As shown, Figure 5 This is a driving license template image, which includes multiple standard text boxes, and each text box indicates the area where different types of information such as name, gender, and place of origin are located.

[0104] Below, the present application embodiment will briefly describe the process of obtaining the template image described above:

[0105] Among them, the template image includes multiple generated standard text boxes. Specifically, multiple sample text images of the target type are obtained, and each sample text image is annotated to obtain a text box set corresponding to each sample text image. The text box set includes multiple candidate text boxes, and each candidate text box includes multiple texts in the corresponding sample text image whose distances from each other are less than a preset distance threshold. For each information type contained in the sample text image, the candidate text box corresponding to the information type is obtained from each text box set, and the size mean and position mean of the obtained candidate text box are calculated. Based on the size mean and position mean of the candidate text box corresponding to each information type, a standard text box corresponding to each information type is generated, and the template image is obtained based on each generated standard text box.

[0106] Step 203: The server performs matching processing on the detection text box in the candidate text image and the standard text box in the template image.

[0107] In this embodiment, the detection text boxes in the candidate text image and the standard text boxes in the template image are matched. For example, if there are 32 detection text boxes in the candidate text image and 32 standard text boxes in the template image, then the 32 detection text boxes need to be matched one by one with the 32 standard text boxes. Figure 6 As shown, Figure 6 This is the driving license template matching result.

[0108] Since the candidate text image contains multiple detection text boxes and the template image includes multiple standard text boxes, the matching process is not a pixel-level match, but a match between two text boxes. This can save time in extracting key text information and improve the speed of extracting key text information.

[0109] Step 204: Based on the result of the matching process, the server extracts target type information from the candidate text image.

[0110] In this embodiment, when the detection text box matches the standard text box successfully, the target type of information can be extracted from the text image to be tested. For example, when the 32 detection text boxes on the driving license are successfully matched with the 32 standard text boxes one by one, if the target type of information to be extracted is name information, the standard text box corresponding to the name information is found in the candidate text image, and the name information in the standard text box is identified to obtain the name information.

[0111] This application provides two methods for matching a detection text box in a candidate text image with a standard text box in a template image.

[0112] The first one is to perform sliding matching between the detection text box in the candidate text image and the standard text box in the template image.

[0113] Please refer to Figure 7 , which shows a flow chart of a sliding matching process of a detection text box in a candidate text image and a standard text box in a template image provided by an embodiment of the present application, and the method can be applied to a server. Figure 7 As shown, the matching processing method may include the following steps:

[0114] Step 701: Overlay the template text image on the candidate image and slide it.

[0115] In this embodiment, the candidate text image is superimposed on the template image and slid. Specifically, the candidate text image is first superimposed on the template image, and the sliding position is within a range of 10 pixels up, down, left, and right, and the sliding step is 2 pixels, that is, the candidate text image is slid upward or downward or left or right on the template image, and each sliding range in different directions is within 10 pixels, and each sliding step is 2 pixels.

[0116] Since the candidate text image may be blurred or tilted due to factors such as shooting angle and shooting technology, or the size of the candidate text image may be uncertain due to different shooting distances, for example, if the shooting distance is close, the obtained candidate text image will be large. Therefore, there will be a certain gap between the obtained candidate text image and the template image, and there will be an offset after the candidate text image is superimposed on the template image. In order to achieve accurate matching, the candidate text image is slid upward or downward or left or right on the template image to eliminate the offset, so that accurate matching can be achieved.

[0117] Step 702: After each slide, calculate the sum of the intersection and union ratios of each detected text box in the candidate text image and each standard text box in the template image.

[0118] Specifically, the following steps may be included: for each detected text box in the candidate text image, calculating the intersection-and-union ratio between the detected text box and each standard text box in the template image, and using the calculated maximum intersection-and-union ratio as the candidate intersection-and-union ratio corresponding to the detected text box; calculating the sum of the candidate intersection-and-union ratios corresponding to each detected text box in the candidate text image.

[0119] In this embodiment, after each slide, the sum of the candidate intersection-and-union ratios of each detection text box in the candidate text image and each standard text box in the template image is calculated. The intersection-and-union ratio is the ratio of the intersection and union of the areas of two text boxes. For example, the area of ​​a detection text box is S1, and the area of ​​a standard text box is S2. If the two text boxes have an overlapping part, and the area of ​​the overlapping part is S3, then the intersection-and-union ratio of the two text boxes is S3 / (S1+S2). The candidate intersection-and-union ratio is the maximum intersection-and-union ratio of each detection text box and each standard text box in the template image calculated after each slide.

[0120] like Figure 8 As shown, Figure 8 The schematic diagram of calculating the intersection and union ratio of the detection text box and each standard text box in the template image. Among them, the intersection and union ratio of the detection text box 1 and the standard text boxes 4, 5 and 6 are calculated respectively to obtain three intersection and union ratios, and the maximum intersection and union ratio is selected as the candidate intersection and union ratio of the detection text box 1, and then the intersection and union ratio of the detection text box 2 and the standard text boxes 4, 5 and 6 are calculated to obtain three intersection and union ratios, and the maximum intersection and union ratio is selected as the candidate intersection and union ratio of the detection text box 2, and then the intersection and union ratio of the detection text box 3 and the standard text boxes 4, 5 and 6 are calculated to obtain three intersection and union ratios, and the maximum intersection and union ratio is selected as the candidate intersection and union ratio of the detection text box 3. For example, if the intersection and union ratio of the detection text box 1 and the standard text box 4 is the largest, then the intersection and union ratio is used as the candidate intersection and union ratio of the detection text box 1, and the candidate intersection and union ratios of the detection text boxes 1, 2 and 3 are obtained in this way, and then the three candidate intersection and union ratios are summed.

[0121] Step 703: The sliding position of the candidate text image corresponding to the calculated maximum intersection-over-union ratio sum relative to the template image is taken as the final relative position.

[0122] In this embodiment, the final relative position is the sliding position of the candidate text image corresponding to the maximum intersection-and-union ratio sum relative to the template image. Specifically, the candidate intersection-and-union ratio sum in step 2032 is also the maximum intersection-and-union ratio sum here, and the maximum intersection-and-union ratio sum corresponds to the sliding position of a candidate text image relative to the template image. Since the sum of the intersection-and-union ratios of the candidate text image relative to the template image at this sliding position is the largest, that is, the overlapping area of ​​each detection text box in the candidate text image and each standard text box in the template image is the largest, then the sliding position is used as the final relative position.

[0123] Step 704: Match the detection text box in the candidate text image and the standard text box in the template image based on the final relative position.

[0124] Specifically, for each detected text box in the candidate text image located at the final relative position, the intersection-and-union ratio of the detected text box and each standard text box in the template image located at the final relative position is calculated, and the standard text box corresponding to the calculated maximum intersection-and-union ratio is used as the standard text box that matches the detected text box.

[0125] The second method is to perform coordinate normalization matching processing on the detection text box in the candidate text image and the standard text box in the template image.

[0126] Please refer to Fig. 9 , which shows a flowchart of a coordinate normalization matching process for a detection text box in a candidate text image and a standard text box in a template image provided by an embodiment of the present application, and the method can be applied to a server. Fig. 9 As shown, the matching processing method may include the following steps:

[0127] Step 901: determine the extreme points of the detection text boxes in the candidate text image based on the positions of the detection text boxes in the candidate text image, establish a first coordinate system in the candidate text image based on the extreme points of the detection text boxes in the candidate text image, and obtain the coordinates of each detection text box in the first coordinate system.

[0128] like Fig.10 As shown, Fig.10 The original ID card image, such as Fig.11 As shown, Fig.11 Schematic diagram of the positions of the detection text boxes in the ID card candidate text picture. The ID card candidate text picture contains 13 detection text boxes.

[0129] Specifically, the extreme points of the detected text box are the coordinates of the four vertices of the detected text box in the first coordinate system. The coordinates of the four vertices of the detected text box include the maximum value x_max1 and the minimum value x_min1 in the X direction, and the maximum value y_max1 and the minimum value y_min1 in the Y direction.

[0130] The first coordinate system is established in the candidate text image. Specifically, the origin coordinates of the first coordinate system can be the vertex coordinates (x_min1, y_min1) of the lower left corner of the detection text box, and the detection text box is located in the first quadrant of the first coordinate system. (x1, y1) are the coordinates of the detection text box. Specifically, the center point coordinates of the detection text box can be selected as the coordinates of the detection text box.

[0131] Step 902: based on the position of each standard text box in the template image, determine the extreme points of the standard text boxes in the template image, establish a second coordinate system in the template image based on the extreme points of the standard text boxes in the template image, and obtain the coordinates of each standard text box in the second coordinate system.

[0132] like Fig.12 As shown, Fig.12 This is an ID card template image, which contains 13 standard text boxes.

[0133] Specifically, the extreme points of the standard text box are the coordinates of the four vertices of the standard text box in the first coordinate system. The coordinates of the four vertices of the standard text box include the maximum value x_max2 and the minimum value x_min2 in the X direction, and the maximum value y_max2 and the minimum value y_min2 in the Y direction.

[0134] The second coordinate system is established in the template image. Specifically, the origin coordinates of the first coordinate system can be the vertex coordinates (x_min2, y_min2) of the lower left corner of the standard text box, and the standard text box is located in the first quadrant of the second coordinate system. (x2, y2) are the coordinates of the standard text box. Specifically, the center point coordinates of the standard text box can be selected as the coordinates of the standard text box.

[0135] Step 903: Match the detected text boxes in the candidate text image with the standard text boxes in the template image based on the coordinates of each detected text box in the first coordinate system and the coordinates of each standard text box in the second coordinate system. Fig.13 As shown, Fig.13 A schematic diagram of the ID card template matching results.

[0136] Specifically, for each detected text box in the candidate text image, the projected position of the detected text box in the second coordinate system is determined based on the coordinates of the detected text box in the first coordinate system, and the intersection and union ratio of the detected text box and each standard text box is calculated when the detected text box is located at the projected position, and the standard text box corresponding to the calculated maximum intersection and union ratio is used as the standard text box that matches the detected text box.

[0137] Specifically, the coordinates of each detection text box in the first coordinate system and the extreme point are normalized. For example, (x1, y1) is the coordinate of the detection text box, and the normalized coordinates are (x11, y11). The specific normalization method is shown in Formula 1 and Formula 2, where Formula 1 is the calculation method for normalizing x1, and Formula 2 is the calculation method for normalizing y1. The coordinates of each standard text box in the second coordinate system and the extreme point are normalized. For example, (x2, y2) is the coordinate of the detection text box, and the normalized coordinates are (x22, y22). The specific normalization method is shown in Formula 3 and Formula 4, where Formula 3 is the calculation method for normalizing x2, and Formula 4 is the calculation method for normalizing y2.

[0138] x11=x1 / (x_max1-x_min1) Formula 1

[0139] y11=y1 / (y_max1-y_min1) Formula 2

[0140] x22=x2 / (x_max2-x_min2) Formula 3

[0141] y22=y2 / (y_max2-y_min2) Formula 4

[0142] Then the intersection of normalized detection text box and standard text box is calculated, and the standard text box corresponding to the calculated maximum intersection of normalized detection text box is used as the standard text box that matches the detection text box. Since the normalized detection text box and the standard text box are both in the same coordinate system, the offset caused when the detection text box is superimposed on the standard text box can be eliminated, thereby achieving accurate matching.

[0143] Optionally, the template image is resized and then matched with the candidate text image. The matching method may include the following steps:

[0144] The size of the template image is adjusted based on the size of the candidate text image to obtain an adjusted template image, which includes multiple adjusted standard text boxes. Specifically, the sum of the lengths of the detected text boxes in the candidate text image is calculated to obtain a first length and value, for example, the first length and value is 10; the sum of the widths of the detected text boxes in the candidate text image is calculated to obtain a first width and value, for example, the first width and value is 5; the sum of the lengths of the standard text boxes in the template image is calculated to obtain a second length and value, for example, the second length and value is 20; the sum of the widths of the standard text boxes in the template image is calculated to obtain a second width and value, for example, the second width and value is 10.

[0145] The length of the template image is adjusted based on the ratio of the first length sum value to the second length sum value. Specifically, if the ratio of the first length sum value to the second length sum value is 1 / 2, the length of the template image is adjusted to 1 / 2 of the original length. The width of the template image is adjusted based on the ratio of the first width sum value to the second width sum value. If the ratio of the first width sum value to the second width sum value is 1 / 2, the width of the template image is adjusted to 1 / 2 of the original length.

[0146] The process of matching the detection text box in the candidate text image and the adjusted standard text box in the adjusted template image is the same as the above-mentioned matching process, and the embodiments of the present application will not be repeated here. Specifically, since the adjusted template image is the same size as the candidate text image, the detection text box in the candidate text image and the adjusted standard text box in the adjusted template image can be subjected to sliding matching processing. At this time, it is only necessary to calculate the intersection and union ratio of a detection text box and each standard text box in the template image, and use the sliding position corresponding to the maximum intersection and union ratio as the final relative position. Based on the final relative position, the detection text box in the candidate text image and the standard text box in the template image are matched one by one, which makes it easier to match and improves the matching speed, thereby improving the extraction and recognition speed of key text information.

[0147] Optionally, there may be an empty standard text box in the template image. Specifically, it is detected whether there is an empty standard text box in the template image, and the empty standard text box does not match any detection text box in the candidate text image; if there is an empty standard text box in the template image, the missed detection area is intercepted in the candidate text image based on the position of the empty standard text box; the missed detection area is annotated, and the detection text box corresponding to the empty standard text box is obtained through the annotation process.

[0148] Among them, the empty standard text is that the standard text box in the template image cannot be matched with the detection text box, that is, the standard text box exists in the template image, but the detection text box is missing due to missed detection in the candidate text image. For example, there are a total of 14 standard text boxes in the template image, but there are only 13 detection text boxes in the candidate text image, that is, one detection text box is missed. Then after matching, there is an empty standard text box in the template image. At this time, the missed detection area is captured in the candidate text image based on the position of the empty standard text box. For example, the name standard text box in the ID card template image is an empty standard text box. Then, based on the position of the name standard text box in the ID card template image, the image of the name area in the ID card candidate text image is captured, and the captured image is re-annotated to obtain the corresponding name detection box.

[0149] Since the matched template image is further inspected and processed, the missed detection area is re-marked and then detected again to obtain the detection text box, which can avoid the problem of target type information loss due to missed detection of the detection text box and improve the accuracy of target type information detection and extraction.

[0150] Please refer to Fig.14 , which shows a flow chart of a method for extracting target type information provided by an embodiment of the present application, and the method for extracting target type information can be applied to a server. Fig.14 As shown, the method for extracting target type information may include the following steps:

[0151] Step 1401: Obtain from the template image a standard text box for indicating an area where text containing information of the target type is located.

[0152] Step 1402: determine a detection text box corresponding to the standard text box, and extract text from the determined detection text box.

[0153] Step 1403: perform recognition processing on the extracted text to obtain target type information.

[0154] Specifically, when identifying the target type information, a text recognition model based on deep learning can be used for recognition. First, the text information in the detection text box on all candidate images obtained in step 201 is used as sample text information, and a "text information: ID" mapping table is established for all sample text information. For example, the text information can be name, gender, place of origin, etc., and the ID can be number 1, 2, 3, etc., so the established mapping table is "name: 1, gender: 2, place of origin: 3". The mapping table is then used as an input to train the text recognition model based on deep learning. After each training, a text information and the ID corresponding to the text information will be output.

[0155] After the deep learning-based text recognition model is trained, text is extracted from the detection text box corresponding to the standard text box indicating the area where the text containing the target type information is located in the template image, and the text in the detection text box is used as the input of the text recognition model to finally obtain the target type information.

[0156] Please refer to Fig.15 , which shows a block diagram of a text recognition device 1500 provided in an embodiment of the present application, and the text recognition device 1500 can be configured in a server. Fig.15 As shown, the text recognition device 1500 may include: a marking module 1501 , an acquisition module 1502 , a matching module 1503 and an extraction module 1504 .

[0157] Among them, the annotation module 1501 is used to annotate the target type of text image to be tested to obtain a candidate text image, which includes multiple detection text boxes, each detection text box includes multiple characters in the text image to be tested whose distances are less than a preset distance threshold.

[0158] The acquisition module 1502 is used to acquire a template image corresponding to the target type, the template image includes a plurality of standard text boxes, and different standard text boxes are used to indicate the area where text containing different types of information is located in the sample text image of the target type.

[0159] The matching module 1503 is used to match the detection text box in the candidate text image with the standard text box in the template image.

[0160] The extraction module 1504 is used to extract target type information from the candidate text image according to the result of the matching process.

[0161] In one of the embodiments, the device further includes a first adjustment processing unit and a first matching unit.

[0162] The first adjustment processing unit is used to adjust the size of the template image based on the size of the candidate text image to obtain an adjusted template image, where the adjusted template image includes a plurality of adjustment standard text boxes.

[0163] The first matching unit is used to match the detection text box in the candidate text image with the adjustment standard text box in the adjustment template image.

[0164] In one of the embodiments, the device further includes a first calculation unit and a second adjustment processing unit.

[0165] The first calculation unit is used to calculate the sum of the lengths of each detection text box in the candidate text image to obtain a first length sum value; calculate the sum of the widths of each detection text box in the candidate text image to obtain a first width sum value; calculate the sum of the lengths of each standard text box in the template image to obtain a second length sum value; calculate the sum of the widths of each standard text box in the template image to obtain a second width sum value.

[0166] The second adjustment processing unit is used to adjust the length of the template image based on the ratio of the first length sum value to the second length sum value, and to adjust the width of the template image based on the ratio of the first width sum value to the second width sum value.

[0167] In one of the embodiments, the device further includes a sliding unit, a second calculating unit and a third matching unit.

[0168] The sliding unit is used to superimpose the template text image on the candidate image for sliding.

[0169] The second calculation unit is used to calculate the sum of the intersection and union ratios of each detection text box in the candidate text image and each standard text box in the template image after each sliding; and take the sliding position of the candidate text image corresponding to the calculated maximum sum of the intersection and union ratios relative to the template image as the final relative position.

[0170] The third matching unit is used to match the detection text box in the candidate text image with the standard text box in the template image based on the final relative position.

[0171] In one of the embodiments, the device further includes a third computing unit and a fourth computing unit.

[0172] The third calculation unit is used to calculate the intersection-and-union ratio of each detection text box in the candidate text picture and each standard text box in the template picture, and use the calculated maximum intersection-and-union ratio as the candidate intersection-and-union ratio corresponding to the detection text box.

[0173] The fourth calculation unit is used to calculate the sum of candidate intersection-and-union ratios corresponding to each detected text box in the candidate text image.

[0174] In one of the embodiments, the apparatus further includes a fifth computing unit.

[0175] The fifth calculation unit is used to calculate the intersection and union ratio of each detection text box in the candidate text image located at the final relative position and each standard text box in the template image located at the final relative position, and use the standard text box corresponding to the calculated maximum intersection and union ratio as the standard text box that matches the detection text box.

[0176] In one of the embodiments, the device further includes a second acquiring unit, an establishing unit and a fourth matching unit.

[0177] The second acquisition unit is used to determine the extreme points of the detection text boxes in the candidate text image based on the positions of each detection text box in the candidate text image, establish a first coordinate system in the candidate text image based on the extreme points of the detection text boxes in the candidate text image, and obtain the coordinates of each detection text box in the first coordinate system.

[0178] The establishment unit is used to determine the extreme points of the standard text boxes in the template image based on the positions of the standard text boxes in the template image, establish a second coordinate system in the template image based on the extreme points of the standard text boxes in the template image, and obtain the coordinates of each standard text box in the second coordinate system.

[0179] The fourth matching unit is used to match the detection text box in the candidate text image with the standard text box in the template image based on the coordinates of each detection text box in the first coordinate system and the coordinates of each standard text box in the second coordinate system.

[0180] In one of the embodiments, the apparatus further includes a sixth computing unit.

[0181] The sixth calculation unit is used to determine, for each detection text box in the candidate text image, a projected position of the detection text box in the second coordinate system based on the coordinates of the detection text box in the first coordinate system, and calculate the intersection and union ratio of the detection text box with each standard text box when the detection text box is located at the projected position, and use the standard text box corresponding to the calculated maximum intersection and union ratio as the standard text box that matches the detection text box.

[0182] In one of the embodiments, the device further includes a detection unit, a capture unit and a first labeling unit.

[0183] The detection unit is used to detect whether there is an empty standard text box in the template image, and the empty standard text box does not match any detection text box in the candidate text image.

[0184] The clipping unit is used to clip the missed detection area in the candidate text image based on the position of the empty standard text box if there is an empty standard text box in the template image.

[0185] The first labeling unit is used to label the missed detection area, and obtain a detection text box corresponding to the empty standard text box through the labeling process.

[0186] In one of the embodiments, the device further includes a third acquiring unit, a second marking unit, a seventh calculating unit and a generating unit.

[0187] The third acquisition unit is used to acquire multiple sample text images of the target type.

[0188] The second annotation unit is used to perform annotation processing on each sample text image to obtain a text box set corresponding to each sample text image, where the text box set includes multiple candidate text boxes, and each candidate text box includes multiple characters in the corresponding sample text image whose distances from each other are less than a preset distance threshold.

[0189] The seventh calculation unit is used to obtain, for each information type contained in the sample text image, candidate text boxes corresponding to the information type from each text box set, and calculate the size average and position average of the obtained candidate text boxes.

[0190] The generating unit is used to generate a standard text box corresponding to each information type based on the size average and position average of the candidate text boxes corresponding to each information type; and obtain a template image based on the generated standard text boxes.

[0191] In one of the embodiments, the device further includes a fourth acquiring unit, a determining unit, and an identifying unit.

[0192] The fourth acquisition unit is used to acquire, from the template image, a standard text box for indicating an area where text containing information of the target type is located.

[0193] The determination unit is used to determine a detection text box corresponding to the standard text box, and extract text from the determined detection text box.

[0194] The recognition unit is used to perform recognition processing on the extracted text to obtain information of the target type.

[0195] The text recognition device provided in the embodiment of the present application can implement the above method embodiment, and its implementation principle and technical effect are similar, which will not be repeated here.

[0196] For the specific definition of the text recognition device, please refer to the definition of the text recognition method above, which will not be repeated here. Each module in the above-mentioned request text recognition can be implemented in whole or in part by software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations of the above modules.

[0197] In one embodiment of the present application, a computer device is provided, the computer device comprising a memory and a processor, the memory storing a computer program, and the processor implementing the following steps when executing the computer program:

[0198] The target type of text image to be tested is annotated to obtain a candidate text image, wherein the candidate text image includes a plurality of detection text boxes, each detection text box includes a plurality of characters in the text image to be tested whose distances from each other are less than a preset distance threshold; a template image corresponding to the target type is obtained, wherein the template image includes a plurality of standard text boxes, and different standard text boxes are used to indicate the area where text containing different types of information is located in the sample text image of the target type; matching processing is performed on the detection text boxes in the candidate text image and the standard text boxes in the template image; and according to the result of the matching processing, information of the target type is extracted from the candidate text image.

[0199] In one embodiment of the present application, when the processor executes the computer program, it also implements the following steps: matching the detection text box in the candidate text image with the standard text box in the template image, including: adjusting the size of the template image based on the size of the candidate text image to obtain an adjusted template image, the adjusted template image including multiple adjusted standard text boxes; matching the detection text box in the candidate text image with the adjusted standard text box in the adjusted template image.

[0200] In one embodiment of the present application, when the processor executes the computer program, it also implements the following steps: calculating the sum of the lengths of each detected text box in the candidate text image to obtain a first length sum value; calculating the sum of the widths of each detected text box in the candidate text image to obtain a first width sum value; calculating the sum of the lengths of each standard text box in the template image to obtain a second length sum value; calculating the sum of the widths of each standard text box in the template image to obtain a second width sum value; adjusting the length of the template image based on the ratio of the first length sum value to the second length sum value, and adjusting the width of the template image based on the ratio of the first width sum value to the second width sum value.

[0201] In one embodiment of the present application, the processor also implements the following steps when executing the computer program: superimposing the template text image on the candidate image and sliding it; after each slide, calculating the sum of the intersection and union ratios of each detection text box in the candidate text image and each standard text box in the template image; taking the sliding position of the candidate text image corresponding to the calculated maximum intersection and union ratio relative to the template image as the final relative position; and matching the detection text box in the candidate text image and the standard text box in the template image based on the final relative position.

[0202] In one embodiment of the present application, when the processor executes the computer program, the following steps are also implemented: for each detected text box in the candidate text image, the intersection and union ratio of the detected text box and each standard text box in the template image is calculated, and the calculated maximum intersection and union ratio is used as the candidate intersection and union ratio corresponding to the detected text box; the sum of the candidate intersection and union ratios corresponding to each detected text box in the candidate text image is calculated.

[0203] In one embodiment of the present application, when the processor executes the computer program, the following steps are also implemented: for each detected text box in the candidate text image located at the final relative position, the intersection and union ratio of the detected text box and each standard text box in the template image located at the final relative position is calculated, and the standard text box corresponding to the calculated maximum intersection and union ratio is used as the standard text box that matches the detected text box.

[0204] In one embodiment of the present application, the processor also implements the following steps when executing the computer program: based on the position of each detection text box in the candidate text image, determine the extreme points of the detection text box in the candidate text image, based on the extreme points of the detection text box in the candidate text image, establish a first coordinate system in the candidate text image, and obtain the coordinates of each detection text box in the first coordinate system; based on the position of each standard text box in the template image, determine the extreme points of the standard text box in the template image, based on the extreme points of the standard text box in the template image, establish a second coordinate system in the template image, and obtain the coordinates of each standard text box in the second coordinate system; based on the coordinates of each detection text box in the first coordinate system and the coordinates of each standard text box in the second coordinate system, match the detection text boxes in the candidate text image with the standard text boxes in the template image.

[0205] In one embodiment of the present application, the processor further implements the following steps when executing the computer program: for each detected text box in the candidate text image, based on the coordinates of the detected text box in the first coordinate system, determine the projected position of the detected text box in the second coordinate system, and calculate the intersection and union ratio of the detected text box with each standard text box when the detected text box is located at the projected position, and use the standard text box corresponding to the calculated maximum intersection and union ratio as the standard text box that matches the detected text box.

[0206] In one embodiment of the present application, when the processor executes the computer program, the following steps are also implemented: detecting whether there is an empty standard text box in the template image, and the empty standard text box does not match any detection text box in the candidate text image; if there is an empty standard text box in the template image, then intercepting the missed detection area in the candidate text image based on the position of the empty standard text box; annotating the missed detection area, and obtaining the detection text box corresponding to the empty standard text box through the annotation process.

[0207] In one embodiment of the present application, when the processor executes the computer program, it also implements the following steps: obtaining multiple sample text images of the target type; performing annotation processing on each sample text image to obtain a text box set corresponding to each sample text image, the text box set including multiple candidate text boxes, each candidate text box including multiple texts in the corresponding sample text image whose distances from each other are less than a preset distance threshold; for each information type contained in the sample text image, obtaining the candidate text box corresponding to the information type from each text box set, and calculating the size mean and position mean of the obtained candidate text box; generating a standard text box corresponding to each information type based on the size mean and position mean of the candidate text box corresponding to each information type; and obtaining a template image based on the generated standard text boxes.

[0208] In one embodiment of the present application, when the processor executes the computer program, it also implements the following steps: obtaining a standard text box from the template image that is used to indicate the area where text containing information of the target type is located; determining a detection text box corresponding to the standard text box, and extracting text from the determined detection text box; and performing recognition processing on the extracted text to obtain information of the target type.

[0209] The computer device provided in the embodiment of the present application has similar implementation principles and technical effects to those of the above-mentioned method embodiment, and will not be described in detail here.

[0210] In one embodiment of the present application, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the following steps are implemented:

[0211] The target type of text image to be tested is annotated to obtain a candidate text image, wherein the candidate text image includes a plurality of detection text boxes, each detection text box includes a plurality of characters in the text image to be tested whose distances from each other are less than a preset distance threshold; a template image corresponding to the target type is obtained, wherein the template image includes a plurality of standard text boxes, and different standard text boxes are used to indicate the area where text containing different types of information is located in the sample text image of the target type; matching processing is performed on the detection text boxes in the candidate text image and the standard text boxes in the template image; and according to the result of the matching processing, information of the target type is extracted from the candidate text image.

[0212] In one embodiment of the present application, when the computer program is executed by the processor, the following steps are also implemented: matching the detection text box in the candidate text image with the standard text box in the template image, including: adjusting the size of the template image based on the size of the candidate text image to obtain an adjusted template image, the adjusted template image including multiple adjusted standard text boxes; matching the detection text box in the candidate text image with the adjusted standard text box in the adjusted template image.

[0213] In one embodiment of the present application, when the computer program is executed by the processor, the following steps are also implemented: calculating the sum of the lengths of each detected text box in the candidate text image to obtain a first length sum value; calculating the sum of the widths of each detected text box in the candidate text image to obtain a first width sum value; calculating the sum of the lengths of each standard text box in the template image to obtain a second length sum value; calculating the sum of the widths of each standard text box in the template image to obtain a second width sum value; adjusting the length of the template image based on the ratio of the first length sum value to the second length sum value, and adjusting the width of the template image based on the ratio of the first width sum value to the second width sum value.

[0214] In one embodiment of the present application, when the computer program is executed by the processor, the following steps are also implemented: superimposing the template text image on the candidate image and sliding it; after each slide, calculating the sum of the intersection and union ratios of each detection text box in the candidate text image and each standard text box in the template image; taking the sliding position of the candidate text image corresponding to the calculated maximum intersection and union ratio relative to the template image as the final relative position; and matching the detection text box in the candidate text image and the standard text box in the template image based on the final relative position.

[0215] In one embodiment of the present application, when the computer program is executed by a processor, the following steps are also implemented: for each detected text box in the candidate text image, the intersection-and-union ratio of the detected text box and each standard text box in the template image is calculated, and the calculated maximum intersection-and-union ratio is used as the candidate intersection-and-union ratio corresponding to the detected text box; the sum of the candidate intersection-and-union ratios corresponding to each detected text box in the candidate text image is calculated.

[0216] In one embodiment of the present application, when the computer program is executed by the processor, the following steps are also implemented: for each detected text box in the candidate text image located at the final relative position, the intersection and union ratio of the detected text box and each standard text box in the template image located at the final relative position is calculated, and the standard text box corresponding to the calculated maximum intersection and union ratio is used as the standard text box that matches the detected text box.

[0217] In one embodiment of the present application, when the computer program is executed by the processor, the following steps are also implemented: based on the position of each detection text box in the candidate text image, the extreme points of the detection text box in the candidate text image are determined; based on the extreme points of the detection text box in the candidate text image, a first coordinate system is established in the candidate text image, and the coordinates of each detection text box in the first coordinate system are obtained; based on the position of each standard text box in the template image, the extreme points of the standard text box in the template image are determined; based on the extreme points of the standard text box in the template image, a second coordinate system is established in the template image, and the coordinates of each standard text box in the second coordinate system are obtained; based on the coordinates of each detection text box in the first coordinate system and the coordinates of each standard text box in the second coordinate system, the detection text boxes in the candidate text image and the standard text boxes in the template image are matched.

[0218] In one embodiment of the present application, when the computer program is executed by the processor, the following steps are also implemented: for each detected text box in the candidate text image, based on the coordinates of the detected text box in the first coordinate system, the projected position of the detected text box in the second coordinate system is determined, and when the detected text box is located at the projected position, the intersection and union ratio of the detected text box and each standard text box is calculated, and the standard text box corresponding to the calculated maximum intersection and union ratio is used as the standard text box that matches the detected text box.

[0219] In one embodiment of the present application, when the computer program is executed by the processor, the following steps are also implemented: detecting whether there is an empty standard text box in the template image, and the empty standard text box does not match any detection text box in the candidate text image; if there is an empty standard text box in the template image, then intercepting the missed detection area in the candidate text image based on the position of the empty standard text box; annotating the missed detection area, and obtaining the detection text box corresponding to the empty standard text box through the annotation process.

[0220] In one embodiment of the present application, when the computer program is executed by the processor, the following steps are also implemented: obtaining multiple sample text images of the target type; performing annotation processing on each sample text image to obtain a text box set corresponding to each sample text image, the text box set including multiple candidate text boxes, each candidate text box including multiple texts in the corresponding sample text image whose distances from each other are less than a preset distance threshold; for each information type contained in the sample text image, obtaining the candidate text box corresponding to the information type from each text box set, and calculating the size mean and position mean of the obtained candidate text box; generating a standard text box corresponding to each information type based on the size mean and position mean of the candidate text box corresponding to each information type; and obtaining a template image based on the generated standard text boxes.

[0221] In one embodiment of the present application, when the computer program is executed by a processor, the following steps are also implemented: obtaining a standard text box from a template image that indicates the area where text containing information of the target type is located; determining a detection text box corresponding to the standard text box, and extracting text from the determined detection text box; and performing recognition processing on the extracted text to obtain information of the target type.

[0222] The computer-readable storage medium provided in this embodiment has similar implementation principles and technical effects to those of the above method embodiments, and will not be described in detail here.

[0223] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0224] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0225] The above embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the patent application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent application shall be subject to the attached claims.

Claims

1. A text recognition method, characterized in that: The method comprises: Annotate the target type of the text image to be tested to obtain a candidate text image, wherein the candidate text image includes a plurality of detection text frames, each of which includes a plurality of characters in the text image to be tested whose distances from each other are less than a preset distance threshold; Acquire a template image corresponding to the target type, the template image comprising a plurality of standard text boxes, and different standard text boxes are used to indicate regions where text containing different types of information is located in a sample text image of the target type; Performing matching processing on the detection text box in the candidate text image and the standard text box in the template image; Extracting target type information from the candidate text image according to the result of the matching process; The matching process of the detection text box in the candidate text picture and the standard text box in the template picture includes: Overlaying the template image on the candidate text image and sliding it; After each slide, the sum of the intersection and union ratios of each detected text box in the candidate text image and each standard text box in the template image is calculated; The sliding position of the candidate text image corresponding to the calculated maximum intersection-over-union ratio sum relative to the template image is taken as the final relative position; The detected text box in the candidate text image and the standard text box in the template image are matched based on the final relative position.

2. The method according to claim 1, characterized in that: The calculating the sum of the intersection and union ratios of each detected text box in the candidate text picture and each standard text box in the template picture includes: For each detected text box in the candidate text image, calculating the intersection-and-union ratio between the detected text box and each standard text box in the template image, and using the calculated maximum intersection-and-union ratio as the candidate intersection-and-union ratio corresponding to the detected text box; The sum of the candidate intersection-union ratios corresponding to each detected text box in the candidate text image is calculated.

3. The method according to claim 1, characterized in that The matching process of the detection text box in the candidate text picture and the standard text box in the template picture based on the final relative position includes: For each detected text box in the candidate text image located at the final relative position, the intersection-and-union ratio of the detected text box and each standard text box in the template image located at the final relative position is calculated, and the standard text box corresponding to the calculated maximum intersection-and-union ratio is used as the standard text box that matches the detected text box.

4. The method according to claim 1, characterized in that: The matching process of the detection text box in the candidate text picture and the standard text box in the template picture includes: Based on the position of each detection text box in the candidate text picture, determine the extreme points of the detection text box in the candidate text picture, establish a first coordinate system in the candidate text picture based on the extreme points of the detection text box in the candidate text picture, and obtain the coordinates of each detection text box in the first coordinate system; Based on the position of each standard text box in the template image, determine the extreme points of the standard text boxes in the template image, establish a second coordinate system in the template image based on the extreme points of the standard text boxes in the template image, and obtain the coordinates of each standard text box in the second coordinate system; The detected text boxes in the candidate text image and the standard text boxes in the template image are matched based on the coordinates of each detected text box in the first coordinate system and the coordinates of each standard text box in the second coordinate system.

5. The method according to claim 4, characterized in that The matching process of the detected text box in the candidate text image and the standard text box in the template image based on the coordinates of each detected text box in the first coordinate system and the coordinates of each standard text box in the second coordinate system includes: For each detected text box in the candidate text image, based on the coordinates of the detected text box in the first coordinate system, determine the projected position of the detected text box in the second coordinate system, and calculate the intersection and union ratio of the detected text box with each standard text box when the detected text box is located at the projected position, and use the standard text box corresponding to the calculated maximum intersection and union ratio as the standard text box that matches the detected text box.

6. The method according to claim 1, characterized in that After matching the detected text box in the candidate text picture with the standard text box in the template picture, the method further includes: Detecting whether there is an empty standard text box in the template image, wherein the empty standard text box does not match any detection text box in the candidate text image; If the empty standard text box exists in the template image, intercepting a missed detection area in the candidate text image based on the position of the empty standard text box; The missed detection area is annotated to obtain a detection text box corresponding to the empty standard text box.

7. A text recognition device, characterized in that: The device comprises: A labeling module is used to label the target type of the text image to be tested to obtain a candidate text image, wherein the candidate text image includes a plurality of detection text frames, each of which includes a plurality of characters in the text image to be tested whose distances from each other are less than a preset distance threshold; An acquisition module, used for acquiring a template image corresponding to the target type, wherein the template image includes a plurality of standard text boxes, and different standard text boxes are used for indicating areas where text containing different types of information is located in a sample text image of the target type; A matching module, used for matching the detection text box in the candidate text picture with the standard text box in the template picture; An extraction module, used to extract target type information from the candidate text image according to the result of the matching process; The device further comprises: a sliding unit, a second calculating unit and a third matching unit; The sliding unit is used to slide the template image over the candidate text image; The second calculation unit is used to calculate the sum of intersection-and-union ratios of each detection text box in the candidate text picture and each standard text box in the template picture after each sliding; and use the sliding position of the candidate text picture corresponding to the calculated maximum sum of intersection-and-union ratios relative to the template picture as the final relative position; The third matching unit is used to match the detection text box in the candidate text image with the standard text box in the template image based on the final relative position.

8. The device according to claim 7, characterized in that The device also includes a third computing unit and a fourth computing unit; The third calculation unit is used to calculate the intersection-and-union ratio of each detection text box in the candidate text picture and each standard text box in the template picture, and use the calculated maximum intersection-and-union ratio as the candidate intersection-and-union ratio corresponding to the detection text box; The fourth calculation unit is used to calculate the sum of candidate intersection-and-union ratios corresponding to each detected text box in the candidate text image.

9. A computer device, characterized in that: The method comprises a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the text recognition method according to any one of claims 1 to 6 is implemented.

10. A computer-readable storage medium, characterized in that: A computer program is stored thereon, and when the computer program is executed by a processor, the text recognition method according to any one of claims 1 to 6 is implemented.

Images