Key text information extraction method and device, electronic equipment and storage medium

Abstract

The application provides a key text information extraction method and device, electronic equipment and storage medium, and relates to the technical field of text detection and recognition. The key text information extraction method comprises the following steps: extracting a text heat map of a target text region in a to-be-recognized image; performing edge detection on the text heat map to obtain text region coordinates; cropping a first text region image located in the text region coordinates from the to-be-recognized image; fusing an image feature of a second text region image located in the text region coordinates in the text heat map with the first text region image to obtain a third text region image; and performing text recognition on the third text region image through a text recognition model to obtain key text information in the third text region image. The key text information extraction method, device, electronic equipment and storage medium provided by the application can improve the recognition accuracy of text information.

Description

Technical Field

[0001] This application relates to the field of text detection and recognition technology, and in particular to a method, apparatus, electronic device and storage medium for extracting key text information. Background Technology

[0002] Key text information extraction refers to extracting computer-readable text information from images, such as extracting names and ID numbers from ID card images or bank card numbers from bank card images. It includes three steps: text detection and text recognition, and extraction and parsing.

[0003] For text detection, current methods aim to detect all text in an image. This leads to a significant time commitment in subsequent text recognition and extraction / parsing steps for irrelevant text. The time required for text recognition increases linearly with the amount of irrelevant text, while the extraction / parsing steps require even more, resulting in the entire process being time-consuming on irrelevant text.

[0004] For text recognition, the most common method is to use pixel information in the image (such as the RGB values ​​or binarized values ​​of pixels in the image) for recognition. However, using this method for text recognition results in a single data source, which can affect the accuracy of the recognition process, easily lead to misidentification, and has low recognition efficiency.

[0005] Therefore, how to provide an effective solution for the rapid and accurate recognition of text information in images has become a pressing problem to be solved in existing technologies. Summary of the Invention

[0006] The purpose of this invention is to provide a method, apparatus, electronic device, and storage medium for extracting key text information, so as to solve the problems of poor accuracy and efficiency in text information recognition in the prior art.

[0007] To achieve the above objectives, the present invention adopts the following technical solution:

[0008] In a first aspect, embodiments of this application provide a method for extracting key text information, including:

[0009] Extract the text heatmap of the target text region in the image to be identified;

[0010] Edge detection is performed on the text heatmap to obtain the coordinates of the text regions;

[0011] A first text region image located within the coordinates of the text region is cropped from the image to be identified;

[0012] The image features of the second text region image located within the coordinates of the text region in the text heatmap are fused with those of the first text region image to obtain the third text region image.

[0013] By performing text recognition on the third text region image using a text recognition model, key text information in the third text region image is obtained.

[0014] In one possible design, fusing the image features of the second text region image located within the text region coordinates in the text heatmap with the first text region image to obtain a third text region image includes:

[0015] The brightness values ​​of each pixel in each color channel in the first text region image are combined with the thermal values ​​of the corresponding pixels in the second text region image to obtain the third text region image, whose feature parameters include thermal values ​​and brightness values ​​of each color channel.

[0016] In one possible design, extracting the text heatmap of the image to be identified includes:

[0017] The text heatmap of the image to be identified is extracted using an improved DBNet network;

[0018] The improved DBNet network includes a text region center point prediction branch, which is used to predict the center point heatmap of the text region.

[0019] The improved DBNet network is obtained through supervised learning using the cross-entropy loss of the heatmap of the center points of the text region and the loss of the number of center points of the text region as the overall loss of the model.

[0020] In one possible design, the improved DBNet network employs a lightweight backbone network.

[0021] In one possible design, the edge detection of the text heatmap to obtain text region coordinates includes:

[0022] The text heatmap is binarized;

[0023] The edge detection algorithm is used to perform edge detection on the binarized text heatmap to obtain the edge contour of the text region.

[0024] Based on the edge contour of the text region, determine the minimum bounding rectangle of the text region;

[0025] The coordinates of the text region are obtained based on the smallest bounding rectangle of the text region.

[0026] In one possible design, the method further includes:

[0027] The text heatmap is scaled according to the size ratio between the image to be identified and the text heatmap;

[0028] The step of performing edge detection on the text heatmap to obtain text region coordinates includes:

[0029] Edge detection is performed on the scaled text heatmap to obtain the coordinates of the text region.

[0030] In one possible design, the method further includes:

[0031] The coordinates of the text region are scaled according to the size ratio of the image to be identified and the text heatmap;

[0032] The step of cropping the first text region image located within the coordinates of the text region from the image to be recognized includes:

[0033] A first text region image is cropped from the image to be identified and located within the scaled coordinates of the text region.

[0034] Secondly, embodiments of this application provide a key text information extraction device, including:

[0035] The extraction unit is used to extract the text heatmap of the target text region in the image to be recognized;

[0036] An edge detection unit is used to perform edge detection on the text heatmap to obtain the coordinates of the text region.

[0037] A cropping unit is used to crop out a first text region image located within the coordinates of the text region from the image to be recognized;

[0038] The feature fusion unit is used to fuse the image features of the second text region image located within the coordinates of the text region in the text heatmap with the first text region image to obtain a third text region image.

[0039] The text recognition unit is used to perform text recognition on the third text region image through a text recognition model to obtain key text information in the third text region image.

[0040] Thirdly, embodiments of this application provide an electronic device, including a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the bus;

[0041] Memory, used to store computer programs;

[0042] The processor executes programs stored in memory, implementing the following process:

[0043] Extract the text heatmap of the target text region in the image to be identified;

[0044] Edge detection is performed on the text heatmap to obtain the coordinates of the text regions;

[0045] A first text region image located within the coordinates of the text region is cropped from the image to be identified;

[0046] The image features of the second text region image located within the coordinates of the text region in the text heatmap are fused with those of the first text region image to obtain the third text region image.

[0047] By performing text recognition on the third text region image using a text recognition model, key text information in the third text region image is obtained.

[0048] Fourthly, embodiments of this application provide a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the following process:

[0049] Extract the text heatmap of the target text region in the image to be identified;

[0050] Edge detection is performed on the text heatmap to obtain the coordinates of the text regions;

[0051] A first text region image located within the coordinates of the text region is cropped from the image to be identified;

[0052] The image features of the second text region image located within the coordinates of the text region in the text heatmap are fused with those of the first text region image to obtain the third text region image.

[0053] By performing text recognition on the third text region image using a text recognition model, key text information in the third text region image is obtained.

[0054] The above-described technical solutions employed in one or more embodiments of this application can achieve the following beneficial effects:

[0055] By extracting a text heatmap of the target text region from the image to be identified, edge detection is performed on the text heatmap to obtain the coordinates of the text region. Then, a first text region image located within the text region coordinates is cropped from the image to be identified. The second text region image located within the text region coordinates in the text heatmap is then fused with the first text region image to obtain a third text region image. Finally, a text recognition model is used to recognize the text in the third text region image to obtain the key text information. In this way, the process of recognizing text information can be comprehensively considered from two dimensions: the image features of the image to be identified and the heatmap's heat value features. This allows for faster and more accurate recognition of text information in the image, improving the efficiency and accuracy of text information recognition. Attached Figure Description

[0056] The accompanying drawings, which are included to provide a further understanding of this document and form part of this document, illustrate exemplary embodiments and their descriptions, serving to explain this document and do not constitute an undue limitation thereof. In the drawings:

[0057] Figure 1 A flowchart illustrating a key text information extraction method provided in one embodiment of this application.

[0058] Figure 2 This is a schematic diagram of the structure of an electronic device provided in one embodiment of this application.

[0059] Figure 3 This is a schematic diagram of the structure of a key text information extraction device provided in one embodiment of this application. Detailed Implementation

[0060] In order to accurately identify text information in images, embodiments of this application provide a key text information extraction method, apparatus, electronic device and storage medium. This key text information extraction method, apparatus, electronic device and storage medium can more accurately identify text information in the image to be identified, thereby improving the accuracy of text information recognition.

[0061] The key text information extraction method provided in this application embodiment can be applied to user terminals or servers. The user terminal may be, but is not limited to, personal computers, smartphones, tablets, personal digital assistants (PDAs), etc.

[0062] It is understood that the execution entity described does not constitute a limitation on the embodiments of this application.

[0063] The key text information extraction method provided in the embodiments of this application will be described in detail below.

[0064] like Figure 1 As shown, the key text information extraction method provided in this application embodiment may include the following steps:

[0065] Step S101: Extract the text heatmap of the target text region in the image to be identified.

[0066] The image to be identified refers to the image from which text information needs to be identified, such as a photograph of an ID card, a bank card, etc.

[0067] In this embodiment, a model for text heatmap extraction is pre-trained. This model uses an improved DBNet network, which adds a text region center point prediction branch to the existing DBNet network to predict the heatmap of the center points of the text region. The training labels are derived from the center point positions of the text region. During training, the improved DBNet network can be obtained through supervised learning using the cross-entropy loss of the heatmap of the center points of the text region and the loss of the number of center points of the text region as the overall model loss.

[0068] When extracting text heatmaps from images to be identified, an improved DBNet network can be used. This network incorporates supervised learning during training, taking into account the loss of the number of center points in text regions, to identify whether there are false positives or false negatives in text region detection, thus ensuring the accuracy of text region detection.

[0069] Understandably, in some other embodiments, the improved DBNet network may also disregard the loss of the number of center points in the text region when performing supervised learning.

[0070] Furthermore, to reduce the computational load during text recognition, the improved DBNet network in this embodiment can also adopt a lightweight backbone network. Specifically, this can be achieved by reducing the number of convolutional layers and channels, and then reparameterizing the convolutional layers after reducing their number of layers and channels. This allows for more parameters to accommodate more information, while fewer parameters are needed during training to speed up computation and reduce the computational load, making it suitable for mobile terminal devices with relatively low computing power.

[0071] It should be noted that there may be one or more text regions in the image to be recognized. Therefore, when extracting the text heatmap of the image, there may also be one or more text heatmaps. In the text heatmap, each pixel corresponds to a heat value, which is a floating-point number between 0 and 1, representing the probability that the pixel is located at the center point of the text region. The larger the heat value, the greater the probability that it is located at the center point of the text region.

[0072] Step S102: Perform edge detection on the text heatmap to obtain the coordinates of the text region.

[0073] In this embodiment, each text heatmap is first binarized to obtain a binarized text heatmap. Then, an edge detection algorithm is used to detect edges in each binarized text heatmap to obtain the edge contour of the corresponding text region. Based on the edge contour of each text region, the minimum bounding rectangle of each text region is determined. Finally, the coordinates of the four corner points of the minimum bounding rectangle of each text region are used as the text region coordinates. Assuming there are N text heatmaps, N sets of text region coordinates can be obtained, and each set of text region coordinates includes the coordinates of the four corner points.

[0074] In one or more embodiments, the extracted text heatmap may not be the same size as the image to be identified. To facilitate subsequent calculations, the text heatmap can be scaled according to the size ratio between the image to be identified and the text heatmap before edge detection, so that the size of the text heatmap is the same as the size of the image to be identified. Then, edge detection is performed on the scaled text heatmap to obtain the coordinates of the text region.

[0075] For example, in one embodiment, if the size of the text heatmap is 1 / 4 of the image to be identified, the length and width of the text heatmap can be enlarged by 2 times each, and then edge detection can be performed on the enlarged text heatmap to obtain the coordinates of the text region.

[0076] Step S103: Crop the first text region image located within the text region coordinates from the image to be recognized.

[0077] In this embodiment, if the extracted text heatmap is not the same size as the image to be recognized, and the text heatmap was not scaled before edge detection, the text region coordinates can be scaled according to the size ratio between the image to be recognized and the text heatmap before cropping. Then, the first text region image located within the scaled text region coordinates can be cropped from the image to be recognized. If the text heatmap has been scaled before edge detection, then scaling the text region coordinates is not required in this step.

[0078] Step S104: The second text region image located within the text region coordinates in the text heatmap is fused with the first text region image to obtain the third text region image.

[0079] When performing image feature fusion, the brightness values ​​of each pixel in each color channel in the first text region image can be combined with the thermal values ​​of the corresponding pixels in the second text region image to obtain the third text region image, whose feature parameters include thermal values ​​and brightness values ​​of each color channel.

[0080] For example, in one embodiment, if the first text region image is an RGB image, then the parameters of each pixel in the first text region image include the brightness value of the R color channel, the brightness value of the G color channel, and the brightness value of the B color channel. Assuming that the pixel at coordinates (1, 1) in the first text region image has a brightness value of 50 in the R color channel, 100 in the G color channel, and 150 in the B color channel, and the pixel at coordinates (1, 1) in the second text region image has a heat value of 0.5, then the brightness value of 50 in the R color channel, 100 in the G color channel, 150 in the B color channel, and the heat value of 0.5 can be combined as the feature parameters of the pixel at coordinates (1, 1). The combined feature parameters of the pixel at coordinates (1, 1) can be expressed as (50, 100, 150, 0.5), indicating that the pixel has a brightness value of 50 in the R color channel, 100 in the G color channel, 150 in the B color channel, and a heat value of 0.5.

[0081] Based on the same principle, the brightness values ​​of each pixel in the R, G, and B channels of the first text region image can be combined with the heat values ​​of the corresponding pixels in the second text region image to obtain a third text region image whose feature parameters include heat values ​​and brightness values ​​of each color channel (R, G, and B channels).

[0082] It is understood that the first text region image can also be an image in other color modes, such as CMYK, Lab color mode, etc., and no specific limitation is made in the embodiments of this application.

[0083] Step S105: Perform text recognition on the third text region image using a text recognition model to obtain key text information in the third text region image.

[0084] In this embodiment, a text recognition model for recognizing text information is pre-trained. During training, this model can be trained using an image with pixel feature parameters, including heatmap values ​​and brightness values ​​of each color channel, as input. After fusing the image features of the second text region image located within the text region coordinates in the text heatmap with the first text region image to obtain a third text region image, the text recognition model can be used to perform text recognition on the third text region image to obtain the key text information within it.

[0085] In summary, the key text information extraction method provided in this application extracts a text heatmap of the target text region in the image to be recognized, performs edge detection on the text heatmap to obtain the coordinates of the text region, then crops a first text region image located within the text region coordinates from the image to be recognized, and fuses the image features of a second text region image located within the text region coordinates in the text heatmap with the first text region image to obtain a third text region image. Finally, a text recognition model is used to perform text recognition on the third text region image to obtain the text information in the third text region image. Thus, in the process of recognizing text information, both the image features of the image to be recognized and the heatmap's heat value features are considered comprehensively, thereby enabling more accurate recognition of key text information in the image to be recognized and improving the accuracy of text information recognition. Simultaneously, by extracting the text heatmap of the image to be recognized through an improved DBNet network, supervised learning is also considered during training, taking into account the loss of the number of center points of the text region, to identify whether there are false positives or false negatives in the text region, thereby ensuring the accuracy of text region detection. Secondly, the improved DBNet network can also adopt a lightweight backbone network. By reducing the number of convolutional layers and channels, and then reparameterizing the convolutional layers after reducing the number of convolutional layers and channels, it can have more parameters to accommodate more information, while having fewer parameters during training. This can speed up the operation and reduce the amount of computation, making it suitable for some mobile terminal devices with relatively low computing power.

[0086] Figure 2 This is a schematic diagram of the structure of an electronic device provided in one embodiment of this application. Please refer to it. Figure 2 At the hardware level, the electronic device includes a processor, and optionally also includes an internal bus, a network interface, and memory. The memory may include main memory, such as high-speed random-access memory (RAM), or non-volatile memory, such as at least one disk drive. Of course, the electronic device may also include other hardware required for other business operations.

[0087] The processor, network interface, and memory can be interconnected via an internal bus, which can be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus, or an EISA (Extended Industry Standard Architecture) bus, etc. This bus can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 2 The symbol is represented by a single double-headed arrow, but this does not mean that there is only one bus or one type of bus.

[0088] Memory is used to store programs. Specifically, programs may include program code, which includes computer operation instructions. Memory may include main memory and non-volatile memory, and provides instructions and data to the processor.

[0089] The processor reads the corresponding computer program from non-volatile memory into main memory and then runs it, forming a key text information extraction device at the logical level. The processor executes the program stored in memory and specifically performs the following operations:

[0090] Extract the text heatmap of the target text region in the image to be identified;

[0091] Edge detection is performed on the text heatmap to obtain the coordinates of the text regions;

[0092] A first text region image located within the coordinates of the text region is cropped from the image to be identified;

[0093] The image features of the second text region image located within the coordinates of the text region in the text heatmap are fused with those of the first text region image to obtain the third text region image.

[0094] By performing text recognition on the third text region image using a text recognition model, key text information in the third text region image is obtained.

[0095] The above is as stated in this application. Figure 2The method executed by the key text information extraction device disclosed in the illustrated embodiment can be applied to a processor or implemented by a processor. The processor may be an integrated circuit chip with signal processing capabilities. During implementation, each step of the above method can be completed by integrated logic circuits in the processor's hardware or by instructions in software form. The processor can be a general-purpose processor, including a Central Processing Unit (CPU), a Network Processor (NP), etc.; it can also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field-Programmable Gate Array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in one or more embodiments of this application. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in one or more embodiments of this application can be directly embodied in the execution of a hardware decoding processor, or executed by a combination of hardware and software modules in the decoding processor. The software module can reside in a mature storage medium in the field, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, or registers. This storage medium is located in memory, and the processor reads information from the memory and, in conjunction with its hardware, completes the steps of the above method.

[0096] The electronic device can also perform Figure 1 The method, and the realization of the key text information extraction device in Figure 2 The functions of the embodiments shown are not described again in this application.

[0097] Of course, in addition to software implementation, the electronic device of this application does not exclude other implementation methods, such as logic devices or a combination of hardware and software, etc. In other words, the execution subject of the following processing flow is not limited to each logic unit, but can also be hardware or logic devices.

[0098] This application also proposes a computer-readable storage medium that stores one or more programs, the programs including instructions that, when executed by a portable electronic device including multiple applications, enable the portable electronic device to perform... Figure 1 The method of the illustrated embodiment is specifically used to perform the following operations:

[0099] Extract the text heatmap of the target text region in the image to be identified;

[0100] Edge detection is performed on the text heatmap to obtain the coordinates of the text regions;

[0101] A first text region image located within the coordinates of the text region is cropped from the image to be identified;

[0102] The image features of the second text region image located within the coordinates of the text region in the text heatmap are fused with those of the first text region image to obtain the third text region image.

[0103] By performing text recognition on the third text region image using a text recognition model, key text information in the third text region image is obtained.

[0104] Figure 3 This is a schematic diagram of the structure of a key text information extraction device provided in one embodiment of this application. Please refer to... Figure 3 In one software implementation, the key text information extraction device includes:

[0105] The extraction unit is used to extract the text heatmap of the target text region in the image to be recognized;

[0106] An edge detection unit is used to perform edge detection on the text heatmap to obtain the coordinates of the text region.

[0107] A cropping unit is used to crop out a first text region image located within the coordinates of the text region from the image to be recognized;

[0108] The feature fusion unit is used to fuse the image features of the second text region image located within the coordinates of the text region in the text heatmap with the first text region image to obtain a third text region image.

[0109] The text recognition unit is used to perform text recognition on the third text region image through a text recognition model to obtain key text information in the third text region image.

[0110] In summary, the above description is merely a preferred embodiment of this document and is not intended to limit the scope of protection of this document. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this document should be included within the scope of protection of this document.

[0111] The systems, devices, modules, or units described in the above embodiments can be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, a computer can be, for example, a personal computer, laptop computer, cellular phone, camera phone, smartphone, personal digital assistant, media player, navigation device, email device, game console, tablet computer, wearable device, or any combination of these devices.

[0112] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0113] The various embodiments in this document are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the system embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments.

Claims

1. A method for extracting key text information, characterized in that, include: Extract the text heatmap of the target text region in the image to be identified; Edge detection is performed on the text heatmap to obtain the coordinates of the text regions; A first text region image located within the coordinates of the text region is cropped from the image to be identified; The image features of the second text region image located within the coordinates of the text region in the text heatmap are fused with those of the first text region image to obtain the third text region image. The text recognition model is used to perform text recognition on the third text region image to obtain the key text information in the third text region image; The step of extracting the text heatmap of the target text region in the image to be identified includes: The text heatmap of the target text region in the image to be identified is extracted using an improved DBNet network; The improved DBNet network includes a text region center point prediction branch, which is used to predict the center point heatmap of the text region. The improved DBNet network is obtained through supervised learning using the cross-entropy loss of the heatmap of the center points of the text region and the loss of the number of center points of the text region as the overall loss of the model.

2. The method according to claim 1, characterized in that, The step of fusing image features of the second text region image located within the text region coordinates in the text heatmap with the first text region image to obtain a third text region image includes: The brightness values ​​of each pixel in each color channel in the first text region image are combined with the thermal values ​​of the corresponding pixels in the second text region image to obtain the third text region image, whose feature parameters include thermal values ​​and brightness values ​​of each color channel.

3. The method according to claim 1, characterized in that, The improved DBNet network uses a lightweight backbone network.

4. The method according to claim 1, characterized in that, The step of performing edge detection on the text heatmap to obtain text region coordinates includes: The text heatmap is binarized; The edge detection algorithm is used to perform edge detection on the binarized text heatmap to obtain the edge contour of the text region. Based on the edge contour of the text region, determine the minimum bounding rectangle of the text region; The coordinates of the text region are obtained based on the smallest bounding rectangle of the text region.

5. The method according to claim 1, characterized in that, The method further includes: The text heatmap is scaled according to the size ratio between the image to be identified and the text heatmap; The step of performing edge detection on the text heatmap to obtain text region coordinates includes: Edge detection is performed on the scaled text heatmap to obtain the coordinates of the text region.

6. The method according to claim 1, characterized in that, The method further includes: The coordinates of the text region are scaled according to the size ratio of the image to be identified and the text heatmap; The step of cropping the first text region image located within the coordinates of the text region from the image to be recognized includes: A first text region image is cropped from the image to be identified and located within the scaled coordinates of the text region.

7. A key text information extraction device, characterized in that, include: The extraction unit is used to extract the text heatmap of the target text region in the image to be recognized; An edge detection unit is used to perform edge detection on the text heatmap to obtain the coordinates of the text region. A cropping unit is used to crop out a first text region image located within the coordinates of the text region from the image to be recognized; The feature fusion unit is used to fuse the image features of the second text region image located within the coordinates of the text region in the text heatmap with the first text region image to obtain a third text region image. The text recognition unit is used to perform text recognition on the third text region image through a text recognition model to obtain key text information in the third text region image. The extraction unit, when used to extract the text heatmap of the target text region in the image to be recognized, is specifically used for: The text heatmap of the target text region in the image to be identified is extracted using an improved DBNet network; The improved DBNet network includes a text region center point prediction branch, which is used to predict the center point heatmap of the text region. The improved DBNet network is obtained through supervised learning using the cross-entropy loss of the heatmap of the center points of the text region and the loss of the number of center points of the text region as the overall loss of the model.

8. An electronic device, characterized in that, It includes a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the bus; Memory, used to store computer programs; The processor executes programs stored in memory, implementing the following process: Extract the text heatmap of the target text region in the image to be identified; Edge detection is performed on the text heatmap to obtain the coordinates of the text regions; A first text region image located within the coordinates of the text region is cropped from the image to be identified; The image features of the second text region image located within the coordinates of the text region in the text heatmap are fused with those of the first text region image to obtain the third text region image. The text recognition model is used to perform text recognition on the third text region image to obtain the key text information in the third text region image; The step of extracting the text heatmap of the target text region in the image to be identified includes: The text heatmap of the target text region in the image to be identified is extracted using an improved DBNet network; The improved DBNet network includes a text region center point prediction branch, which is used to predict the center point heatmap of the text region. The improved DBNet network is obtained through supervised learning using the cross-entropy loss of the heatmap of the center points of the text region and the loss of the number of center points of the text region as the overall loss of the model.

9. A computer-readable storage medium, characterized in that, The storage medium stores a computer program, which, when executed by a processor, performs the following process: Extract the text heatmap of the target text region in the image to be identified; Edge detection is performed on the text heatmap to obtain the coordinates of the text regions; A first text region image located within the coordinates of the text region is cropped from the image to be identified; The image features of the second text region image located within the coordinates of the text region in the text heatmap are fused with those of the first text region image to obtain the third text region image. The text recognition model is used to perform text recognition on the third text region image to obtain the key text information in the third text region image; The step of extracting the text heatmap of the target text region in the image to be identified includes: The text heatmap of the target text region in the image to be identified is extracted using an improved DBNet network; The improved DBNet network includes a text region center point prediction branch, which is used to predict the center point heatmap of the text region. The improved DBNet network is obtained through supervised learning using the cross-entropy loss of the heatmap of the center points of the text region and the loss of the number of center points of the text region as the overall loss of the model.

Images